Share defaults and service budgets¶

Apply policy fallbacks once, override selected fields for a service, and distinguish keyless RPS from shared token accounting.

Lab sequence and validation scope

Start with lab preparation and common API conventions. These commands belong to a multi-page lab, not standalone mutations. The Quickstart provides the canonical mutation lifecycle. Read the evidence limits before recording a result.

Before you start¶

Finish quotas and save every default and service-budget scope that will be changed.

Focus of this page: Compare inherited defaults with explicit scope values and a listener token budget.

Topology and execution locations¶

flowchart TD
    C["Client namespace l3h1<br/>192.0.2.1"] --> G["Gateway llb1<br/>VIP 192.0.2.254"]
    G --> L["Llama echo l3ep1<br/>198.51.100.1"]
    G --> M["Mistral echo l3ep2<br/>203.0.113.1"]
    K["Keycloak kc-aigw<br/>Docker bridge"] -.->|"JWT issuer / JWKS"| G
    G -.-> P["Policy store pg-jwtauth<br/>Docker bridge"]
    O["Operator on Linux host"] -.->|"REST via l3h1 / port 11111"| G
    O -.->|"independent fixture observations"| L
    style G fill:#e1f5fe,stroke:#0288d1,color:#1f2937
    style L fill:#e8f5e9,stroke:#43a047,color:#1f2937
    style M fill:#e8f5e9,stroke:#43a047,color:#1f2937

These are Linux containers/namespaces on one disposable host. The model names identify synthetic echoes, not loaded GPU models. The baseline listener is VIP 192.0.2.254:2040; TLS, capacity and failure exercises add the explicit listener ports shown in their steps. Keycloak/JWKS and PostgreSQL use inspected Docker-bridge addresses.

Where	What happens here
Host operator	Prepare private LAB files, issue credentials and run namespace commands.
l3h1	Send management calls and client traffic into llb1.
llb1	Enforce identity, quota and capacity before choosing a backend.
l3ep1 / l3ep2	Observe independent Llama/Mistral receipt and hold/release fixtures.
kc-aigw / pg-jwtauth	Supply identity and policy state; they are not on the inference response path.

Walkthrough¶

Prerequisites, configuration and expected precedence¶

Use q4/q7/t1 credentials and bootstrap ports 2059–2062. Save every pre-existing scope row before changing it. Each positive field resolves explicit → rule → global → no limit; zero falls through. POST is a scope-row upsert: include all fields to preserve. An all-zero defaults row is rejected; use DELETE.

Defaults input	Required / example / unit	Meaning and impact
`scope`	Required global or rule	Select the policy scope
`rule_ident`	Required for rule, `192.0.2.254:2059`	VIP:port; not the LB deletion full key
`default_user_rps`, `default_tenant_rps`	Optional positive requests/second	Fallback for the named identity scope
`default_user_tpm`, `default_tenant_tpm`	Optional positive tokens/minute	Per-field fallback token budget
`vip_shared_rps`	Optional 100 requests/second here	Keyless traffic on the service shares this RPS; not authenticated-tenant aggregate RPS
`vip_shared_tpm`	Optional 10 tokens/minute here	Shared actual usage from credentialed/keyless token-metered responses; keyless has no pre-reservation

After POST/GET, send three quick q7 requests to 2059: rule user RPS9 wins over global RPS1. Send q4 twice to 2060: its zero user field falls through to global RPS1, so the second is denied. On 2061 q4 usage12 spends shared TPM10; immediately send t1 to the same port and expect shared-budget 429/receipt0, then after two seconds send t1 to 2040 and require 200. Avoid attributing a user-RPS refusal to shared TPM. Check typed errors and vip token quota metrics.

The blocks include scoped DELETE and GET404, global restoration, then VIP defaults delete→two-tenant recovery→restore. Only POST a saved JSON body when its original GET was 200; never replay a 404 error body. Preserve bootstrap-owned VIP defaults after the experiment.

sudo ip netns exec l3h1 curl -sS -i "$API/config/ai/ratelimit/defaults/global" > "$LAB/results/defaults-before.txt"
sudo ip netns exec l3h1 curl -sS -i -X POST "$API/config/ai/ratelimit/defaults" \
  -H 'Content-Type: application/json' -d '{"scope":"global","default_user_rps":1}'
sudo ip netns exec l3h1 curl -sS -i "$API/config/ai/ratelimit/defaults/global"
sudo ip netns exec l3h1 curl -sS -i -X POST "$API/config/ai/ratelimit/defaults" \
  -H 'Content-Type: application/json' \
  -d '{"scope":"rule","rule_ident":"192.0.2.254:2059","default_user_rps":9}'
sudo ip netns exec l3h1 curl -sS -i "$API/config/ai/ratelimit/defaults/rule?rule_ident=192.0.2.254:2059"
sudo ip netns exec l3h1 curl -sS -i -X POST "$API/config/ai/ratelimit/defaults" \
  -H 'Content-Type: application/json' \
  -d '{"scope":"rule","rule_ident":"192.0.2.254:2060","default_tenant_tpm":100000}'
sudo ip netns exec l3h1 curl -sS -i "$API/config/ai/ratelimit/defaults/rule?rule_ident=192.0.2.254:2060"
sudo ip netns exec l3h1 curl -sS \
  "$API/config/ai/ratelimit/defaults/rule?rule_ident=192.0.2.254:2060" | jq .

Verify before removing defaults¶

Run the pairs together so refill does not obscure the intended refusal.

sudo ip netns exec l3h1 curl -sS -i --max-time 15 --config "$LAB/q7.curl" \
  -H 'Content-Type: application/json' -H "X-Test-Nonce: $RUN-rule-1" \
  --data-binary @"$LAB/examples/llama-request.json" "http://$VIP:2059/v1/chat/completions"
sudo ip netns exec l3h1 curl -sS -i --max-time 15 --config "$LAB/q7.curl" \
  -H 'Content-Type: application/json' -H "X-Test-Nonce: $RUN-rule-2" \
  --data-binary @"$LAB/examples/llama-request.json" "http://$VIP:2059/v1/chat/completions"
sudo ip netns exec l3h1 curl -sS -i --max-time 15 --config "$LAB/q7.curl" \
  -H 'Content-Type: application/json' -H "X-Test-Nonce: $RUN-rule-3" \
  --data-binary @"$LAB/examples/llama-request.json" "http://$VIP:2059/v1/chat/completions"
sudo ip netns exec l3h1 curl -sS -i --max-time 15 --config "$LAB/q4.curl" \
  -H 'Content-Type: application/json' -H "X-Test-Nonce: $RUN-global-1" \
  --data-binary @"$LAB/examples/llama-request.json" "http://$VIP:2060/v1/chat/completions"
sudo ip netns exec l3h1 curl -sS -i --max-time 15 --config "$LAB/q4.curl" \
  -H 'Content-Type: application/json' -H "X-Test-Nonce: $RUN-global-2" \
  --data-binary @"$LAB/examples/llama-request.json" "http://$VIP:2060/v1/chat/completions"
sleep 2
sudo ip netns exec l3h1 curl -sS -i --max-time 15 --config "$LAB/q4.curl" \
  -H 'Content-Type: application/json' -H "X-Test-Nonce: $RUN-vip-spend" \
  --data-binary @"$LAB/examples/llama-request.json" "http://$VIP:2061/v1/chat/completions"
sudo ip netns exec l3h1 curl -sS -i --max-time 15 --config "$LAB/t1.curl" \
  -H 'Content-Type: application/json' -H "X-Test-Nonce: $RUN-vip-denied" \
  --data-binary @"$LAB/examples/llama-request.json" "http://$VIP:2061/v1/chat/completions"
sleep 2
sudo ip netns exec l3h1 curl -sS -i --max-time 15 --config "$LAB/t1.curl" \
  -H 'Content-Type: application/json' -H "X-Test-Nonce: $RUN-vip-control" \
  --data-binary @"$LAB/examples/llama-request.json" "http://$VIP:2040/v1/chat/completions"

sudo ip netns exec l3h1 curl -sS "$API/metrics" | grep 'loxilb_ai_vip_token_quota'
sudo ip netns exec l3h1 curl -sS -i -X DELETE \
  "$API/config/ai/ratelimit/defaults/rule?rule_ident=192.0.2.254:2059"
sudo ip netns exec l3h1 curl -sS -i -X DELETE \
  "$API/config/ai/ratelimit/defaults/rule?rule_ident=192.0.2.254:2060"

Cleanup when the original global GET was 404¶

Run this branch only if the row did not exist.

sudo ip netns exec l3h1 curl -sS -i -X DELETE "$API/config/ai/ratelimit/defaults/global"
sudo ip netns exec l3h1 curl -sS -i "$API/config/ai/ratelimit/defaults/global"

Restore when the original global GET was 200¶

Run this alternative branch only for a saved successful response.

python3 -c 'import pathlib,sys;d=pathlib.Path(sys.argv[1]).read_bytes();pathlib.Path(sys.argv[2]).write_bytes(d.split(b"\r\n\r\n",1)[1])' "$LAB/results/defaults-before.txt" "$LAB/defaults-restore.json"
sudo ip netns exec l3h1 curl -sS -i -X POST "$API/config/ai/ratelimit/defaults" \
  -H 'Content-Type: application/json' --data-binary @"$LAB/defaults-restore.json"

sudo ip netns exec l3h1 curl -sS -i "$API/config/ai/ratelimit/defaults/global"
sudo ip netns exec l3h1 curl -sS -i "$API/config/ai/ratelimit/defaults/rule?rule_ident=192.0.2.254:2059"
sudo ip netns exec l3h1 curl -sS -i "$API/config/ai/ratelimit/defaults/rule?rule_ident=192.0.2.254:2060"
sudo ip netns exec l3h1 curl -sS -i "$API/config/ai/ratelimit/defaults/rule?rule_ident=192.0.2.254:2061"

sudo ip netns exec l3h1 curl -sS "$API/config/ai/ratelimit/defaults/rule?rule_ident=192.0.2.254:2061" > "$LAB/vip-default-before.json"
jq . "$LAB/vip-default-before.json"
sudo ip netns exec l3h1 curl -sS -i -X DELETE "$API/config/ai/ratelimit/defaults/rule?rule_ident=192.0.2.254:2061"
sudo ip netns exec l3h1 curl -sS -i "$API/config/ai/ratelimit/defaults/rule?rule_ident=192.0.2.254:2061"
sudo ip netns exec l3h1 curl -sS -i --config "$LAB/q4.curl" -H 'Content-Type: application/json' \
  -H "X-Test-Nonce: $RUN-vip-deleted-q4" --data-binary @"$LAB/examples/llama-request.json" "http://$VIP:2061/v1/chat/completions"
sudo ip netns exec l3h1 curl -sS -i --config "$LAB/t1.curl" -H 'Content-Type: application/json' \
  -H "X-Test-Nonce: $RUN-vip-deleted-t1" --data-binary @"$LAB/examples/llama-request.json" "http://$VIP:2061/v1/chat/completions"
sudo ip netns exec l3h1 curl -sS -i -X POST "$API/config/ai/ratelimit/defaults" \
  -H 'Content-Type: application/json' --data-binary @"$LAB/vip-default-before.json"
sudo ip netns exec l3h1 curl -sS "$API/config/ai/ratelimit/defaults/rule?rule_ident=192.0.2.254:2061" | jq .

Verification and cleanup¶

After each scope change read the exact scope path. A 200 from POST is not traffic proof. Require appropriate per-nonce receipts (1 for delivered, 0 for denied), isolated metric deltas, default GET absence or exact restored values, and the same users' normal traffic after cleanup. See common rollback semantics.

Troubleshooting and cleanup¶

Use diagnosis and scoped cleanup. Restore this chapter's changes before moving to another policy experiment.

Check your result¶

Read both the configured limit and the observed admission decision; restore complete original requests.

Compare API readback with the normal request, the deliberate denial, and an independent backend observation. Check the chapter's exact HTTP code and error body; HTTP 200 alone does not prove quota, identity, queue order, or resource return.

Next steps and cleanup¶

Restore the chapter's temporary policy before continuing in the same lab. Keep the original configuration and private evidence; use the final cleanup chapter when the entire lab is finished. Read the lab starting point if required values are missing. Use cleanup and recovery when ending the run.