Skip to content

Quickstart

Build one model-aware service through the complete beginner path: install, readiness, management authentication, rule creation, traffic, metrics, and cleanup. The configuration and traffic steps use the public ai-model-routing scenario; operational steps are checked against the frozen Gateway Swagger union and CLI contract.

Choose a compatible version pair

The readiness route is implemented on current Gateway main, not Gateway v0.9.8.9-rc.1. loxicmd get ready is present in CLI v0.9.8.9-rc.2, but it still needs a compatible Gateway main build. If you deploy the Gateway release, use GET /version as a process-liveness probe and do not treat it as configuration readiness.

What you will build

  • Management API: https://gateway.example.com/netlox/v1
  • Data-plane VIP: 192.0.2.10:2020
  • Model: llama-70b
  • Backend: 198.51.100.11:8080
  • Rule mode: fullproxy (mode: 4)
  • Credential policy: omitted for this isolated data-plane lab

The addresses are documentation placeholders. Replace them with your own reachable addresses before running the commands.

flowchart LR
    O[Operator] -->|Bearer token| C[Management API]
    U[Inference client] -->|model llama-70b| V[VIP 192.0.2.10:2020]
    C --> V
    V --> B[Backend 198.51.100.11:8080]
    P[Prometheus client] -->|GET /metrics| C

1. Install

Example status: verified for documentation syntax and the published container/startup contract; Linux data-plane execution is not performed by the documentation checks.

Prerequisites

  • A Linux host with Docker, a modern eBPF-capable kernel, and permission to run the privileged Gateway container.
  • A version-pinned public Gateway image or a build from the exact source commit you intend to test.
  • A persistent host directory for /etc/loxilb.

Exact command

Complete the Installation page's container startup procedure. Then record the running image identity:

docker inspect --format '{{.Image}}' loxilb > gateway-image.id
test -s gateway-image.id

Expected result

The command exits 0 and gateway-image.id contains one nonempty image ID.

Validate

docker ps --filter name='^/loxilb$' --filter status=running --format '{{.Names}}' \
  | grep -qx loxilb
curl --fail-with-body --silent --show-error \
  http://127.0.0.1:11111/netlox/v1/version > gateway-version.json
jq -e 'type == "object"' gateway-version.json

Cleanup

Do not remove the Gateway yet; the remaining steps use it. At the end, remove only the container and persistent directory that you created for this lab.

Diagnose

Symptom Check
Container is absent Inspect docker ps -a and the container log.
/version is refused Confirm host networking or the published management port.
eBPF initialization fails Recheck kernel support, privileges, and required mounts.

2. Check readiness

Example status: verified against current-main Swagger, CLI v0.9.8.9-rc.2, and the frozen readiness contract. This is source/static evidence, not a live Linux result.

Prerequisites

  • A current-main Gateway build that exposes GET /status/ready.
  • CLI v0.9.8.9-rc.2 or later if you use the CLI form.
  • A management token in CONTROL_PLANE_TOKEN and a trusted TLS endpoint.

Prepare protected credential files once:

export CONTROL_API='https://gateway.example.com/netlox/v1'
install -m 600 /dev/null ./gateway.token ./control-plane.headers
printf '%s\n' "$CONTROL_PLANE_TOKEN" > ./gateway.token
printf 'Authorization: Bearer %s\n' "$CONTROL_PLANE_TOKEN" > ./control-plane.headers

Exact command

ready_status=$(curl --silent --show-error \
  --header @control-plane.headers \
  --output ready.json \
  --write-out '%{http_code}' \
  "$CONTROL_API/status/ready")
printf '%s\n' "$ready_status"

loxicmd get ready --token-file ./gateway.token --output json > ready.json

Expected result

Ready returns HTTP 200; not ready returns 503. Both responses carry the typed body. The CLI exits nonzero for not-ready while preserving that body.

Validate

test "${ready_status:-200}" = 200
jq -e '.ready == true and (.reasons | length == 0)' ready.json

Cleanup

Keep the protected credential files for the next steps. Remove ready.json if it contains deployment details you do not want to retain.

Diagnose

Symptom Check
HTTP 404 The Gateway is the release build; /status/ready is current-main only.
HTTP 503 Read reasons[], boot replay, external dependency, and persist state.
HTTP 401 Recheck the management token file and the configured auth mode.

3. Verify management authentication

Example status: verified against the management-auth contract. The negative request is mandatory: an authenticated success alone cannot prove the listener is protected.

Prerequisites

  • One supported management authentication mode is enabled.
  • control-plane.headers contains the valid bearer token from the readiness step.

Exact command

unauthenticated_status=$(curl --silent --show-error \
  --output unauthenticated.json \
  --write-out '%{http_code}' \
  "$CONTROL_API/config/loadbalancer/all")

authenticated_status=$(curl --silent --show-error \
  --header @control-plane.headers \
  --output authenticated.json \
  --write-out '%{http_code}' \
  "$CONTROL_API/config/loadbalancer/all")

Expected result

The unauthenticated request returns 401; the authenticated request returns 200. Do not continue if both return 200.

Validate

test "$unauthenticated_status" = 401
test "$authenticated_status" = 200
jq -e 'type == "object" or type == "array"' authenticated.json

Cleanup

Keep the credential files until final cleanup. Remove the two response files after reviewing only sanitized fields.

Diagnose

Symptom Check
Unauthenticated request returns 200 No management authenticator is active; do not expose the listener.
Authenticated request returns 401 Token, issuer, expiry, or header-file contents are wrong.
TLS verification fails Install the correct CA; do not make insecure TLS the permanent fix.

Management and inference credentials are separate

The bearer token above protects configuration on port 11111. It is not an inference API key or JWT. This lab deliberately omits api_key_auth, so a backend-owned X-Api-Key passes through unchanged. Follow Data-Plane Authentication and JWT before exposing a protected inference VIP.

4. Create one model rule

Example status: verified against the Swagger request schema, CLI flags, and the public model-routing scenario. Run either the REST tab or the CLI tab, not both.

Prerequisites

  • The VIP is assigned to the Gateway node.
  • 198.51.100.11:8080 is an HTTP backend that returns the marker backend-llama.
  • The management authentication negative and positive checks passed.
  • This is an isolated lab with no concurrent configuration writer or traffic generator; exact state and metric deltas depend on that isolation.

Exact command

create_status=$(curl --silent --show-error \
  --request POST \
  --header @control-plane.headers \
  --header 'Content-Type: application/json' \
  --data '{
    "serviceArguments": {
      "externalIP": "192.0.2.10",
      "port": 2020,
      "protocol": "tcp",
      "sel": 0,
      "mode": 4,
      "host": "192.0.2.10",
      "path_prefix": "/",
      "path_match_mode": "prefix",
      "model_name": "llama-70b",
      "inactiveTimeOut": 30
    },
    "endpoints": [
      {"endpointIP": "198.51.100.11", "targetPort": 8080, "weight": 1}
    ]
  }' \
  --output create.json \
  --write-out '%{http_code}' \
  "$CONTROL_API/config/loadbalancer")

loxicmd create lb 192.0.2.10 \
  --tcp=2020:8080 \
  --endpoints=198.51.100.11:1 \
  --mode=fullproxy \
  --host=192.0.2.10 \
  --path-prefix=/ \
  --path-match-mode=prefix \
  --model-name=llama-70b \
  --inatimeout=30 \
  --token-file ./gateway.token

Expected result

The REST form returns HTTP 200; the CLI exits 0. A duplicate rule or an invalid body returns a non-success status and must not be counted as created.

Validate

test "${create_status:-200}" = 200
curl --fail-with-body --silent --show-error \
  --header @control-plane.headers \
  "$CONTROL_API/config/loadbalancer/all" > rules.json
jq -e '
  .lbAttr[] |
  select(.serviceArguments.externalIP == "192.0.2.10") |
  select(.serviceArguments.port == 2020) |
  .serviceArguments.model_name == "llama-70b" and
  .serviceArguments.mode == 4
' rules.json

Also prove a rejected mutation leaves the complete readback unchanged:

# docs-example: expect-schema-error
jq -S . rules.json > rules-before-invalid.json
invalid_status=$(curl --silent --show-error \
  --request POST \
  --header @control-plane.headers \
  --header 'Content-Type: application/json' \
  --data '{
    "serviceArguments": {
      "externalIP": "192.0.2.10",
      "port": 2020,
      "protocol": "not-a-protocol",
      "sel": 0,
      "mode": 4
    },
    "endpoints": [
      {"endpointIP": "198.51.100.11", "targetPort": 8080, "weight": 1}
    ]
  }' \
  --output invalid-create.json \
  --write-out '%{http_code}' \
  "$CONTROL_API/config/loadbalancer")
test "$invalid_status" = 400
curl --fail-with-body --silent --show-error \
  --header @control-plane.headers \
  "$CONTROL_API/config/loadbalancer/all" \
  | jq -S . > rules-after-invalid.json
cmp --silent rules-before-invalid.json rules-after-invalid.json

Cleanup

If validation fails after creation, run the exact delete in step 7 before changing any key field.

Diagnose

Symptom Check
HTTP 400 on the valid create Validate JSON field casing and required endpoint fields.
HTTP 409 An equivalent rule already exists; inspect before deleting it.
Rule reads back without the model Recheck exact model_name spelling and the complete L7 key.
Invalid create changes readback Stop and restore the pre-change snapshot; rejection was not atomic.

5. Send traffic

Example status: verified against the public model-routing scenario. The backend marker is the independent delivery oracle; HTTP 200 without the expected marker is not a pass.

Prerequisites

  • The rule readback passed.
  • The backend is healthy, returns backend-llama, and appends each received X-Docs-Nonce value to a line-oriented log available to the operator.
  • Set BACKEND_RECEIPT_LOG to that backend-owned log. It must not be produced from Gateway or client output.

Exact command

export BACKEND_RECEIPT_LOG='/var/tmp/docs-backend-receipts.log'
test -r "$BACKEND_RECEIPT_LOG"
receipt_count() {
  grep -cF -- "$1" "$BACKEND_RECEIPT_LOG" || true
}

curl --fail-with-body --silent --show-error \
  "$CONTROL_API/metrics" > metrics-before.prom

positive_nonce="docs-positive-$(date +%s)-$$"
positive_receipts_before=$(receipt_count "$positive_nonce")
traffic_status=$(curl --silent --show-error \
  --request POST \
  --header 'Content-Type: application/json' \
  --header "X-Docs-Nonce: $positive_nonce" \
  --data '{"model":"llama-70b","messages":[{"role":"user","content":"hello"}]}' \
  --output inference.json \
  --write-out '%{http_code}' \
  http://192.0.2.10:2020/v1/chat/completions)

Expected result

The request returns HTTP 200, and the backend-specific marker appears in the response. The exact OpenAI response shape belongs to the backend.

Validate

test "$traffic_status" = 200
grep -q 'backend-llama' inference.json
positive_receipts_after=$(receipt_count "$positive_nonce")
test "$((positive_receipts_after - positive_receipts_before))" -eq 1

Also prove a wrong model does not reach this backend:

wrong_model_nonce="docs-negative-$(date +%s)-$$"
negative_receipts_before=$(receipt_count "$wrong_model_nonce")
wrong_model_status=$(curl --silent --show-error \
  --request POST \
  --header 'Content-Type: application/json' \
  --header "X-Docs-Nonce: $wrong_model_nonce" \
  --data '{"model":"unknown-model","messages":[]}' \
  --output wrong-model.json \
  --write-out '%{http_code}' \
  http://192.0.2.10:2020/v1/chat/completions)
test "$wrong_model_status" = 503
! grep -q 'backend-llama' wrong-model.json
negative_receipts_after=$(receipt_count "$wrong_model_nonce")
test "$negative_receipts_after" -eq "$negative_receipts_before"

Cleanup

Keep the rule until after the metrics check. Remove response files when they are no longer needed.

Diagnose

Symptom Check
HTTP 503 model_unavailable Model matching is case-sensitive; confirm llama-70b.
HTTP 200 without the marker or receipt delta The request did not prove the expected active backend path.
Connection refused Confirm the VIP, fullproxy listener, and backend reachability.
Wrong-model receipt count increases Stop; the rejected request reached the backend.

6. Inspect metrics

Example status: verified against the frozen release-scope metric manifest and current writer mapping. The documentation check validates the name and labels; it does not claim a live scrape occurred on your host.

Prerequisites

  • Metrics export is enabled and GET /metrics is reachable.
  • metrics-before.prom was captured immediately before the single positive request on an otherwise idle, isolated Gateway.

Exact command

metrics_status=$(curl --silent --show-error \
  --output metrics-after.prom \
  --write-out '%{http_code}' \
  "$CONTROL_API/metrics")

Expected result

The scrape returns HTTP 200. The sum of loxilb_service_requests_total across its documented service label grows by exactly one for the one positive request. The rejected wrong-model request must not add to that service-delivery counter.

Validate

test "$metrics_status" = 200
grep -E '^loxilb_service_requests_total\{service="[^"]+"\} [0-9]+' metrics-after.prom
service_request_total() {
  awk '$1 ~ /^loxilb_service_requests_total\{service=/ {sum += $2}
       END {printf "%.0f\n", sum + 0}' "$1"
}
requests_before=$(service_request_total metrics-before.prom)
requests_after=$(service_request_total metrics-after.prom)
test "$((requests_after - requests_before))" -eq 1

Cleanup

Remove both metric snapshots after retaining only the sanitized evidence required by your deployment process.

Diagnose

Symptom Check
HTTP 404 Confirm the API base path and the running Gateway build.
HTTP 200 but no series Confirm export activation and that traffic crossed the active service path.
Delta is not exactly one Stop other traffic, repeat from a clean isolated rule, and compare backend receipts.
Unexpected labels Compare the query with the frozen metric reference.

7. Clean up

Example status: verified against the exact model-keyed REST and CLI delete contracts. Run the cleanup form that matches the create form you chose.

Prerequisites

  • Use the same VIP, port, host, path, path mode, and model name used at create.
  • Confirm that the matching rule belongs to this lab.

Exact command

cleanup_status=$(curl --silent --show-error \
  --request DELETE \
  --header @control-plane.headers \
  --output cleanup.json \
  --write-out '%{http_code}' \
  "$CONTROL_API/config/loadbalancer/hosturl/192.0.2.10/externalipaddress/192.0.2.10/port/2020/protocol/tcp?path_prefix=%2F&path_match_mode=prefix&model_name=llama-70b")

loxicmd delete lb 192.0.2.10 \
  --tcp=2020 \
  --host=192.0.2.10 \
  --path-prefix=/ \
  --path-match-mode=prefix \
  --model-name=llama-70b \
  --token-file ./gateway.token

Expected result

The REST form returns 200; the CLI exits 0.

Validate

test "${cleanup_status:-200}" = 200
curl --fail-with-body --silent --show-error \
  --header @control-plane.headers \
  "$CONTROL_API/config/loadbalancer/all" > rules-after-cleanup.json
! jq -e '
  .lbAttr[] |
  select(.serviceArguments.externalIP == "192.0.2.10") |
  select(.serviceArguments.port == 2020) |
  select(.serviceArguments.model_name == "llama-70b")
' rules-after-cleanup.json
curl --fail-with-body --silent --show-error \
  --header @control-plane.headers \
  "$CONTROL_API/status/ready" > ready-after-cleanup.json
jq -e '.ready == true and (.reasons | length == 0)' ready-after-cleanup.json

Cleanup

Remove local credential and response files:

rm -f ./gateway.token ./control-plane.headers ready.json \
  unauthenticated.json authenticated.json create.json rules.json \
  invalid-create.json rules-before-invalid.json rules-after-invalid.json \
  inference.json wrong-model.json metrics-before.prom metrics-after.prom cleanup.json \
  rules-after-cleanup.json ready-after-cleanup.json gateway-version.json gateway-image.id
unset CONTROL_PLANE_TOKEN CONTROL_API BACKEND_RECEIPT_LOG

Diagnose

Symptom Check
HTTP 404 One rule-key component differs from create; inspect the exact readback.
Rule remains Repeat the complete model-keyed delete; do not use delete-all on a shared Gateway.
Other rules disappeared Stop and restore from the pre-change snapshot; cleanup scope was too broad.
Readiness fails after cleanup Inspect reasons[]; cleanup is not complete until readiness recovers.

REST-only surfaces

Do not invent CLI flags for surfaces the frozen CLI does not implement:

Surface Supported path
JWT profile CRUD and jwt / apikey-or-jwt rule binding REST-only
Global/rule-default and user/user-model QoS CRUD REST-only
Model-profile discovery and kvexactstatus REST-only
kvModelProfile and kvExactApiMode REST-only
Capability readiness and sockmapreset REST-only

The CLI reference carries the canonical compatibility matrix.

Next steps