Quickstart¶

Build one model-aware service through the complete beginner path: install, readiness, management authentication, rule creation, traffic, metrics, and cleanup. The configuration and traffic steps use the public ai-model-routing scenario; operational steps are checked against the frozen Gateway Swagger union and CLI contract.

The selected development source pair is recorded in Verification Status. Historical model results retain their own recorded revisions. Do not mix their evidence with this Quickstart's source pair. Build and run the Gateway and CLI on Linux; from a macOS workstation use a Linux host for builds and executable checks.

Choose a compatible version pair

The readiness route is implemented on current Gateway main, not Gateway v0.9.8.9-rc.1. loxicmd get ready is present in CLI v0.9.8.9-rc.2, but it still needs a compatible Gateway main build. If you deploy the Gateway release, use GET /version as a process-liveness probe and do not treat it as configuration readiness.

What you will build¶

Management API: https://gateway.example.com/netlox/v1
Data-plane VIP: 192.0.2.10:2020
Model: llama-70b
Backend: 198.51.100.11:8080
Rule mode: fullproxy (mode: 4)
Credential policy: omitted for this isolated data-plane lab

The addresses are documentation placeholders. Replace them with your own reachable addresses before running the commands.

flowchart LR
    O[Operator] -->|Bearer token| C[Management API]
    U[Inference client] -->|model llama-70b| V[VIP 192.0.2.10:2020]
    C --> V
    V --> B[Backend 198.51.100.11:8080]
    P[Prometheus client] -->|GET /metrics| C

1. Install¶

Example status: verified for documentation syntax and the published container/startup contract; Linux data-plane execution is not performed by the documentation checks.

Prerequisites¶

A Linux host with Docker, a modern eBPF-capable kernel, and permission to run the privileged Gateway container.
A version-pinned public Gateway image or a build from the exact source commit you intend to test.
A persistent host directory for /etc/loxilb.

Exact command¶

Complete the Installation page's container startup procedure. Then record the running image identity:

docker inspect --format '{{.Image}}' loxilb > gateway-image.id
test -s gateway-image.id

Expected result¶

The command exits 0 and gateway-image.id contains one nonempty image ID.

Validate¶

docker ps --filter name='^/loxilb$' --filter status=running --format '{{.Names}}' \
  | grep -qx loxilb
curl --fail-with-body --silent --show-error \
  http://127.0.0.1:11111/netlox/v1/version > gateway-version.json
jq -e 'type == "object"' gateway-version.json

Cleanup¶

Do not remove the Gateway yet; the remaining steps use it. At the end, remove only the container and persistent directory that you created for this lab.

Diagnose¶

Symptom	Check
Container is absent	Inspect `docker ps -a` and the container log.
`/version` is refused	Confirm host networking or the published management port.
eBPF initialization fails	Recheck kernel support, privileges, and required mounts.

2. Check readiness¶

Example status: verified against the selected source pair and frozen readiness contract in Verification Status. This is source/static evidence, not a live Linux result.

Prerequisites¶

The development Gateway and CLI pair listed on the Verification Status page.
A management token in CONTROL_PLANE_TOKEN and a trusted TLS endpoint.
A copy of the endpoint's trusted CA at ./gateway-ca.crt. Use a fresh lab working directory; the credential files below belong only to this run.
The CLI token file must be an absolute, owner-only regular file. The commands use $PWD/gateway.token and create it with mode 0600.

Prepare protected credential files once:

set -euo pipefail
export CONTROL_SCHEME=https CONTROL_HOST=gateway.example.com CONTROL_PORT=443
export CONTROL_API="$CONTROL_SCHEME://$CONTROL_HOST:$CONTROL_PORT/netlox/v1"
: "${CONTROL_PLANE_TOKEN:?Obtain a valid management token before continuing}"
test -r ./gateway-ca.crt
install -m 600 /dev/null ./gateway.token
install -m 600 /dev/null ./control-plane.headers
printf '%s\n' "$CONTROL_PLANE_TOKEN" > ./gateway.token
printf 'Authorization: Bearer %s\n' "$CONTROL_PLANE_TOKEN" > ./control-plane.headers

Exact command¶

RESTloxicmd

ready_status=$(curl --silent --show-error \
  --cacert ./gateway-ca.crt \
  --header @control-plane.headers \
  --output ready.json \
  --write-out '%{http_code}' \
  "$CONTROL_API/status/ready")
printf '%s\n' "$ready_status"

loxicmd get ready --protocol "$CONTROL_SCHEME" \
  --apiserver "$CONTROL_HOST" --port "$CONTROL_PORT" \
  --cacert ./gateway-ca.crt --token-file "$PWD/gateway.token" --output json > ready.json

Expected result¶

Ready returns HTTP 200; not ready returns 503. Both responses carry the typed body. The selected CLI exits 7 for this typed not-ready verdict while preserving the JSON body. A transport or authentication error is a different failure.

Validate¶

test "${ready_status:-200}" = 200
jq -e '.ready == true and (.reasons | length == 0)' ready.json

Cleanup¶

Keep the protected credential files for the next steps. Remove ready.json if it contains deployment details you do not want to retain.

Diagnose¶

Symptom	Check
HTTP `404`	The Gateway is the release build; `/status/ready` is current-main only.
HTTP `503`	Read `reasons[]`, boot replay, external dependency, and persist state.
HTTP `401`	Recheck the management token file and the configured auth mode.

3. Verify management authentication¶

Example status: verified against the management-auth contract. The negative request is mandatory: an authenticated success alone cannot prove the listener is protected.

Prerequisites¶

One supported management authentication mode is enabled.
control-plane.headers contains the valid bearer token from the readiness step.

Exact command¶

unauthenticated_status=$(curl --silent --show-error \
  --cacert ./gateway-ca.crt \
  --output unauthenticated.json \
  --write-out '%{http_code}' \
  "$CONTROL_API/config/loadbalancer/all")

authenticated_status=$(curl --silent --show-error \
  --cacert ./gateway-ca.crt \
  --header @control-plane.headers \
  --output authenticated.json \
  --write-out '%{http_code}' \
  "$CONTROL_API/config/loadbalancer/all")

Expected result¶

The unauthenticated request returns 401; the authenticated request returns 200. Do not continue if both return 200.

Validate¶

test "$unauthenticated_status" = 401
test "$authenticated_status" = 200
jq -e 'type == "object" or type == "array"' authenticated.json

Cleanup¶

Keep the credential files until final cleanup. Remove the two response files after reviewing only sanitized fields.

Diagnose¶

Symptom	Check
Unauthenticated request returns `200`	No management authenticator is active; do not expose the listener.
Authenticated request returns `401`	Token, issuer, expiry, or header-file contents are wrong.
TLS verification fails	Install the correct CA; do not make insecure TLS the permanent fix.

Management and inference credentials are separate

The bearer token above protects configuration on port 11111. It is not an inference API key or JWT. This lab deliberately omits api_key_auth, so a backend-owned X-Api-Key passes through unchanged. Follow Data-Plane Authentication and JWT before exposing a protected inference VIP.

Prepare the receipt backend¶

Use a separate terminal on the Linux backend host. Download the small receipt backend to that host. It is a CPU fixture, not a model-serving engine. It returns backend-llama and appends only the caller's X-Docs-Nonce to a backend-owned log.

python3 receipt-backend.py --listen 0.0.0.0 --port 8080 \
  --receipt /var/tmp/docs-backend-receipts.log

Expected result: the server remains running and creates the receipt file. Allow the Gateway to reach this backend; replace 198.51.100.11 with its reachable lab address in the rule. Assign the chosen VIP to the Gateway's lab interface before creating the rule, following your network procedure.

In the operator terminal, step 5 copies receipts with scp from qa@backend.example.com. Replace that SSH destination with your backend account. If both terminals share the same file system, set BACKEND_RECEIPT_SOURCE='' and use the original backend file directly. A missing file or failed copy is an oracle failure, never zero deliveries. Do not generate the receipt file from the client response or Gateway log. Stop the server with Ctrl-C after rule cleanup; remove only this lab's receipt file after retaining the required evidence.

4. Create one model rule¶

Example status: verified against the Swagger request schema, CLI flags, and the public model-routing scenario. Run either the REST tab or the CLI tab, not both.

Prerequisites¶

The VIP is assigned to the Gateway node.
198.51.100.11:8080 is an HTTP backend that returns the marker backend-llama.
The management authentication negative and positive checks passed.
This is an isolated lab with no concurrent configuration writer or traffic generator; exact state and metric deltas depend on that isolation.

Exact command¶

RESTloxicmd

create_status=$(curl --silent --show-error \
  --cacert ./gateway-ca.crt \
  --request POST \
  --header @control-plane.headers \
  --header 'Content-Type: application/json' \
  --data '{
    "serviceArguments": {
      "externalIP": "192.0.2.10",
      "port": 2020,
      "protocol": "tcp",
      "sel": 0,
      "mode": 4,
      "host": "192.0.2.10",
      "path_prefix": "/",
      "path_match_mode": "prefix",
      "model_name": "llama-70b",
      "inactiveTimeOut": 30
    },
    "endpoints": [
      {"endpointIP": "198.51.100.11", "targetPort": 8080, "weight": 1}
    ]
  }' \
  --output create.json \
  --write-out '%{http_code}' \
  "$CONTROL_API/config/loadbalancer")

loxicmd create lb 192.0.2.10 \
  --tcp=2020:8080 \
  --endpoints=198.51.100.11:1 \
  --mode=fullproxy \
  --host=192.0.2.10 \
  --path-prefix=/ \
  --path-match-mode=prefix \
  --model-name=llama-70b \
  --inatimeout=30 \
  --protocol "$CONTROL_SCHEME" --apiserver "$CONTROL_HOST" --port "$CONTROL_PORT" \
  --cacert ./gateway-ca.crt \
  --token-file "$PWD/gateway.token"

Expected result¶

The REST form returns HTTP 200; the CLI exits 0. A duplicate rule or an invalid body returns a non-success status and must not be counted as created.

Validate¶

test "${create_status:-200}" = 200
curl --fail-with-body --silent --show-error \
  --cacert ./gateway-ca.crt \
  --header @control-plane.headers \
  "$CONTROL_API/config/loadbalancer/all" > rules.json
jq -e '
  .lbAttr[] |
  select(.serviceArguments.externalIP == "192.0.2.10") |
  select(.serviceArguments.port == 2020) |
  .serviceArguments.model_name == "llama-70b" and
  .serviceArguments.mode == 4
' rules.json

Also prove a rejected mutation leaves the complete readback unchanged:

# docs-example: expect-schema-error
canonical_rules() {
  jq -S '
    .lbAttr | map({serviceArguments,
      endpoints: (.endpoints | map({endpointIP,targetPort,weight}) |
        sort_by(.endpointIP,.targetPort))}) |
    sort_by(.serviceArguments.externalIP,.serviceArguments.port,
      .serviceArguments.protocol,.serviceArguments.host,
      .serviceArguments.path_prefix,.serviceArguments.path_match_mode,
      .serviceArguments.model_name)
  ' "$1"
}
canonical_rules rules.json > rules-before-invalid.json
invalid_status=$(curl --silent --show-error \
  --cacert ./gateway-ca.crt \
  --request POST \
  --header @control-plane.headers \
  --header 'Content-Type: application/json' \
  --data '{
    "serviceArguments": {
      "externalIP": "192.0.2.10",
      "port": 2020,
      "protocol": "not-a-protocol",
      "sel": 0,
      "mode": 4
    },
    "endpoints": [
      {"endpointIP": "198.51.100.11", "targetPort": 8080, "weight": 1}
    ]
  }' \
  --output invalid-create.json \
  --write-out '%{http_code}' \
  "$CONTROL_API/config/loadbalancer")
test "$invalid_status" = 422
curl --fail-with-body --silent --show-error \
  --cacert ./gateway-ca.crt \
  --header @control-plane.headers \
  "$CONTROL_API/config/loadbalancer/all" > rules-after-invalid-raw.json
canonical_rules rules-after-invalid-raw.json > rules-after-invalid.json
cmp --silent rules-before-invalid.json rules-after-invalid.json

This invalid enum is rejected by the generated request binder with 422 (error code 606), before the handler applies configuration. Handler-level validation can instead return 400. Do not accept every 4xx as the same result. The comparison sorts rules by their material key and excludes endpoint counters; jq -S alone sorts object keys, not the rule array. Backend health and traffic counters are observations, not proof that the invalid mutation changed configuration.

Cleanup¶

If validation fails after creation, run the exact delete in step 7 before changing any key field.

Diagnose¶

Symptom	Check
HTTP `400` on the valid create	Validate JSON field casing and required endpoint fields.
HTTP `409`	An equivalent rule already exists; inspect before deleting it.
Rule reads back without the model	Recheck exact `model_name` spelling and the complete L7 key.
Invalid create changes readback	Stop and restore the pre-change snapshot; rejection was not atomic.

5. Send traffic¶

Example status: verified against the public model-routing scenario. The backend marker is the independent delivery oracle; HTTP 200 without the expected marker is not a pass.

Prerequisites¶

The rule readback passed.
The backend is healthy, returns backend-llama, and appends each received X-Docs-Nonce value to a line-oriented log available to the operator.
Complete the "Prepare the receipt backend" section above first. Set BACKEND_RECEIPT_LOG to a readable backend-owned log or its read-only SSH copy. It must not be produced from Gateway or client output.

Exact command¶

export BACKEND_RECEIPT_LOG='/var/tmp/docs-backend-receipts.log'
# Set to an empty string only when the backend log is directly accessible.
export BACKEND_RECEIPT_SOURCE='qa@backend.example.com:/var/tmp/docs-backend-receipts.log'
receipt_count() {
  if [ -n "${BACKEND_RECEIPT_SOURCE:-}" ]; then
    scp -q "$BACKEND_RECEIPT_SOURCE" "$BACKEND_RECEIPT_LOG" || return
  fi
  test -r "$BACKEND_RECEIPT_LOG" || return
  grep -cxF -- "$1" "$BACKEND_RECEIPT_LOG" || test "$?" -eq 1
}

curl --fail-with-body --silent --show-error \
  --cacert ./gateway-ca.crt --header @control-plane.headers \
  "$CONTROL_API/config/metrics" > metrics-config-before.json
curl --fail-with-body --silent --show-error --request POST \
  --cacert ./gateway-ca.crt --header @control-plane.headers \
  "$CONTROL_API/config/metrics" > metrics-enable.json
curl --fail-with-body --silent --show-error \
  --cacert ./gateway-ca.crt --header @control-plane.headers \
  "$CONTROL_API/config/metrics" | jq -e '.prometheus == true'
curl --fail-with-body --silent --show-error \
  --cacert ./gateway-ca.crt --header @control-plane.headers \
  "$CONTROL_API/metrics" > metrics-before.prom

positive_nonce="docs-positive-$(date +%s)-$$"
positive_receipts_before=$(receipt_count "$positive_nonce")
traffic_status=$(curl --silent --show-error \
  --request POST \
  --header 'Content-Type: application/json' \
  --header "X-Docs-Nonce: $positive_nonce" \
  --data '{"model":"llama-70b","messages":[{"role":"user","content":"hello"}]}' \
  --output inference.json \
  --write-out '%{http_code}' \
  http://192.0.2.10:2020/v1/chat/completions)

Expected result¶

The request returns HTTP 200, and the backend-specific marker appears in the response. The exact OpenAI response shape belongs to the backend. Metrics are explicitly enabled and their previous state is retained for cleanup. A disabled exporter answers 503; an enabled exporter requiring management authentication answers 401 without a valid credential. These are separate failure causes.

Validate¶

test "$traffic_status" = 200
grep -q 'backend-llama' inference.json
positive_receipts_after=$(receipt_count "$positive_nonce")
test "$((positive_receipts_after - positive_receipts_before))" -eq 1

Also prove a wrong model does not reach this backend:

wrong_model_nonce="docs-negative-$(date +%s)-$$"
negative_receipts_before=$(receipt_count "$wrong_model_nonce")
wrong_model_status=$(curl --silent --show-error \
  --request POST \
  --header 'Content-Type: application/json' \
  --header "X-Docs-Nonce: $wrong_model_nonce" \
  --data '{"model":"unknown-model","messages":[]}' \
  --output wrong-model.json \
  --write-out '%{http_code}' \
  http://192.0.2.10:2020/v1/chat/completions)
test "$wrong_model_status" = 503
! grep -q 'backend-llama' wrong-model.json
negative_receipts_after=$(receipt_count "$wrong_model_nonce")
test "$negative_receipts_after" -eq "$negative_receipts_before"

Cleanup¶

Keep the rule until after the metrics check. Remove response files when they are no longer needed.

Diagnose¶

Symptom	Check
HTTP `503 model_unavailable`	Model matching is case-sensitive; confirm `llama-70b`.
HTTP `200` without the marker or receipt delta	The request did not prove the expected active backend path.
Connection refused	Confirm the VIP, fullproxy listener, and backend reachability.
Wrong-model receipt count increases	Stop; the rejected request reached the backend.

6. Inspect metrics¶

Example status: verified against the frozen release-scope metric manifest and current writer mapping. The documentation check validates the name and labels; it does not claim a live scrape occurred on your host.

Prerequisites¶

Metrics export is enabled and GET /metrics is reachable.
metrics-before.prom was captured immediately before the single positive request on an otherwise idle, isolated Gateway.

Exact command¶

metrics_status=$(curl --silent --show-error \
  --cacert ./gateway-ca.crt --header @control-plane.headers \
  --output metrics-after.prom \
  --write-out '%{http_code}' \
  "$CONTROL_API/metrics")

Expected result¶

The scrape returns HTTP 200 with Prometheus exposition. Delivery is proved by the backend nonce log in step 5: positive delta 1, rejected-model delta 0. loxilb_service_requests_total counts sampled new sessions observed by the conntrack sweep, not completed HTTP requests. Short sessions can be missed and keep-alive can carry multiple requests. A sample may be absent; absence is not zero.

Validate¶

test "$metrics_status" = 200
grep -q '^# HELP ' metrics-after.prom
if grep -q '^loxilb_service_requests_total{' metrics-after.prom; then
  grep '^loxilb_service_requests_total{' metrics-after.prom
else
  printf '%s\n' 'No sampled session series emitted; use backend receipts for delivery.'
fi

Cleanup¶

Remove both metric snapshots after retaining only the sanitized evidence required by your deployment process.

Diagnose¶

Symptom	Check
HTTP `404`	Confirm the API base path and the running Gateway build.
HTTP `200` but no series	Confirm export activation and that traffic crossed the active service path.
Session count differs from request count	Expected for sampling or connection reuse; compare backend nonce receipts.
Unexpected labels	Compare the query with the frozen metric reference.

7. Clean up¶

Example status: verified against the exact model-keyed REST and CLI delete contracts. Run the cleanup form that matches the create form you chose.

Prerequisites¶

Use the same VIP, port, host, path, path mode, and model name used at create.
Confirm that the matching rule belongs to this lab.

Exact command¶

RESTloxicmd

cleanup_status=$(curl --silent --show-error \
  --cacert ./gateway-ca.crt \
  --request DELETE \
  --header @control-plane.headers \
  --output cleanup.json \
  --write-out '%{http_code}' \
  "$CONTROL_API/config/loadbalancer/hosturl/192.0.2.10/externalipaddress/192.0.2.10/port/2020/protocol/tcp?path_prefix=%2F&path_match_mode=prefix&model_name=llama-70b")

loxicmd delete lb 192.0.2.10 \
  --tcp=2020 \
  --host=192.0.2.10 \
  --path-prefix=/ \
  --path-match-mode=prefix \
  --model-name=llama-70b \
  --protocol "$CONTROL_SCHEME" --apiserver "$CONTROL_HOST" --port "$CONTROL_PORT" \
  --cacert ./gateway-ca.crt \
  --token-file "$PWD/gateway.token"

Expected result¶

The REST form returns 200; the CLI exits 0.

Validate¶

test "${cleanup_status:-200}" = 200
curl --fail-with-body --silent --show-error \
  --cacert ./gateway-ca.crt \
  --header @control-plane.headers \
  "$CONTROL_API/config/loadbalancer/all" > rules-after-cleanup.json
! jq -e '
  .lbAttr[] |
  select(.serviceArguments.externalIP == "192.0.2.10") |
  select(.serviceArguments.port == 2020) |
  select(.serviceArguments.model_name == "llama-70b")
' rules-after-cleanup.json
curl --fail-with-body --silent --show-error \
  --cacert ./gateway-ca.crt \
  --header @control-plane.headers \
  "$CONTROL_API/status/ready" > ready-after-cleanup.json
jq -e '.ready == true and (.reasons | length == 0)' ready-after-cleanup.json

Cleanup¶

Restore the previous metrics setting before removing credential files:

if jq -e '.prometheus == false' metrics-config-before.json >/dev/null; then
  curl --fail-with-body --silent --show-error --request DELETE \
    --cacert ./gateway-ca.crt --header @control-plane.headers \
    "$CONTROL_API/config/metrics" > metrics-restore.json
  curl --fail-with-body --silent --show-error \
    --cacert ./gateway-ca.crt --header @control-plane.headers \
    "$CONTROL_API/config/metrics" | jq -e '.prometheus == false'
fi

Remove only this lab's local credential and response files. Stop the receipt backend in its terminal with Ctrl-C; preserve its log until the receipt checks have been reviewed.

rm -f ./gateway.token ./control-plane.headers ready.json \
  unauthenticated.json authenticated.json create.json rules.json \
  invalid-create.json rules-before-invalid.json rules-after-invalid.json rules-after-invalid-raw.json \
  metrics-config-before.json metrics-enable.json metrics-restore.json \
  inference.json wrong-model.json metrics-before.prom metrics-after.prom cleanup.json \
  rules-after-cleanup.json ready-after-cleanup.json gateway-version.json gateway-image.id
unset CONTROL_PLANE_TOKEN CONTROL_API CONTROL_SCHEME CONTROL_HOST CONTROL_PORT \
  BACKEND_RECEIPT_LOG BACKEND_RECEIPT_SOURCE

Diagnose¶

Symptom	Check
HTTP `404`	One rule-key component differs from create; inspect the exact readback.
Rule remains	Repeat the complete model-keyed delete; do not use delete-all on a shared Gateway.
Other rules disappeared	Stop and restore from the pre-change snapshot; cleanup scope was too broad.
Readiness fails after cleanup	Inspect `reasons[]`; cleanup is not complete until readiness recovers.

REST-only surfaces¶

Do not invent CLI flags for surfaces the frozen CLI does not implement:

Surface	Supported path
JWT profile create/list/delete	REST-only
JWT service binding	Selected CLI supports `--api-key-auth=jwt` or `apikey-or-jwt` with `--jwt-auth-profile`; the comparison CLI release supports only `disabled` and `required`
Global/rule-default and user/user-model QoS CRUD	REST-only
Model-profile discovery and `kvexactstatus`	REST-only
`kvModelProfile` and `kvExactApiMode`	REST-only
Capability readiness and `sockmapreset`	REST-only

The CLI reference carries the canonical compatibility matrix.

Next steps¶

Model Load Balancing explains multi-pool and wildcard routing.
Management API Authentication covers supported operator authentication modes.
Data-Plane Authentication and JWT adds inference credentials without confusing the two trust planes.
Monitoring and Metrics defines activation and evidence limits for every metric family.
Verification Status explains why source/static, CI, Linux runtime, GPU, HA, and release evidence are separate.

Continue with model-serving engines¶

After this basic lifecycle, follow the LLM integration reading path for engine/topology concepts and the complete converged or P/D guide.