Model Profile Catalog and Discovery¶
The model reference identifies the five models with paired RR/Exact evaluation configurations and separately supplied QA profiles. Earlier recommendation labels are historical installation baselines. The Gateway's strict KV-exact profile catalog is a separate list. A model name, a profile in a source checkout, a supported engine row, a published runtime profile and a ready rule answer different questions.
Before you start¶
Keep the selected model snapshot and engine version at hand; no service mutation is required here.
Focus of this page: Compare profile identity, block hash contract and registry readiness.
Topology and execution locations¶
flowchart TD
C["Client workload"] --> G["Selected Gateway / routing mode"]
G --> W["Selected engine worker pool<br/>count and roles from its runbook"]
O["Operator"] -.->|"configuration and readback"| G
O -.->|"direct engine / receipt evidence"| W
style G fill:#e1f5fe,stroke:#0288d1,color:#1f2937
style W fill:#e8f5e9,stroke:#43a047,color:#1f2937This reference diagram locates the client, Gateway and engine evidence. It does not prescribe a replica count or promote historical results to a new topology. Use the exact worker count, roles, ports and artifact identities from the selected runbook or retained result; do not mix that tuple with another guide.
| Where | What happens here |
|---|---|
| Operator terminal | Select the version tuple, private run directory, model snapshot and profiles. |
| Gateway host / namespace | Start the dedicated Gateway; apply and read back only the selected service. |
| Each worker host | Launch the chosen engine and prove direct inference before Gateway traffic. |
| Client | Send the same request through the Gateway and retain its HTTP/SSE observations. |
| Independent evidence | Correlate worker requests/receipts with the selected rule and request identifiers. |
Walkthrough¶
Which list should I use?¶
| List or surface | What it tells you | What it does not prove |
|---|---|---|
| Engine support catalog / capability matrix | Engine event/hash/probe contract and exact engine identity promotion | A model-profile row is validated or a newer card engine is interchangeable |
| Five standard model cards | Current vLLM 0.31 / SGLang 0.5.19 / TRT rc29 settings, paired RR/Exact results and retained operating limitations | Installed release support or registry membership |
Release docs/SUPPORTED-MODELS.md / install-models.sh --list |
Profile IDs, exact revisions, engine tuples, qualification state and required launch arguments | The profile is installed on this Gateway |
GET /config/ai/model-profiles |
Profiles currently published in the running registry generation | A discovered profile's source tuple has been promoted to supported |
| Profile detail | Model/alias policy, API surfaces, tokenizer/template identity | Rule admission or enforced Exact readiness |
Rule kvexactstatus |
The requested binding and its actual attestation/enforcement state | Capacity, fail-over or sustained latency qualification |
Only a source-catalog tuple with state: validated is supported under that
catalog's contract. candidate means evidence exists but promotion is incomplete;
blocked means that model/engine pair is refused for a named reason. Discovery
has no authority to promote either state. Follow the catalog matching the installed
Gateway release, not an unrelated checkout or the newest engine tag.
An engine-contract profile (for example sglang-kv-rank-v1 or
trtllm-kv-http-preview-v1) describes how Gateway integrates with that engine.
A model profile (for example r1-distill-qwen-15b-v1) pins the model's tokenizer,
chat template and model identity. The former can be validated while a particular
model-engine tuple in the latter catalog is still candidate. An engine
validated label must not be used to promote a model row. See
framework capability versus card verification.
Historical card profiles are not public catalog rows¶
The original cards and the committed catalog were compared by exact profile ID, model repository/revision and engine version, not by model family name. All five card IDs are absent from this catalog snapshot:
| Historical card | Historical profile ID | Membership in reviewed public catalog | Different catalog model in the same family |
|---|---|---|---|
| Gemma2 9B | gemma2-9b-it-v1 |
Absent | Gemma-3 1B / Gemma-4 E2B |
| Qwen2.5 7B | qwen25-7b-instruct-v1 |
Absent | Qwen3.6 / Qwen3.8 27B FP8 |
| Llama3.1 8B | llama31-8b-instruct-v1 |
Absent | Llama-3.2 1B |
| R1 Distill Qwen 7B | r1-distill-qwen-7b-v1 |
Absent | R1 Distill Qwen 1.5B |
| EXAONE3.5 7.8B | exaone35-78b-instruct-v1 |
Absent | EXAONE-4.0 1.2B |
Absent, candidate and blocked are different states. Absent means that this catalog supplies no definition/installable tuple for that ID. Candidate means a definition exists but has not been promoted to supported. Blocked is an explicit refusal for a listed tuple. The four old "recommended" cards describe observations on a different candidate runtime, not any of those source-catalog states. Gemma's historical limited card does not become supported through its profile ID.
For example, r1-distill-qwen-7b-v1 and r1-distill-qwen-15b-v1 refer to
7B and 1.5B models, respectively; 15b in that catalog ID must not be read
as a substitute for the 7B card. The Llama, EXAONE, Qwen and Gemma families also
have different model generations/sizes/revisions here. A family match is not a
valid alias, tokenizer/template match, engine contract or migration path.
--include-candidates opts into existing candidate rows only. It does not add
the five missing definitions, install the old private registry assets or promote
any tuple. Even seeing a historically named profile in an independently supplied
runtime registry would prove only discovery; support requires a matching release
catalog tuple and its qualification state.
Use the five-model manual for the separately supplied
evaluation-profiles/ definitions, pinned manifests, independent token fixtures
and both routing modes. These assets are not installed by --include-candidates
and do not alter the catalog snapshot. The result table
distinguishes functional evidence from native architecture blockers. The separate
R1 1.5B guide remains a different evaluation tuple.
Supported Exact still requires a matching validated release tuple; this catalog
snapshot offers none. READY and profile discovery do not promote support.
The core-team registration proposal adds five profiles and 15 engine tuples (13 candidate, two TRT blocked), with SGLang 0.5.19 as the current guide baseline. Installer positive checks passed for those 13 candidate tuples, but core version-selector and installer negative gates remain open. The proposal is separate from the historical source snapshot below.
Additional source-catalog profiles¶
The following snapshot was reviewed on 2026-10-20 from the committed Gateway supported-model manifest. This is a historical source snapshot, retained without rewriting its qualification claims. All executable vLLM instructions in this use case now select 0.31.0. The snapshot is not a 0.31 compatibility list; matching new manifest/fixture/runtime evidence is required. Its engine columns refer to vLLM v0.28.0, SGLang v0.5.18 and TensorRT-LLM 1.3.0rc24, respectively. The current five-model evaluation uses vLLM 0.31.0, TRT rc29 and SGLang 0.5.19, which retains tagged-array events and includes the native tokenize integer-bound fix. The five model profiles remain separate from these catalog rows. Do not mix a profile from one tuple with a card's engine version.
| Model repository | Profile ID | vLLM | SGLang | TRT / PyTorch |
|---|---|---|---|---|
allenai/OLMo-2-0425-1B-Instruct |
olmo2-0425-1b-v1 |
candidate | candidate | blocked · architecture_unsupported |
deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B |
r1-distill-qwen-15b-v1 |
candidate | candidate | candidate |
google/gemma-3-1b-it |
gemma3-1b-it-v1 |
candidate | candidate | candidate |
google/gemma-4-E2B-it |
gemma4-e2b-it-v1 |
candidate | candidate | blocked · hardware_unsupported |
meta-llama/Llama-3.2-1B-Instruct |
llama32-1b-v1 |
candidate | candidate | candidate |
LGAI-EXAONE/EXAONE-4.0-1.2B |
exaone4-12b-v1 |
candidate | candidate | candidate |
microsoft/Phi-4-mini-instruct |
phi4-mini-instruct-v1 |
candidate | candidate | candidate |
ibm-granite/granite-4.2-3b |
granite42-3b-v1 |
candidate | candidate | blocked · architecture_unsupported |
mistralai/Ministral-3-3B-Instruct-2512 |
ministral3-3b-v1 |
blocked · engine_renderer |
blocked · engine_renderer |
blocked · checkpoint_format |
skt/A.X-3.1-Light |
ax31-light-v1 |
candidate | candidate | candidate |
Qwen/Qwen3.6-27B-FP8 |
qwen36-27b-fp8-v1 |
candidate | candidate | candidate |
Qwen/Qwen3.8-27B-FP8 |
qwen38-27b-fp8-v1 |
candidate | candidate | candidate |
openai/gpt-oss-20b |
gptoss-20b-v1 |
blocked · engine_renderer |
candidate | blocked · engine_renderer |
This snapshot contains 13 profiles / 39 model-engine tuples: 31 candidate,
8 blocked, 0 validated. It is a catalog of available definitions and limitations,
not a list of 39 production-supported choices. The normal installer must not be
assumed to install candidates by default. --include-candidates is an explicit
evaluation opt-in; it does not change support state.
The full revisions, engine versions and declared mode/API coverage are in the public catalog snapshot JSON and the configuration ZIP. The snapshot omits internal evidence locations and runtime credentials. Always recheck the upstream supported-model page for the exact release you install.
Read the blocked reasons and mode limits¶
architecture_unsupported: that pinned engine has no usable model implementation.hardware_unsupported: the tested hardware/backend combination is blocked; this does not assert that every GPU is unsupported.engine_renderer: the engine's renderer/tokenizer differs from the strict profile's verified template contract; do not bypass it by editing the template.checkpoint_format: the pinned loader cannot consume that official checkpoint format.
TRT entries in this snapshot cover converged chat only; P/D or completions must not be inferred. Even candidate vLLM/SGLang entries can have incomplete evidence for a declared API surface. Read the qualification columns and required launch arguments in the release catalog. For example, a hybrid or sliding-window model can have different cache groups and block/chunk geometry; copying Qwen's block settings is not a registration procedure. llama.cpp has no strict KV-exact rows in this catalog and uses its ordinary integration path.
Inspect the release catalog locally¶
Run these commands from the approved Gateway source checkout matching your installed image, not from the extracted configuration package:
scripts/models/install-models.sh --list
scripts/models/install-models.sh --dry-run --engine vllm --models olmo2-0425-1b-v1
The first command lists definitions; the second previews installation without
writing files. With only candidate/blocked rows, a preview can skip/refuse the
requested model under normal support policy. That is not an installation failure
to work around silently. For an approved evaluation, add --include-candidates
explicitly and retain the unpromoted state. Obtain appropriate model access for
gated repositories. Full installation, artifact verification, registry mounts
and reload are covered by Prepare a Supported Model.
The public model-registration tool is the only script dependency in this path;
there is no new profile-generation script in this package.
Discover what this Gateway has published¶
On the controller, use CONTROL_API, RUN and the protected management header
from Gateway prerequisites:
curl -fsS --max-time 10 --header @control-plane.headers \
"$CONTROL_API/config/ai/model-profiles" > "$RUN/model-profiles-live.json"
jq '{registryGeneration,setDigest,profiles:[.profiles[] | {
profileId,gen,baseModel,aliasPolicy,allowedAliases,supportedApis,
tokenizerSha256,templateSha256
}]}' "$RUN/model-profiles-live.json"
Expected: the published generation and zero or more entries. Generation 0 with
an empty array means no published registry, not a ready strict model. A populated
array is only runtime discovery. Match an entry to a validated release tuple
before calling it supported. The five current QA profile IDs are absent from the reviewed source catalog;
they appear in live discovery only after their separate evaluation registry is published;
never substitute an unrelated profile just because its model has a similar name.
Choose an exact ID returned by that list. The following example ID is usable only if it appeared in your discovery response and is permitted by your release or explicitly authorized candidate-evaluation policy:
export PROFILE_ID=olmo2-0425-1b-v1
curl -fsS --max-time 10 --header @control-plane.headers \
"$CONTROL_API/config/ai/model-profiles/$PROFILE_ID" > "$RUN/model-profile-detail.json"
jq '{profileId,gen,baseModel,aliasPolicy,allowedAliases,supportedApis,
tokenizerSha256,templateSha256}' "$RUN/model-profile-detail.json"
Expected: detail matches the selected published profile; an absent profile returns
404. Use profileId, not repository name, in kvModelProfile. Confirm that
model_name is admitted by baseModel/alias policy, chat is in supportedApis,
the engine tuple and tokenizer/template are exact matches, and the final
rule binding reaches READY with
the matching generation and unfenced enforcement. Retain setDigest and
bindingDigest separately: one identifies the published set, the other the rule's
binding. Never replay discovery or status JSON as a create body.
Missing or changed profiles¶
If a profile is absent, mismatched, blocked or still candidate, keep that state
visible and stop the supported Exact path. Use a separately qualified RR service
if appropriate. New model support requires a published profile and engine contract,
verified tokenizer/template/artifacts and the release's qualification/promotion
process; an operator cannot create support by changing model_name in a JSON file.
Registry upgrades also require rule migration/readiness and chat/SSE rechecks.
Check your result¶
Catalog membership, a staged profile and installed readiness are different states; verify each before Exact traffic.
Select a supported model/engine tuple before interpreting results. Keep direct-worker success separate from Gateway delivery, routing evidence, engine cache reuse, performance and release qualification. Historical PASS, FAIL and BLOCKED results retain their original artifact identities.
Next steps and cleanup¶
Use the canonical deployment sequence rather than combining fragments from different engine/version pages. Drain before switching or deleting owned resources; retained result pages are evidence, not another deployment recipe. Read the lab starting point if required values are missing. Use cleanup and recovery when ending the run.