Skip to content

Model Aliases & Testing (Foundry)

The ASK services reference model names in three layers, and not all of them are config-overridable. Where a model name is hardcoded in source (L3), the only air-gap fix is to create an alias on Foundry (the LiteLLM gateway) that maps the hardcoded name to a model we actually serve. This page covers the four operational tasks around that workaround:

  1. See all aliases of a model
  2. Per-service Helm file + key to update (with the alias)
  3. Test each model change with curl and kubectl
  4. E2E testing without the frontend

Why aliases at all — the 3 reference layers

Layer Where the model name lives How to change it
L1 — config TOML default.toml default, overridden in values-…yaml (e.g. [foundry] llm_model) edit the Helm value → resync. No alias needed.
L2 — baked catalog core-service/helm/values.yaml model list (user-selectable ids) edit the chart values + redeploy or add a Foundry alias for the id.
L3 — hardcoded in source a model name compiled into the code with no config key cannot change via config → create a Foundry alias with that exact name → a served model.

Full layer explanation: LLM → Service Config Mappings. Foundry endpoint + key handling: Foundry / LLM Endpoint.

Models actually served (air-gap)

These are the only model IDs Foundry/vLLM/Infinity serves. Every L1/L2/L3 reference must resolve to one of these — directly or via an alias.

Served model ID Kind Used for
qwen/qwen3-5-122b-a10b chat / vision (flagship) orchestrator, data-viz, VLM
qwen/qwen3-5-122b-a10b-kms chat / vision (KMS-dedicated instance) KMS OCR/VLM only
Qwen/Qwen3-4B-Instruct-2507 chat (small) summaries, history compression, guardrails
zai-org/glm-5-2 (also z-ai/glm-5) chat / reasoning NEW (2026-07) — GLM-5.2; role TBD (likely code-gen / data-analysis)
qwen/qwen3-embedding-0-6b embedding (1024-dim) tool/KMS embeddings
nvidia/llama-nemotron-rerank-1b-v2 rerank reranking

Live snapshot — /v1/models + /model/info (2026-07-05, post-maintenance)

Served backends confirmed via /model/info (model_name → litellm_params.model):

Public name(s) Backend (vLLM)
z-ai/glm-5, zai-org/glm-5-2 hosted_vllm/zai-org/GLM-5.2 ← new
qwen/qwen3-5-122b-a10b, foundry-stg/core42-oicm-auh1-shd/qwen-3-5-122b-a10b hosted_vllm/Qwen/Qwen3-5-122B-A10B
qwen/qwen3-5-122b-a10b-kms hosted_vllm/Qwen/Qwen3-5-122B-A10B (separate instance)
Qwen/Qwen3-4B-Instruct-2507 hosted_vllm/Qwen/Qwen3-4B-Instruct-2507
qwen/qwen3-embedding-0-6b, AI71ai/agrillm-embedding-bge-m3-v1 hosted_vllm/Qwen/Qwen3-Embedding-0.6B
nvidia/llama-nemotron-rerank-1b-v2 hosted_vllm/nvidia/llama-nemotron-rerank-1b-v2

Delta vs 2026-06-28: GLM-5.2 added (as z-ai/glm-5 + zai-org/glm-5-2). openai/gpt-oss-120b and its aliases (gpt-oss-120b, aicloud/gpt-oss-120b) no longer appear in /v1/models or /model/info → treat as removed/replaced pending confirmation. The bare model_group_alias entries also don't appear, and /model/group/info returns Not Found on this LiteLLM build — confirm whether those aliases are still callable at request time:

```bash KEY=$(kubectl get secret -n ask assistant-service-secret -o jsonpath='{.data.AI71_FOUNDRY_KEY}' | base64 -d) for m in gpt-oss-120b openai/gpt-oss-120b qwen3-5-122b-a10b z-ai/glm-5; do kubectl exec -i -n ask deploy/assistant-service-intelligence -- env KEY="$KEY" M="$m" python3 - <<'EOF'

import os, httpx m=os.environ["M"] r=httpx.post("http://foundry-litellm.foundry.svc.cluster.local:4000/v1/chat/completions", headers={"Authorization":"Bearer "+os.environ["KEY"]}, json={"model":m,"messages":[{"role":"user","content":"ping"}],"max_tokens":1}, timeout=20) print(f"{m:30s} -> HTTP {r.status_code}") EOF done ```

qwen/qwen3-5-4b removed from Foundry

The prior small model (Qwen/Qwen3-5-4B / alias qwen3-5-4b) is no longer served. It has been replaced by Qwen/Qwen3-4B-Instruct-2507 (alias qwen3-4b-instruct-2507). All L1 config references to the old name must point to Qwen/Qwen3-4B-Instruct-2507 — already applied in values-mod-auh1-dev-ask.yaml (Fix 1, 2026-06-27).

1. See all aliases on Foundry

Aliases live on the LiteLLM proxy (foundry-litellm.foundry.svc.cluster.local:4000). Each entry in the proxy's model_list has a public model_name (the alias) mapped to an underlying litellm_params.model (the served backend). Query the running proxy to list them — you do not edit anything to read them.

Keys

  • Virtual key (any model call / /v1/models): assistant-service-secret → AI71_FOUNDRY_KEY.
  • Master/admin key (/model/info, /model/group/info, admin endpoints): core-service-secret → FOUNDRY_ADMIN_KEY. App images often lack curl; run from a pod via python3/httpx, which is always present in assistant-service-intelligence.

a) List every public model name (= every alias)

KEY=$(kubectl get secret -n ask assistant-service-secret -o jsonpath='{.data.AI71_FOUNDRY_KEY}' | base64 -d)
kubectl exec -i -n ask deploy/assistant-service-intelligence -- env KEY="$KEY" python3 - <<'EOF'
import os, httpx, json
r = httpx.get("http://foundry-litellm.foundry.svc.cluster.local:4000/v1/models",
              headers={"Authorization": "Bearer " + os.environ["KEY"]}, timeout=15)
print(r.status_code)
print("\n".join(sorted(m["id"] for m in r.json().get("data", []))))
EOF
/v1/models returns the OpenAI-style list — every id is a model name the gateway accepts, including aliases. If a name your code/config calls appears in this list, its alias exists.

Earlier 2026-06-28 snapshot superseded

A prior /v1/models snapshot (15 names including gpt-oss-120b and 9 bare model_group_alias entries) has been superseded by the 2026-07-05 live snapshot above. gpt-oss-120b and its aliases were decommissioned (roles moved to qwen/qwen3-5-122b-a10b); GLM-5.2 was added. Re-run the /model/info command in §1b for the current mapping.

Upstream default.toml defaults are L1 — not gaps

assistant-service/config/default.toml ships some upstream default model names that Foundry does not serve, but our air-gap overlay (values-mod-auh1-dev-ask.yaml) overrides every one with a served model — so those defaults are never actually called and need no alias. Only a name that is both not served and not overridden by our config would be a real gap.

Config-declared aliases vs models.yml (gap analysis)

The mlops-mod-production/07-foundry-litellm/01-values.yaml router_settings.model_group_alias block is the static alias config baked into the Helm values (as opposed to dynamically-added model_list entries). This table compares every declared alias against ask-core-service/src/model_provider/models/models.yml (the platform UI model catalogue) to identify gaps.

Alias (clients call this) Resolves to In models.yml? Notes
qwen3-4b-instruct-2507 Qwen/Qwen3-4B-Instruct-2507 ❌ no 4B instruct bare alias. Used by small_llm_model, history_summary.model (L1 config)
~~gpt-oss-120b~~ removed id still in core-service catalogue (dead entry) Decommissioned 2026-07 — roles moved to qwen/qwen3-5-122b-a10b
~~aicloud/gpt-oss-120b~~ removed ❌ no Legacy alias — gone with gpt-oss-120b
foundry-stg/core42-oicm-auh1-shd/qwen-3-5-122b-a10b qwen/qwen3-5-122b-a10b ❌ no Core42 PP staging full path — compat alias
AI71ai/agrillm-embedding-bge-m3-v1 qwen/qwen3-embedding-0-6b ✅ yes — id: AI71ai/agrillm-embedding-bge-m3-v1 Key alias: models.yml UI ID transparently served by local Qwen3 embedding
qwen3-5-122b-a10b qwen/qwen3-5-122b-a10b ❌ no Bare alias for flagship chat/VLM
qwen3-5-122b-a10b-kms qwen/qwen3-5-122b-a10b-kms ❌ no Bare alias for KMS-dedicated vLLM instance
qwen3-embedding-0-6b qwen/qwen3-embedding-0-6b ❌ no Bare alias for embedding
llama-nemotron-rerank-1b-v2 nvidia/llama-nemotron-rerank-1b-v2 ❌ no Bare alias for reranker
qwen/qwen3-5-4b Qwen/Qwen3-4B-Instruct-2507 ❌ — compat redirect Old small model removed; callers using provider-prefixed form redirected
qwen3-5-4b Qwen/Qwen3-4B-Instruct-2507 ❌ — compat redirect Old small model removed; bare-name callers redirected
~~gpt-oss-safeguard-120b~~ removed ❌ no Guardrails now use qwen-3-4b-it-2507 / qwen3-5-122b-a10b (commit 32aaae8) — ⚠️ bare names 400, see warning above

Summary: aliases doing active work for the app

models.yml id Foundry alias resolves to Why it matters
AI71ai/agrillm-embedding-bge-m3-v1 qwen/qwen3-embedding-0-6b UI embedding ID → local Qwen3 embedding
AI71ai/agrillm-embedding-bge-m3-v1 qwen/qwen3-embedding-0-6b AgriLLM embedding ID in UI → local Qwen3 embedding

The remaining 10 aliases handle backwards compatibility (legacy paths), compat redirects for the removed qwen3-5-4b model, and backend-only models (KMS, reranker, 4B instruct) not surfaced in the UI catalogue.

b) See the alias → backend mapping (admin)

ADMIN=$(kubectl get secret -n ask core-service-secret -o jsonpath='{.data.FOUNDRY_ADMIN_KEY}' | base64 -d)
kubectl exec -i -n ask deploy/assistant-service-intelligence -- env ADMIN="$ADMIN" python3 - <<'EOF'
import os, httpx, json
r = httpx.get("http://foundry-litellm.foundry.svc.cluster.local:4000/model/info",
              headers={"Authorization": "Bearer " + os.environ["ADMIN"]}, timeout=15)
for m in r.json().get("data", []):
    print(f'{m.get("model_name"):40s} -> {m.get("litellm_params", {}).get("model")}')
EOF
/model/info shows model_name (the alias clients call) → litellm_params.model (what it actually routes to). This is how you confirm what each name resolves to.

Live alias/backend mapping

For the authoritative, current alias → backend mapping, run the /model/info command in §1b above. (The prior hand-maintained 2026-06-28 alias listing was removed — gpt-oss-120b is decommissioned and GLM-5.2 added, so /model/info is the source of truth.)

-kms IS a dedicated vLLM instance (confirmed)

qwen/qwen3-5-122b-a10b-kms has its own api_base: http://vllm-qwen3-5-122b-a10b-kms-engine-service.foundry.svc.cluster.local/v1 — a separate vLLM deployment from the main vllm-qwen3-5-122b-a10b-engine-service. Both serve the same model weights (Qwen3-5-122B-A10B) but on isolated compute, so KMS OCR/VLM load does not contend with orchestrator traffic. The previous warning about this being unverified is now resolved.

c) Confirm one specific name resolves

Probe any model name you intend to use (here the bare alias qwen3-5-122b-a10b): 

kubectl exec -i -n ask deploy/assistant-service-intelligence -- env KEY="$KEY" python3 - <<'EOF'
import os, httpx
r = httpx.post("http://foundry-litellm.foundry.svc.cluster.local:4000/v1/chat/completions",
   headers={"Authorization":"Bearer "+os.environ["KEY"]},
   json={"model":"qwen3-5-122b-a10b","messages":[{"role":"user","content":"ping"}],"max_tokens":5}, timeout=30)
print(r.status_code, r.text[:200])
EOF
A 200 means the name is live. A 400 "Invalid model name" means it is not registered → point your config at a served name (above) or ask the Foundry team to add the alias.

Who edits the alias list

The LiteLLM model_list lives in the Foundry namespace (managed by the Foundry/platform team), not in our service repos. To add an alias, request the Foundry team add a model_list entry: 

- model_name: <name-your-code-calls>            # the exact name the code/config uses
  litellm_params:
    model: hosted_vllm/Qwen/Qwen3-5-122B-A10B   # a served backend
We can only read the aliases (this section) and point our config at them (next section).

2. Per-service model references — what to edit

For L1 (config) references, change the model string directly in the service's env values file and resync — no alias required. Use an alias only for L2 catalog IDs and L3 hardcoded names that you can't move.

assistant-service

mod-ask-assistant-service/helm/environments/values-mod-auh1-dev-ask.yaml — TOML under configToml:

Feature TOML key Current value Layer
Orchestrator (flagship) [foundry] llm_model qwen/qwen3-5-122b-a10b L1
Small / summaries / guardrails [foundry] small_llm_model Qwen/Qwen3-4B-Instruct-2507 L1
Code generation [foundry] code_generator_model qwen/qwen3-5-122b-a10b L1
DOCX context [foundry] docx_prepare_context_model qwen/qwen3-5-122b-a10b L1
Data visualization [foundry] data_visualization_llm qwen/qwen3-5-122b-a10b L1
Tool embedding [foundry] tool_embedding_model qwen/qwen3-embedding-0-6b L1
Reranker [foundry.reranker] model nvidia/llama-nemotron-rerank-1b-v2 L1
Data-analysis tools [tooling.data_analysis] profiler_model / code_generator_model / query_validator_model / column_mapper_model qwen/qwen3-5-122b-a10b L1
Guardrails [guardrails] safety_filters_model / keyword_filters_model / custom_policy_model qwen-3-4b-it-2507 / qwen3-5-122b-a10b / qwen3-5-122b-a10b — ⚠️ bare names 400 on Foundry (see warning) L1

Guardrail model names do not resolve on Foundry (verify with @Adrian)

Commit 32aaae8 (2026-07) moved gpt-oss-120b's roles to qwen/qwen3-5-122b-a10b and added a [guardrails] block, but its model names use the bare catalogue-id form (qwen-3-4b-it-2507, qwen3-5-122b-a10b). A direct Foundry call to those names returns HTTP 400 (not served) — only the prefixed names are live (Qwen/Qwen3-4B-Instruct-2507, qwen/qwen3-5-122b-a10b). Either the guardrail path resolves catalogue-id → served-name via models.yaml/an alias (confirm it does), or the guardrail config must use the prefixed served names. Confirmed via /v1/chat/completions test, 2026-07-05.

knowledge-management-service (KMS)

mod-ask-knowledge-management-service/helm/environments/values-mod-auh1-dev-ask.yaml:

Feature Key Current value Layer
Embedding (1024-dim, matches Weaviate) embedding_model_id qwen/qwen3-embedding-0-6b L1
OCR / VLM (KMS-dedicated instance) vlm_model_name qwen/qwen3-5-122b-a10b-kms L1
Reranker (enabled — default.toml ships @none) [retriever.reranker] model nvidia/llama-nemotron-rerank-1b-v2 L1

KMS reranker — enabled in config (was off by default)

KMS default.toml ships [retriever] reranker = @none (no reranking). We added [mod-auh1-dev-ask.retriever.reranker] (model + candidate_pool_multiplier=2.0, elbow_cutoff_min_k=5, elbow_min_points=5) to the chart env file (the devops overlay only sets image:, so it doesn't shadow this). No endpoint needed — RerankerService reuses the Foundry client.

core-service

mod-ask-devops/applications/core-service/values-mod-auh1-dev-ask.yaml (overlay) and mod-ask-core-service/helm/values.yaml (baked catalog):

Feature Where Notes
Foundry admin client overlay [mod-auh1-dev-ask.foundry] base_url / api_key provisions per-user keys; not a model ref
User-selectable model catalog helm/values.yaml model list (each entry's id) L2 — each id must be Foundry-served or aliased, else selecting it fails. Trim the catalog to served IDs, or add an alias per id.
Any name compiled into source hardcoded (no config key) L3 — fixable only via a Foundry alias for that exact name. None confirmed in this deployment (the upstream default.toml model names are L1 and overridden by our overlay).

Decision rule

  • Reference is in a values-…yaml you control → edit the value (L1).
  • Reference is a baked catalog id or hardcoded in source → add a Foundry alias (L2/L3) so the existing name routes to a served model. Don't fork the image just to rename a model.

After any L1 edit — apply it

cd mod-ask-<service> && git add -A && git commit -m "models: <change>" && git push
# config is mounted via subPath configmap → NOT hot-reloaded; roll the pods after sync
kubectl annotate application -n argocd mod-auh1-dev-ask.ask.<service> argocd.argoproj.io/refresh=hard --overwrite
argocd app sync mod-auh1-dev-ask.ask.<service> --grpc-web
kubectl rollout restart deploy/<service> -n ask

3. Test each model with curl and kubectl

Run from assistant-service-intelligence (has python3+httpx; app images lack curl). Set the key once:

KEY=$(kubectl get secret -n ask assistant-service-secret -o jsonpath='{.data.AI71_FOUNDRY_KEY}' | base64 -d)
B=http://foundry-litellm.foundry.svc.cluster.local:4000

Chat / VLM models (qwen3-5-122b-a10b, qwen3-5-4b, gpt-oss-120b, aliases)

kubectl exec -i -n ask deploy/assistant-service-intelligence -- env KEY="$KEY" python3 - <<'EOF'
import os, httpx
B="http://foundry-litellm.foundry.svc.cluster.local:4000"
for model in ["qwen/qwen3-5-122b-a10b","Qwen/Qwen3-4B-Instruct-2507","qwen/qwen3-5-122b-a10b"]:
    r=httpx.post(B+"/v1/chat/completions",
      headers={"Authorization":"Bearer "+os.environ["KEY"]},
      json={"model":model,"messages":[{"role":"user","content":"Say OK"}],"max_tokens":5},timeout=60)
    print(f'{model:32s} {r.status_code} {r.text[:80]!r}')
EOF
Expect 200 for each. A 400/404 means the model (or alias) isn't served — fix the name or the alias.

Embedding model (qwen3-embedding-0-6b)

kubectl exec -i -n ask deploy/assistant-service-intelligence -- env KEY="$KEY" python3 - <<'EOF'
import os, httpx
B="http://foundry-litellm.foundry.svc.cluster.local:4000"
r=httpx.post(B+"/v1/embeddings",
  headers={"Authorization":"Bearer "+os.environ["KEY"]},
  json={"model":"qwen/qwen3-embedding-0-6b","input":"hello"},timeout=60)
print(r.status_code, "dim=", len(r.json()["data"][0]["embedding"]) if r.status_code==200 else r.text[:120])
EOF
Expect 200 and dim= 1024 (must match the Weaviate vector dimension).

Reranker (llama-nemotron-rerank-1b-v2)

kubectl exec -i -n ask deploy/assistant-service-intelligence -- env KEY="$KEY" python3 - <<'EOF'
import os, httpx
B="http://foundry-litellm.foundry.svc.cluster.local:4000"
r=httpx.post(B+"/v1/rerank",
  headers={"Authorization":"Bearer "+os.environ["KEY"]},
  json={"model":"nvidia/llama-nemotron-rerank-1b-v2","query":"capital of UAE",
        "documents":["Abu Dhabi is the capital of the UAE","Bananas are yellow"]},timeout=60)
print(r.status_code, r.text[:200])
EOF
Expect 200 with results ranked (Abu Dhabi doc first).

VLM / KMS-dedicated instance (qwen3-5-122b-a10b-kms)

Test from a KMS pod (it holds the KMS Foundry key) — same chat call with model="qwen/qwen3-5-122b-a10b-kms": 

kubectl exec -i -n ask deploy/knowledge-management-service -- python3 - <<'EOF'
import os, httpx
key=os.environ.get("FOUNDRY__API_KEY") or os.environ.get("AI71_FOUNDRY_KEY")
B="http://foundry-litellm.foundry.svc.cluster.local:4000"
r=httpx.post(B+"/v1/chat/completions",headers={"Authorization":"Bearer "+key},
  json={"model":"qwen/qwen3-5-122b-a10b-kms","messages":[{"role":"user","content":"OK?"}],"max_tokens":5},timeout=60)
print(r.status_code, r.text[:120])
EOF

4. E2E testing without the frontend

The frontend is blocked on the air-gap image rebuild, so validate the platform via API calls per service. This proves every feature in the model matrix works end-to-end without a browser.

Service smoke (each service is up)

Each service has a different health path (core /api/v1/health/ready, KMS /health/, assistant /health on the intelligence/tooling pods) — a uniform path returns 404/401 even when the service is healthy:

kubectl exec -i -n ask deploy/assistant-service-intelligence -- python3 - <<'EOF'
import httpx
checks = {
  "core-service":      "http://core-service.ask.svc.cluster.local/api/v1/health/ready",
  "kms":               "http://knowledge-management-service.ask.svc.cluster.local/health/",
  "assistant-intel":   "http://assistant-service-intelligence:8003/health",
  "assistant-tooling": "http://assistant-service-tooling:8001/health/mcp",
}
for name,url in checks.items():
    try: print(f"{name:18s}", httpx.get(url,timeout=8).status_code)
    except Exception as e: print(f"{name:18s} ERR {e}")
EOF
Verified output (2026-06-22): 
core-service       200
kms                200
assistant-intel    200
assistant-tooling  200
All 200 = healthy. (A 404/401 from /api/v1/health/ready on KMS/assistant just means wrong path, not a down service — see E2E Stage 2.)

Feature matrix → how to exercise each (no frontend)

Feature (sheet) Model How to test E2E
Orchestrator qwen3-5-122b-a10b assistant-service chat run (create assistant → send message); or direct Foundry chat (§3)
Guardrails (policy + safety) Qwen/Qwen3-4B-Instruct-2507 (via qwen3-4b-instruct-2507 alias) send a message that trips a policy/keyword filter; confirm it's blocked. Model call = §3 small-LLM test
Data visualization tool see note trigger a data-viz tool run via assistant-service; confirm a chart payload returns
Data-analysis tools gpt-oss-120b trigger a data-analysis tool (profiler/query) via assistant-service tooling
KMS Embedding qwen3-embedding-0-6b KMS ingest a doc → confirm vectors written to Weaviate (1024-dim)
KMS OCR / VLM qwen3-5-122b-a10b-kms KMS ingest an image/PDF → confirm OCR text extracted
KMS Reranker llama-nemotron-rerank-1b-v2 KMS search query → confirm reranked results; model call = §3 rerank test
Mem0 / Voice / Meta-Agent NA (disabled) not deployed — confirm the features are off, not erroring

Data-viz model discrepancy — verify before relying on it

The tracker sheet lists the Data visualization tool → gpt-oss-120b, but the deployed config has [foundry] data_visualization_llm = qwen/qwen3-5-122b-a10b (and the [tooling.data_analysis] block uses gpt-oss-120b). Confirm which path the UI's "data visualization" actually uses and align the config to the sheet if needed — don't assume.

Per-service API references (the real endpoints)

These already-documented runbooks have the concrete request bodies for each service's flows — use them as the no-frontend test scripts:

Order of E2E validation

  1. Foundry models green (§3) → 2. service /health/ready green → 3. core-service discovery/login →
  2. KMS ingest+search → 5. assistant chat + each tool. If a feature fails, isolate whether it's the model (re-run §3 for that model/alias) or the service (check the service logs).