ASK Assistant Service¶

Repo: ask-assistant-service-main
Language: Python ≥ 3.12
Framework: FastAPI + Gunicorn/UvicornWorker
Helm Chart: assistant-service v0.2.2
Ports: 8000 (assistants), 8001 (tooling), 8003 (intelligence), 8080 (MCP gateway)

Architecture¶

The Assistant Service is a multi-image, multi-process service that implements the AI assistant layer of the ASK platform. It is split into four distinct containers, each serving a specific role in the assistant pipeline:

                    ┌─────────────────────────┐
                    │  Assistants API          │
                    │  (port 8000)             │
                    │  Dockerfile              │
                    └──────────┬──────────────┘
                               │
              ┌────────────────┼────────────────┐
              ▼                ▼                ▼
  ┌─────────────────┐  ┌─────────────┐  ┌─────────────────┐
  │  Intelligence   │  │  Tooling    │  │  MCP Gateway    │
  │  (port 8003)    │  │  (port 8001)│  │  (port 8080)    │
  │  Dockerfile-    │  │  Dockerfile-│  │  Dockerfile-    │
  │  intelligence   │  │  tooling    │  │  gateway        │
  └─────────────────┘  └─────────────┘  └─────────────────┘
              │                │
              ▼                ▼
        PostgreSQL           Redis
         (external)        (Bitnami subchart
                            or external)

Additionally, two sandbox environments are managed via Daytona:

  ┌──────────────────┐    ┌────────────────────┐
  │  Daytona Runner  │    │  Daytona Sandbox    │
  │  daytona/apps/   │    │  daytona/apps/      │
  │  runner/         │    │  sandbox/           │
  └──────────────────┘    └────────────────────┘

Container Images¶

Multi-Stage Build Pattern¶

Stage	Base Image	Notes
`builder`	`python:3.12-trixie`	Install deps via `uv sync` with 4 private GitLab indexes
`runtime`	`python:3.12-slim-trixie`	Minimal Debian Trixie base

The DEPENDENCY_PROXY_PREFIX ARG allows routing through a registry mirror.

Images Produced¶

Image Name	Dockerfile	Port	Worker Config	Purpose
`assistant-service`	`Dockerfile`	8000	1–4 Gunicorn workers, 300s timeout	Core Assistants API
`assistant-service-intelligence`	`Dockerfile-intelligence`	8003	1 Gunicorn worker, 300s timeout	LLM orchestration (LangGraph), higher memory (10Gi limit)
`assistant-service-tooling`	`Dockerfile-tooling`	8001	1 Gunicorn worker, 300s timeout	ToolBus — tool execution, code execution
`assistant-service-mcp-gateway`	`Dockerfile-gateway`	8080	Uvicorn	MCP Gateway — routes MCP protocol calls

All images push to:

ai71uaenprodask71acr01.azurecr.io/assistant-service[:<suffix>]:<tag>

MCP Gateway — Additional Runtime Components¶

The gateway container installs Node.js 20 at runtime and pre-caches:

npm Package	Version	Purpose
`supergateway`	3.4.3	MCP protocol gateway
`@modelcontextprotocol/server-filesystem`	latest (pinned)	Filesystem MCP server

Daytona Sandbox Image¶

The daytona/apps/sandbox/Dockerfile builds a rich execution environment:

Base: daytonaio/sandbox:0.5.0-slim

System packages added: git, curl, gnupg, python3, libreoffice, pandoc, poppler-utils, qpdf, tesseract-ocr, gcc, build-essential, Node.js 20, pnpm, docx, pptxgenjs

Key Python packages in sandbox: - anthropic SDK — direct Claude API calls - mcp — MCP protocol - daytona — Daytona runtime SDK - PDF/image processing: pypdf2, pillow, pdf2image, pytesseract - Document: python-docx, openpyxl, markdown - Data: pandas, numpy, matplotlib, seaborn

Daytona Runner Image¶

Base: daytonaio/daytona-runner
Adds: mount-s3 binary, FUSE kernel module support (for S3 filesystem mounts inside sandboxes)

Build Arguments¶

ARG	Default	Description
`DEPENDENCY_PROXY_PREFIX`	`""`	Registry mirror prefix
`GITLAB_USERNAME`	—	Username for private PyPI indexes

Build secrets required: tii_gitlab_username, tii_gitlab_password, gitlab_password

Helm Chart¶

Chart: helm/Chart.yaml

name: assistant-service
version: "0.2.2"
appVersion: "0.1.0"

Chart Dependencies (Subcharts)¶

Subchart	Version	Source	Condition
`redis`	22.0.7	`oci://registry-1.docker.io/bitnamicharts`	`redis.enabled`

The Redis subchart provides an optional in-cluster Redis. In production, Redis is external (Azure Cache for Redis); the subchart is disabled and an external connection string is used.

Resource Limits (representative)¶

Deployment	CPU Request	Memory Request	Memory Limit
Main (assistants)	300m	500Mi	768Mi
Intelligence	500m	2Gi	10Gi
Tooling	300m	500Mi	2Gi
MCP Gateway	100m	256Mi	512Mi
Generic Worker	300m	500Mi	1Gi

Environments¶

Environment File	Environment	Cloud/DC	Notes
`values-ai71-uaen-dev-ask71.yaml`	Development	Azure UAE North	Primary dev cluster
`values-ai71-uaen-dev-ask71-preview.yaml`	Dev Preview	Azure UAE North	Feature branch preview envs
`values-ai71-uaen-stg-ask71.yaml`	Staging	Azure UAE North	Pre-prod
`values-ai71-uaen-prod-mufti.yaml`	Production (Mufti)	Azure UAE North	Production tenant A
`values-ai71-pp-auh1-prod-ask.yaml`	PP Production	On-prem Abu Dhabi	Gov cloud, air-gapped
`values-edge-uaen-prod-ask.yaml`	Edge Production	Edge UAE North	Edge deployment
`values-ai71-modgpt-auh1-dev-ask.yaml`	core42 Dev	On-prem RKE2 (core42)	🔄 In progress — see below

PP Environment — Assistant Service¶

Full PP environment details: PP Environment page

PP Images¶

Container	PP Registry Path
Main (assistants API)	`registry.pp.gov.ae/ask/ask-assistant-service`
Redis subchart	`registry.pp.gov.ae/bitnamilegacy/redis:8.2.1-debian-12-r0`

imagePullSecrets: harbor-registry-secret

PP Non-Secret Configuration (TOML)¶

Config Key	PP Value	Notes
`app.env`	`ai71-pp-auh1-prod-ask`
`app.base_url`	`http://api.ask.pp.gov.ae`
`app.fastapi_root_path`	`/ask71/v2/astsvc`
`logging.log_level`	`INFO`
`logging.log_sample_rate`	`0.0`	Sampling disabled
`core_service.url`	`http://core-service.ask.svc.cluster.local/api/v1`	In-cluster
`core_service.services_base_url`	`http://core-service.ask.svc.cluster.local`
`database.host`	`postgresql.ask.svc.cluster.local`	Shared in-cluster PostgreSQL
`database.user`	`assistant-service`
`foundry.endpoint`	`http://litellm.foundry.svc.cluster.local:4000/v1`	In-cluster LiteLLM
`foundry.reranker.endpoint`	`http://litellm.foundry.svc.cluster.local:4000/v1/rerank`
`storage.cdn_base_url`	`https://cdn.ask.pp.gov.ae/static`
`kms.base_url`	`http://knowledge-management-service.ask.svc.cluster.local`
`hatchet.client_tls_strategy`	`none`	TLS disabled for in-cluster Hatchet
`hatchet.generic_worker_slots`	`5`
`redis.host`	`assistant-service-redis-master.ask.svc.cluster.local`
`daytona.enable_code_execution`	`false`	Disabled — no Daytona in PP
`daytona.api_key`	`@null`
`tool_approval.flow_enabled`	`false`
`feature_flags.mode`	`offline`	No LaunchDarkly/Unleash in PP
`feature_flags.bootstrap_file`	`./config/feature-flags/pp-prod.json`	Bundled in image
`assistants.frontend_url`	`https://ask.pp.gov.ae/`
`assistants.service_url`	`https://api.ask.pp.gov.ae/ask71/v2/astsvc`
`assistants.intelligence_base_url`	`http://assistant-service-intelligence:8003`
`assistants.tooling_base_url`	`http://assistant-service-tooling:8001`
`assistants.default_public_assistant_name`	`mujeeb`	PP-specific default assistant
`tooling.data_analysis.*_model`	`qwen/qwen3-5-122b-a10b`	OSS on-prem model (vs cloud GPT-5 elsewhere)
`postDeployJob.enabled`	`false`

PP Secrets (from Vault `kv/data/ask/prod/assistant-service/environment-secrets`)¶

Secret Key	Description
`DATABASE__PASSWORD`	PostgreSQL password
`HATCHET__CLIENT_TOKEN`	Hatchet client token
`CORE_SERVICE__JWT_SECRET`	JWT secret for Core Service
`CORE_SERVICE_ENCRYPTION_KEY`	Core Service encryption key
`AI71_FOUNDRY_KEY`	LiteLLM API key
`AI71_AUTH_STATE_SECRET`	OAuth state secret
`REDIS_PASSWORD`	Redis password

PP Ingress¶

Host	Path	Rewrite	Timeout
`api.ask.pp.gov.ae`	`/ask71/v2/astsvc(/\|$)(.*)`	`/$2`	proxy read/send 420s

TLS: ask-pp-gov-ae-cert.

Key PP Differences¶

Data analysis models: All four data analysis models use qwen/qwen3-5-122b-a10b (on-prem OSS) instead of cloud default gpt-5-chat
Daytona disabled: No code execution sandbox in PP

core42 Environment — Assistant Service (`ai71-modgpt-auh1-dev-ask`)¶

core42 Images¶

Container	core42 Registry Path
Main	`harbor.cl1.sq4.aegis.internal/ask/assistant-service:<git-sha>`
MCP Gateway	`harbor.cl1.sq4.aegis.internal/ask/assistant-service-mcp-gateway:<git-sha>`
Redis subchart	`harbor.cl1.sq4.aegis.internal/bitnamilegacy/redis:8.2.1-debian-12-r0`

imagePullSecrets: harbor-pull-secret

core42 TOML Config Changes vs PP¶

Config Key	PP Value	core42 Value
`app.env`	`ai71-pp-auh1-prod-ask`	`ai71-modgpt-auh1-dev-ask`
`app.base_url`	`http://api.ask.pp.gov.ae`	`https://api.cl1.sq4.aegis.internal`
`storage.cdn_base_url`	`https://cdn.ask.pp.gov.ae/static`	`https://cdn.cl1.sq4.aegis.internal/static`
`assistants.frontend_url`	`https://ask.pp.gov.ae/`	`https://ask.cl1.sq4.aegis.internal/`
`assistants.service_url`	`https://api.ask.pp.gov.ae/ask71/v2/astsvc`	`https://api.cl1.sq4.aegis.internal/ask71/v2/astsvc`
All other keys	—	✅ Same as PP (in-cluster service URLs unchanged)

configTomlFileMountPath: /app/config/ai71-modgpt-auh1-dev-ask.toml

core42 Vault Secrets¶

Vault path: secret/cl1/assistant-service

Env Var	Vault Key	Description
`HATCHET__CLIENT_TOKEN`	`hatchet_client_token`	Hatchet workflow token
`DATABASE__PASSWORD`	`db_password`	PostgreSQL password
`REDIS_PASSWORD`	`redis_password`	Redis password

core42 Ingress¶

Host	Path
`api.cl1.sq4.aegis.internal`	`/ask71/v2/astsvc(/\|$)(.*)`
- Feature flags offline: Bootstrap JSON file bundled in the image at `./config/feature-flags/pp-prod.json`
- Default assistant name: `mujeeb` (PP-specific branding)

Configuration Variables¶

Non-Secret Config (representative, from dev values)¶

Variable	Example (dev)	Description
`DATABASE__HOST`	`ai71-uaen-dev-ask71-psql-01.postgres.database.azure.com`	PostgreSQL hostname
`REDIS_HOST`	(in-cluster)	Redis host
`CORE_SERVICES_BASE_URL`	`http://core-service`	Core Service in-cluster URL
`FOUNDRY__BASE_URL`	`https://foundry.core.uaen.ai71services.ai/v1`	AI71 Foundry LLM endpoint
`STORAGE__CDN_URL`	`https://cdn.ask.dev.ai71services.ai/static`	Static assets CDN
`KMS__BASE_URL`	`https://api.ask.dev.ai71.ai/ask71/v2/kmsvc`	Key Management Service
`DAYTONA__BASE_URL`	`https://daytona.ask.dev.ai71services.ai/api`	Daytona sandbox API
`APP_ENV`	`dev`	Dynaconf environment name
`GUNICORN_WORKERS`	`1`	Number of Gunicorn workers
`PORT`	`8000`	Container listen port

Secrets¶

Secrets are injected via Azure Key Vault + External Secrets Operator. Pattern: {release-name}--{SECTION__KEY} in Key Vault.

Environment Variable	Description
`AI71_FOUNDRY_KEY`	AI71 Foundry API key
`AI71_AUTH_STATE_SECRET`	OAuth state encryption secret
`DATABASE__PASSWORD`	PostgreSQL password
`REDIS_PASSWORD`	Redis AUTH password
`CORE_SERVICE_ENCRYPTION_KEY`	Encryption key for core service tokens
`CORE_SERVICE__JWT_SECRET`	JWT signing secret for inter-service calls
`IAM__MACHINE_TOKEN`	Zitadel machine account token

Local development uses 1Password CLI for secret injection (op://ask-assistant-service-nonprod/local/<key>). The .env.tpl file is processed by op inject to produce .env.

ExternalSecret refresh interval: 1 minute.

Key Dependencies¶

Python Packages¶

Package	Version	Purpose
`fastapi`	0.128	HTTP framework
`uvicorn`	0.35	ASGI server
`gunicorn`	—	Production WSGI/ASGI process manager
`sqlalchemy`	2.0.38	ORM (async)
`asyncpg`	—	Async PostgreSQL driver
`alembic`	1.15.1	Database migrations
`dynaconf`	3.2.12	Configuration management
`pydantic-settings`	2.10.1	Settings validation
`openai`	2.32	OpenAI-compatible client (used with Foundry)
`langchain-core`	0.3.72	LangChain core abstractions
`langgraph`	0.6.11	Agent workflow graphs
`fastmcp`	3.2.4	Fast MCP framework
`mcp`	1.23	Model Context Protocol SDK
`hatchet-sdk`	1.28	Async job queue
`document_understanding`	0.1.88	Document parsing (private)
`storage-lib`	—	Azure/MinIO storage SDK (private)
`ask-feature-flags`	1.1.0	Feature flag SDK (private)
`sentry-sdk`	2.48+	Error tracking (fastapi, langgraph, langchain, sqlalchemy, redis, httpx integrations)
`spacy`	3.8.7	NLP pipeline
`fasttext`	0.9.3	Language detection
`nltk`	3.9.3	NLP toolkit
`mem0ai`	2.0	Agent memory (NLP extras)
`fastembed`	0.8	Local embedding models
`mlflow`	3.4	ML experiment tracking
`tavily-python`	0.7.12	Web search API client
`arxiv`	2.2.0	arXiv paper search
`youtube-transcript-api`	1.2.2	YouTube transcript fetching
`python-docx`	1.1.2	Word document generation
`reportlab`	4.0	PDF generation
`kubernetes`	31.0	Kubernetes API client (Daytona integration)
`authlib`	1.6.9	OAuth 2.0 / OIDC
`PyJWT`	2.9	JWT encoding/decoding
`polars`	1.0	Fast DataFrame library
`json-repair`	0.52.4	LLM JSON output repair
`thefuzz`	0.22.1	Fuzzy string matching
`evals-sdk`	0.8	Evaluation pipeline SDK (private)
`arabic-reshaper`	3.0	Arabic text rendering
`python-bidi`	0.4.2	Bidirectional text support
`langcodes`	3.5	Language code handling

Private GitLab PyPI Indexes¶

Index Name	Package Provided
`ask-group` (TII GitLab)	`document_understanding`, `storage-lib`, `connectors-lib`
`core-group` (TII GitLab)	`iam-client`, `foundry-client`, `logger-sdk`
`gitlab-eval`	`evals-sdk`
`ask-feature-flags`	`ask-feature-flags`

CI/CD Pipeline¶

File: .gitlab-ci.yml
Tool: GitLab CI

Stage	Key Jobs	Description
`validate`	pre-commit, pytest, security, migration-checks	Code quality, tests, Alembic migration validation
`build`	`docker-acr-build`	Build 4 Docker images (main + intelligence + tooling + gateway) and push to ACR
`helm-release`	`helm-release`	Publish chart to GitLab OCI registry
`.post`	cleanup	Remove temp resources

Key CI variables:

Variable	Value
`UV_VERSION`	`0.9.21`
`PYTHON_VERSION`	`3.12`
`BASE_LAYER`	`bookworm`

ArgoCD dev deployment is triggered via .gitlab/ci/argocd-dev-deploy.yaml on merge to main.