ASK Knowledge Management Service¶

Repo: ask-knowledge-management-service-main
Language: Python ≥ 3.12
Framework: FastAPI + Gunicorn
Helm Chart: knowledge-management-service v1.0.0
Port: 8000

Architecture¶

The Knowledge Management Service (KMS) handles document ingestion, vector embedding, and semantic search for the ASK platform. It manages a multi-database stack (PostgreSQL for relational data, Weaviate for vector search) and runs several parallel worker processes for different ingestion pipelines.

┌──────────────────────────────────────────────────────────┐
│            knowledge-management-service (port 8000)      │
│                     Gunicorn workers                     │
├──────────────┬─────────────────┬────────────────────────┤
│  Ingestion   │  Generic        │  Live Ingestion        │
│  Worker      │  Worker         │  Worker                │
└──────────────┴─────────────────┴────────────────────────┘
       │                │                │
       ▼                ▼                ▼
  PostgreSQL         Weaviate         MinIO / Azure Blob
  (Bitnami sub-    (Weaviate sub-     (object storage)
   chart or ext.)   chart or ext.)

Sync Wave Deployment Order¶

Helm sync waves ensure ordered startup:

Wave 0 — ConfigMap creation
Wave 1 — PostgreSQL init (if subchart enabled)
Wave 2 — Weaviate startup
Wave 3 — Alembic migration Job
Wave 4 — Application Deployments

Container Images¶

Multi-Stage Build Pattern¶

Stage	Base Image	Purpose
`builder`	`ghcr.io/astral-sh/uv:0.10.2-python3.12-trixie`	Install deps via `uv sync` with 5 private GitLab indexes
`runtime`	`python:3.12-slim-trixie`	Minimal Debian Trixie runtime

The KMS uses the uv official image as the builder (vs python:3.12 + pip install uv in other services) — this is the recommended approach for faster builds.

Images Produced¶

Image Name	Dockerfile	Port	Purpose
`ask-knowledge-management-service`	`Dockerfile`	8000	Main FastAPI + Gunicorn
`ask-knowledge-management-service-ci-runner`	`Dockerfile.ci-runner`	—	CI jobs requiring Docker-in-Docker and PostgreSQL client

Production registry:

ai71uaenprodask71acr01.azurecr.io/ask-knowledge-management-service:<tag>

CI Runner Image¶

Base: ghcr.io/astral-sh/uv:${UV_VERSION}-python${PYTHON_VERSION}-${BASE_LAYER} (default: 0.10.2-3.12-trixie)

Additional packages: docker-ce-cli, docker-compose-plugin, docker-buildx-plugin, PostgreSQL client (psql)

Build Arguments¶

ARG	Default	Description
`DEPENDENCY_PROXY_PREFIX`	`""`	Registry mirror prefix
`GITLAB_USERNAME`	`__token__`	Username for private PyPI indexes

Build secret required: gitlab_password

Local Development (docker-compose)¶

services:
  postgresql:   # PostgreSQL 15
  weaviate:     # Weaviate vector DB
  minio:        # MinIO object storage

Helm Chart¶

Chart: helm/Chart.yaml

name: knowledge-management-service
version: "1.0.0"
appVersion: "0.1.0"

Chart Dependencies (Subcharts)¶

Subchart	Version	Source	Condition
`postgresql`	16.4.1	`https://charts.bitnami.com/bitnami`	`postgresql.enabled`
`weaviate`	17.7.0	`https://weaviate.github.io/weaviate-helm`	`weaviate.enabled`

In production environments, both PostgreSQL and Weaviate are typically deployed as standalone services (Azure Database for PostgreSQL Flexible Server + standalone Weaviate cluster) and the subcharts are disabled. The subcharts are used in development/CI.

Environments¶

Environment File	TOML Config	Environment	Cloud/DC
`values-ai71-uaen-dev-ask71.yaml`	`ai71-uaen-dev-ask71.toml`	Development	Azure UAE North
`values-ai71-uaen-stg-ask71.yaml`	`ai71-uaen-stg-ask71.toml`	Staging	Azure UAE North
`values-ai71-uaen-prod-mufti.yaml`	`ai71-uaen-prod-mufti.toml`	Production (Mufti)	Azure UAE North
`values-ai71-pp-auh1-prod-ask.yaml`	`ai71-pp-auh1-prod-ask.toml`	PP Production	On-prem Abu Dhabi
`values-edge-uaen-prod-ask.yaml`	`edge-uaen-prod-ask.toml`	Edge Production	Edge UAE North
`values-ai71-modgpt-auh1-dev-ask.yaml`	`ai71-modgpt-auh1-dev-ask.toml`	core42 Dev	On-prem RKE2 (core42)

PP Environment — Knowledge Management Service¶

Full PP environment details: PP Environment page

PP Image¶

registry.pp.gov.ae/ask/ask-knowledge-management-service:<tag>

imagePullSecrets: harbor-registry-secret

PP Non-Secret Configuration (TOML — `ai71-pp-auh1-prod-ask.toml`)¶

Config Key	PP Value	Notes
`core_service.services_base_url`	`http://core-service.ask.svc.cluster.local`	In-cluster
`database.host`	Dynamic: `{{ .Release.Name }}-postgresql-primary`	Injected from Helm via `DATABASE__HOST` env var
`database.port`	`5432`
`database.name`	`knowledge_management`
`database.user`	`kms_user`
`database.db_schema`	`knowledge_service`
`database.pool_size`	`200`	High pool — 5+ ingestion workers
`database.max_overflow`	`10`
`document_understanding.di_resource_name`	`azure-document-intelligence-pp-prod`	Azure DI still used in PP
`document_understanding.foundry_endpoint`	`http://litellm.foundry.svc.cluster.local/v1`	In-cluster LiteLLM
`organization.vlm_model_name`	`Qwen/Qwen3.5-9B`	On-prem VLM
`storage.type`	`s3`	S3-compatible object store — not Azure Blob
`storage.s3.endpoint_url`	`https://s3.pp.gov.ae/`	PP internal S3
`storage.s3.bucket_name`	`ask-knowledge-management`
`storage.s3.region`	`us-east-1`	(required field even for non-AWS)
`storage.s3.verify_ssl`	`false`	Private CA cert
`vector_db.http_host`	`weaviate.weaviate.svc.cluster.local`	In-cluster Weaviate
`vector_db.http_port`	`80`
`vector_db.grpc_host`	`weaviate-grpc.weaviate.svc.cluster.local`
`live_file.parsing_strategy`	`generic`

PP Secrets (from Vault `kv/data/ask/prod/knowledge-management-service/environment-secrets`)¶

Secret Key	Description
`DATABASE__PASSWORD`	PostgreSQL password for `kms_user`
`DATABASE__HOST`	Injected as env var (Helm dynamic host)
`APP__MASTER_ENCRYPTION_KEY`	Master encryption key
`CORE_SERVICE__JWT_SECRET`	JWT secret
`DOCUMENT_UNDERSTANDING__DI_SUBSCRIPTION_KEY`	Azure Document Intelligence key
`DOCUMENT_UNDERSTANDING__FOUNDRY_KEY`	LiteLLM / Foundry key for DU
`DOCUMENT_UNDERSTANDING__KEYWORD_EXTRACTION__NER_TOKEN`	NER model auth token
`STORAGE__S3__ACCESS_KEY_ID`	S3 access key for `s3.pp.gov.ae`
`STORAGE__S3__SECRET_ACCESS_KEY`	S3 secret key
`VECTOR_DB__API_KEY`	Weaviate API key

PP PostgreSQL Subchart (Bitnami `18.1.0`, in-cluster)¶

The KMS deploys its own PostgreSQL (unlike other services that share the platform PostgreSQL).

Setting	Value
Chart	`bitnami/postgresql:18.1.0`
`fullnameOverride`	`knowledge-management-service-postgresql`
Architecture	Primary + 1 read replica
Storage class	`longhorn`
Primary storage	300 Gi
Replica storage	300 Gi
Primary memory limit	16 Gi
Primary CPU request	500m
Image registry	`registry.pp.gov.ae`
Image repo	`bitnami/postgresql`
`auth.database`	`knowledge_management`
`auth.username`	`kms_user`
`auth.existingSecret`	`knowledge-management-service-secret`
`max_connections`	300
`shared_buffers`	4 GB
`work_mem`	16 MB
Idle transaction timeout	10 min
Idle session timeout	15 min

PP S3 CA Bundle¶

The S3 endpoint (s3.pp.gov.ae) uses a private CA. A CA certificate bundle is mounted into the pod:

volumes:
  - name: ca-bundle
    secret:
      secretName: s3-ca-bundle
      items:
        - key: s3-cert.pem
          path: s3-cert.pem
volumeMounts:
  - name: ca-bundle
    mountPath: /config/s3/s3-cert.pem
    subPath: s3-cert.pem
    readOnly: true

The Secret s3-ca-bundle must be pre-created in the ask namespace before deploying.

PP Ingestion Autoscaling¶

ingestionAutoscaling.minReplicas: 5 — minimum 5 ingestion worker replicas to handle the PP document load.

PP Ingress¶

Host	Path	Rewrite
`api.ask.pp.gov.ae`	`/ask71/v2/kmsvc(/\|$)(.*)`	`/$2`

TLS: ask-pp-gov-ae-cert.

Key PP Differences¶

Storage backend is S3 (s3.pp.gov.ae) not Azure Blob — requires STORAGE__S3__* secrets and S3 CA bundle mount
Own PostgreSQL subchart with Longhorn persistent volumes (not shared platform DB)
VLM model: Qwen/Qwen3.5-9B (on-prem) instead of cloud VLM
DATABASE__HOST is dynamic — set via extraEnv Helm value from the release name

core42 Environment — KMS (`ai71-modgpt-auh1-dev-ask`)¶

core42 Image¶

harbor.cl1.sq4.aegis.internal/ask/ask-knowledge-management-service:<git-sha>

imagePullSecrets: harbor-pull-secret

core42 TOML Config Changes vs PP¶

Config Key	PP Value	core42 Value
`storage.s3.endpoint_url`	`https://s3.pp.gov.ae/`	`https://s3.mod.auh1.dev.dir` (Ceph RGW VIP)
`storage.s3.verify_ssl`	`false`	`false` ✅ (HTTP — no SSL)
All other keys	—	✅ Same as PP

configTomlFileMountPath: /app/config/ai71-modgpt-auh1-dev-ask.toml

core42 Vault Secrets¶

Vault path: secret/cl1/knowledge-management-service

Env Var	Vault Key	Description
`DATABASE__PASSWORD`	`db_password`	PostgreSQL password for kms_user
`STORAGE__S3__ACCESS_KEY_ID`	`storage_s3_access_key_id`	Ceph RGW S3 access key
`STORAGE__S3__SECRET_ACCESS_KEY`	`storage_s3_secret_access_key`	Ceph RGW S3 secret key

core42 PostgreSQL Subchart Changes¶

Setting	PP Value	core42 Value
Image registry	`registry.pp.gov.ae`	`harbor.cl1.sq4.aegis.internal`
Image tag	`18.1.0`	`17.6.0-debian-12-r4`
Storage class	`longhorn`	`csi-rbd-sc` (Ceph RBD)
Primary storage	`300Gi`	`50Gi` (current sizing)
Replica storage	`300Gi`	`50Gi`

core42 S3 CA Bundle¶

Not needed — Ceph RGW runs on HTTP (https://s3.mod.auh1.dev.dir). Remove volumes and volumeMounts from values:

volumes: []
volumeMounts: []

core42 Ingress¶

Host	Path
`api.cl1.sq4.aegis.internal`	`/ask71/v2/kmsvc(/\|$)(.*)`

Configuration Variables¶

Non-Secret Config (representative)¶

Variable	Example	Description
`APP__ENV`	`dev`	Dynaconf environment
`DATABASE__HOST`	(PostgreSQL FQDN)	PostgreSQL hostname
`DATABASE__PORT`	`5432`	PostgreSQL port
`DATABASE__NAME`	`kms`	Database name
`VECTOR_DB__URL`	(Weaviate URL)	Weaviate endpoint
`VECTOR_DB__CLASS_PREFIX`	`Ask`	Weaviate class name prefix
`STORAGE__PROVIDER`	`azure` or `minio`	Object storage backend
`STORAGE__AZURE__ACCOUNT_NAME`	(Azure storage account)	Azure Blob account
`STORAGE__MINIO__ENDPOINT`	(MinIO URL)	MinIO endpoint
`CORE_SERVICE__BASE_URL`	`http://core-service`	Core Service in-cluster URL
`DOCUMENT_UNDERSTANDING__ENDPOINT`	(Azure AI endpoint)	Document Intelligence endpoint
`MLFLOW__TRACKING_URI`	(MLflow server URL)	MLflow tracking server

Secrets¶

Secrets are injected via Azure Key Vault + External Secrets Operator. Pattern: {release-name}--{SECTION__KEY} in Key Vault.

Environment Variable	Description
`APP__MASTER_ENCRYPTION_KEY`	Master encryption key for stored data
`CORE_SERVICE__JWT_SECRET`	JWT secret for inter-service authentication
`DATABASE__PASSWORD`	PostgreSQL password
`DOCUMENT_UNDERSTANDING__DI_SUBSCRIPTION_KEY`	Azure Document Intelligence subscription key
`DOCUMENT_UNDERSTANDING__FOUNDRY_KEY`	AI71 Foundry key for document processing
`DOCUMENT_UNDERSTANDING__KEYWORD_EXTRACTION__NER_TOKEN`	NER model auth token
`STORAGE__AZURE__ACCOUNT_KEY`	Azure Blob Storage account key
`STORAGE__MINIO__ACCESS_KEY`	MinIO access key
`STORAGE__MINIO__SECRET_KEY`	MinIO secret key
`VECTOR_DB__API_KEY`	Weaviate API key
`TESTING__USER_PASSWORD`	Test fixture user password (dev/CI only)
`MLFLOW__TRACKING_TOKEN`	MLflow tracking server auth token (optional)
`MLFLOW__AZURE_STORAGE_CONNECTION_STRING`	Azure storage for MLflow artifacts (optional)
`EVALS__ACCOUNT_KEY`	Evaluation pipeline storage key (optional)

Local development uses 1Password CLI for secret injection (op://ask-knowledge-management-service-nonprod/local/<key>).

ExternalSecret refresh interval: 1 minute.

Key Dependencies¶

Python Packages¶

Package	Version	Purpose
`fastapi`	0.135.2	HTTP framework
`uvicorn-worker`	0.4	ASGI worker for Gunicorn
`gunicorn`	—	Process manager
`sqlalchemy[asyncio]`	2.0.48	ORM (async)
`asyncpg`	0.31	Async PostgreSQL driver
`psycopg2-binary`	2.9.11	Sync PostgreSQL driver
`alembic`	1.18.4	Database schema migrations
`sqlalchemy-utils`	0.41	SQLAlchemy helper utilities
`sqlglot`	30.1	SQL parsing and transpilation
`weaviate-client`	4.20.4	Weaviate vector database client
`document-understanding`	0.4.3	Document parsing SDK (private)
`iam-client`	2.1.4	Zitadel IAM SDK (private)
`foundry-client`	0.1.7	AI71 Foundry SDK (private)
`storage-lib`	0.2.2	Azure/MinIO storage SDK (private)
`connectors-lib`	0.2.8	External connector integrations (private)
`hatchet-sdk`	1.33.5	Async job queue
`dynaconf`	3.2.13	Configuration management
`pydantic`	2.12.5	Data validation
`mlflow`	3.4+	ML experiment tracking
`scikit-learn`	1.8	ML utilities
`numpy`	2.4.4	Numerical computing
`mcp`	1.27	Model Context Protocol SDK
`fastmcp`	3.2	Fast MCP framework
`opentelemetry-sdk`	1.40	Distributed tracing
`sentry-sdk`	2.57	Error tracking
`httpx`	0.28.1	Async HTTP client
`cryptography`	46.0.7	Encryption utilities
`redis`	7.4	Redis client
`oracledb`	3.4.2	Oracle database driver (connector support)

Private GitLab PyPI Indexes¶

Index Name	Package Provided
`CONNECTORS_LIB`	External connector integrations
`DOCUMENT_UNDERSTANDING`	Document parsing SDK
`IAM_CLIENT`	Zitadel IAM SDK
`STORAGE_LIB`	Azure/MinIO storage abstraction
`FOUNDRY_CLIENT`	AI71 Foundry LLM proxy SDK

CI/CD Pipeline¶

File: .gitlab-ci.yml
Tool: GitLab CI

Stage	Key Jobs	Description
`validate`	`env-validation`, `migration-validation`, `performance`, `pre-commit`, `pytest`, `security-ci`, `sonarcloud`	Full validation: schema checks, migration tests, load tests, lint, unit tests, security scan, code quality
`build`	`docker-build`, `docker-size-check`	Build main + ci-runner images; validate image size compliance
`helm-release`	`helm-release`	Publish chart to GitLab OCI registry

Key CI variables:

Variable	Value
`UV_VERSION`	`0.10.2`
`PYTHON_VERSION`	`3.12`
`BASE_LAYER`	`trixie`

The docker-size-check job enforces image size limits to prevent accidental dependency bloat.