Architecture: what AltaForge RA is

AltaForge Research Assistant (RA) is a multi-tenant Retrieval-Augmented Generation (RAG) API. A caller sends a natural-language question scoped to a document category; RA retrieves the relevant document chunks, has an LLM synthesise a grounded answer with inline citations, and streams the result back. One deployment serves many organizations ("tenants"), each with its own documents, configuration, and prompt overrides.

The backend is a Python 3.11+ FastAPI application (package altaml_research_assistant), built on the LogicStream agent-workflow framework and LangGraph. Document ingestion and embedding run out-of-band in a separate Azure Function (rag_pipeline/); this page is about the API service under backend/.

What a request looks like

There are two request shapes, because a research query and an administrative operation are served differently.

Administrative / lookup requests (REST)

Ordinary REST calls — list documents, read categories, manage escalation routing, read or write an organization's config — flow through the standard middleware stack:

1. AuthMiddleware (middleware.py) resolves the tenant from the X-Organization-Id header, authenticates the Authorization: Bearer <jwt> token against that tenant, and attaches the resulting user (request.state.user) and org id (request.state.organization_id).
2. Role gate. Paths under /admin require the App.Admin.FullAccess role; paths under /platform require App.Platform.Operator. Both checks live in AuthMiddleware.dispatch.
3. The matching router handler (routers/*.py) runs, scoped to that org.

Research queries (WebSocket)

A research query runs as a LogicStream workflow over a WebSocket, not a REST call, so partial results can stream to the client as they are produced:

1. The client opens ws://<host>/workflow?token=<jwt>&organization_id=<uuid> (browsers cannot set custom headers on the WebSocket handshake, so the token and org id are query parameters — see routers/workflows.py::run_workflow).
2. authenticate_websocket (middleware.py) validates the token and org id.
3. The client sends one StartWorkflow JSON frame carrying a WorkflowInput payload. The server overrides the client-supplied user_id, organization_id, and is_virtual_user with the authenticated identity so the save and load paths always agree.
4. The langgraph-rag-chat workflow executes and streams WorkflowEvent frames back until it completes.

The RAG workflow (LangGraph)

The default workflow id is langgraph-rag-chat (routers/workflows.py). It is a LangGraph StateGraph compiled once at module load and reused across requests. Every optional node is present in the compiled graph; which nodes actually run is a per-request, per-organization decision resolved at runtime through ConfigService — one compiled graph serves every tenant with independently configured behaviour (workflows/langgraph_rag_chat.py).

Two retrieval modes coexist, selected per request via WorkflowInput.search_mode (workflows/rag_chat_models.py):

Mode	Path	Behaviour
azure_ai_search (default)	retrieve (or multi_query_retrieve) → assistant → END	Hybrid semantic + vector retrieval against Azure AI Search, then answer synthesis.
agentic	agentic_retrieve → END	A single agent runs a plan↔tools loop (search / read) over a per-category text index, and writes the final answer itself.

When search_mode is omitted, the workflow resolves it to the org's DEFAULT_SEARCH_MODE override, falling back to the deploy-level default (azure_ai_search).

Optional nodes on the Azure AI Search path, each gated by a per-org flag:

• Query rewriting (ENABLE_QUERY_REWRITING) — rewrites the question into retrieval-optimised language before retrieval.
• Multi-query retrieval (ENABLE_MULTI_QUERY_RETRIEVAL) — fans out N reformulations in parallel, then deduplicates and merges.
• Escalation intent classification (ENABLE_ESCALATION_WORKFLOW) — detects queries that should be routed to a human escalation instead of answered from documents.
• LLM reranking (ENABLE_LLM_RERANKING) — re-ranks retrieved chunks with an extra LLM pass.

The answer is returned as an AgentResponse / WorkflowOutput (workflows/rag_chat_models.py): Markdown text with inline citations [1], [2], plus a list of reference_footnotes linking each citation to a document name, page, and chunk id.

The multi-tenant model

RA is designed so a single deployment serves multiple organizations. There is no ambient deploy-wide default tenant — every request must carry its org id explicitly:

• Tenant identity. The X-Organization-Id header (REST) or organization_id query param (WebSocket) carries a UUID. Org IDs are deterministic UUIDv5 values derived from an org slug (organizations.py::org_id_from_slug), so the same org resolves to the same UUID across every environment. Worked example: slug aara → 36b2b7ae-cd2f-51ab-b9a5-369ca259991d.
• Per-org configuration. Feature flags, model settings, and secrets-with-safe-defaults are resolved per org by ConfigService (config_service.py): an org's override in the organization_config DB table wins; otherwise the deploy-level Config.<KEY> default applies. Prompt overrides (e.g. the escalation email footer, the query-rewrite prompt) are namespaced under a nested prompts sub-key and resolved via get_prompt.
• Per-org auth. The Azure AD app registration used to validate a tenant's tokens (client id, tenant id, issuer, JWKS URL, and which JWT claim carries the user id) is itself per-org config, resolved from the DB with no code or env fallback. A missing/incomplete org config surfaces as a 401, not a 500 — from the caller's perspective a token that cannot be validated is invalid regardless of why (middleware.py::JwtAuthBackend).
• Per-org data scope. Every DB call made while handling a request is scoped to the resolved org via a context variable set by the middleware.

Authentication backends

RA selects an auth backend at startup from environment variables (middleware.py::_create_auth_backend):

Backend	Selected when	Validates
JwtAuthBackend	default	Azure AD (Entra ID) RS256 JWTs, against the requesting org's per-org config. Cross-checks the token's tid claim against the org's configured tenant.
LocalJwtAuthBackend	IP_ALLOWLIST_JWT_SECRET is set	Short-lived HMAC JWTs issued by POST /auth/ip-token to allowlisted client IPs. Always assigns the fixed App.User.Virtual role.
CompositeAuthBackend	IP_ALLOWLIST_JWT_SECRET is set	Routes each token to the backend registered for its iss claim (IP-allowlist issuer → local HMAC backend; everything else → Azure AD).
FakeAuthBackend	DISABLE_AUTH=true	Nothing. Returns a hardcoded dev user. Local development only — never deploy.

Roles (middleware.py): App.Admin.FullAccess, App.User.ReadOnly, App.User.Virtual, App.Platform.Operator. Virtual users have session-scoped history only — endpoints guarded by require_non_virtual_user (chat history, agent memory) return 403 for them.

Key components

Area	Module(s)	Responsibility
App entry point	server.py	Builds the FastAPI app, registers routers, wires the LogicStream WorkflowServer, discovers workflows, starts the agentic grep pool and Arize tracing in the lifespan.
Auth + tenancy	middleware.py, organizations.py, config_service.py	Auth backends, org-id parsing, per-org config/prompt resolution.
API surface	routers/*.py	One router per feature area (see the HTTP API reference).
RAG workflow	workflows/	LangGraph graph, nodes, retrieval modes, escalation, query rewriting, multi-query.
Retrieval tools	tools/	Azure AI Search retrieval (langgraph_retrieve_content.py), citation formatting, agentic search tools.
Persistence	storage/	Chat state, messages, agent memory, escalation routing, org config — SQLAlchemy async over MSSQL or PostgreSQL. Schema in storage/tables.py, migrations in alembic/.
Caching	cache/	Two-level TTL cache: per-worker L1 in-memory + shared L2 Redis, with cross-worker invalidation.
Observability	observability.py	Optional Arize / OpenTelemetry tracing, gated by ARIZE_TRACING_ENABLED.

Storage and migrations

Persistence is SQLAlchemy async and supports both MSSQL (aioodbc) and PostgreSQL (asyncpg), selected by DATABASE_URL. The authoritative schema is storage/tables.py; migrations live in backend/alembic/versions/. Migrations are applied out of band — the backend does not run them on startup, so a routine redeploy cannot silently alter the schema. See backend/README.md — the Database Migrations section — for the full workflow.

Observability

When ARIZE_TRACING_ENABLED=true, server.py initialises Arize tracing and instruments FastAPI at module load (it must run before the ASGI middleware stack freezes, so it is deliberately not inside the lifespan). Traces are enriched per-org: the middleware stamps organization_id onto the active span. When the Arize variables are unset, tracing is off and no data leaves the service.

TODO(SME): confirm whether tracing is intended to be fail-loud (raise at startup when unconfigured, per the AltaForge require_tracing convention) or fail-silent as implemented here (ARIZE_TRACING_ENABLED defaulting to off). The current code path only instruments when the flag is explicitly true.