Brief - DocGen¶
type: internal # framework/accelerator owned by AltaML; deployed on client engagements
status: draft
date: 2026-05-25
author: Mark Ly
written-by: AIA
Length budget: 1-4 pages. One fact, one place - don't restate; cross-reference with [B-*] IDs.
Change History¶
| Date | Author | Change |
|---|---|---|
| 2026-05-20 | Mark Ly | Initial draft |
| 2026-05-23 | Mark Ly | Updated after domain-model + release-plan crystallization: lib-import is the default consumption pattern (not SaaS); SaaS HTTP API is R6-deferrable; UI is React via altaforge-ui (not Python); LLM access via altaforge-llm wrapper. Tenant → organization terminology aligned with other AltaForge projects. Domain-model concepts (texts, prompts, risk_blocks, snippets, tools) defined in R1 PRD. 6-release plan in ./epic.md. |
| 2026-05-25 | Mark Ly | Dependency + tech-stack alignment after R0 (altaforge-llm) shipped and R1 TDD pivoted to LangGraph: Mandatory tech now names the full AltaForge dependency set (altaforge-core for runtime + AltaForgeGraph LangGraph base + multi-tenant session, altaforge-llm, altaforge-logging, altaforge-ui). Orchestration is LangGraph via AltaForgeGraph. Observability: Arize AX for shared-infra deployments, colocated Arize Phoenix for sovereignty deployments. CI added to out-of-scope (handled by the central platform team, not in this repo). |
| 2026-05-26 | Mark Ly | Rearchitect to gather → synthesize → render pipeline. Engine now collects each section's data_needs, gathers data once per document (deduped), synthesizes a shared context, then fans out per-section render. R1 ships prompts as the sole content recipe; texts / snippets / risk_blocks defer to R2. Tools elevated to first-class data sources with 4 R1 built-ins. R1 phase count drops 9 → 8. |
| 2026-05-27 | Mark Ly | Section-owned actions + document templates. Prompts move inline onto section-owned actions — data_needs (gather) + content_outputs (output); the shared prompts table is removed and sections lose content_recipe_* / current_content (content derived from outputs). Documents gain is_template + created_from_template_id + instantiate_template (deep-copy snapshot) so a "Proposal" is configured once and stamped per client. Lands as two phases (action-ownership refactor → templates) before R1.4. See ./reference/domain-model.md [D-13] + docs/design/specs/2026-05-26-document-templates-design.md. |
| 2026-05-29 | Mark Ly | Source traceability: output-level sources on content_outputs; full URL/text chain via gathered_data; span-level deferred to R2 texts. content_outputs gains a sources_json column (JSON list of Source records — gathered_data_id + selector pin); render records sources from the input side at render time. The bridge denormalizes each selector to URL/title/snippet so the UI shows one clickable citation. LLM-emitted citations are validated against context (typed error on hallucinated/out-of-context citation), never trusted. Lands as R1.6 (render). See ./reference/domain-model.md [D-14] + docs/design/specs/2026-05-29-source-traceability.md. |
| 2026-05-31 | Mark Ly | Doc-owned actions refactor (R1.7). Data needs move from sections to the document — one shared gather config visible to every section. The content_outputs table is removed; each section owns a single prompt + render_mode + output_template + output_type. Synthesize becomes a single structured mega-call that builds a dynamic Pydantic schema from the document's from_synth sections and fills every field in one shot. Render dispatches per-section by render_mode: from_synth pulls from the synth payload, template is a pure Jinja2 render, custom_render makes its own LLM call. Adds two org-scoped registries — texts (key/value substitutions) and snippets (reusable Jinja2 templates that reference texts) — that the substitution layer interpolates without an LLM call. Adds document.tools_available for mid-flight tool use during the mega-synth + custom_render paths. |
TL;DR [B-tldr]¶
DocGen is an AltaML toolkit for shipping client document-generation solutions in weeks instead of months. It bundles a Python framework — an engine running a gather → synthesize → render pipeline (sections declare their data needs, the engine collects + synthesizes once per document, then renders each section from the shared context), a catalog of data-gathering tools (web search, file ingestion, MCP connectors, plus domain-specific adapters that graduate from solution repos), and a React UX component library (docgen-ui, built on altaforge-ui) — that AltaML devs consume from each solution repo. R1 ships prompts as the sole content recipe; richer rendering primitives (texts for deterministic Jinja content, snippet composition) land in R2. The default consumption pattern is library import (solutions submodule altaforge-docgen and call its Python engine directly). For data-sovereignty engagements (Odyssey Phoenix is the first), the same libraries are deployed into the client's own cloud as a single-organization install. A SaaS HTTP API + productized DocGen Demo app land as a later, deferrable release for the sales channel. In all modes, AltaML's devs build and operate the solution.
Why this exists [B-why]¶
- We keep rebuilding the same scaffolding. JBRK and Phoenix independently re-solved section definition, multi-source data ingestion, synthesis, review, version history, and export.
- Each client still wants its own UX. Phoenix lets underwriters configure output; JBRK locks templates. Divergent surfaces, common engine.
- N bespoke deployments doesn't scale operationally. Centralizing the heavy infra into one shared core is the only way the portfolio is maintainable.
Who it's for [B-user]¶
Primary persona — the AltaML solution builder. Engineer or AIA configuring DocGen for a specific client engagement. They know the client's domain reasonably well, want to spend their time on the things that are actually unique (data extraction, prompts, business rules), and need the framework's plumbing to be invisible and reliable. They are comfortable in Python and YAML/config; they are not the end user of the generated documents.
Note: end users of the deployed solution (the lawyer at JBRK, the underwriter at Phoenix) interact with a client-specific UX that the solution builder constructs on top of DocGen. Their needs shape each client's UX, but they are downstream of this brief — they are the personas for each client's PRD, not for the framework brief.
Current state [B-current]¶
- Each build picks its own LLM orchestration pattern, its own scrapers, its own export library — no shared conventions across the portfolio.
- Review/edit UX is rebuilt or skipped per engagement, even though the underlying needs (display, edit, regenerate, version) repeat.
- Version history and export format support land in whatever ad-hoc shape the original engagement called for.
Desired outcome [B-outcome]¶
A solution builder hands DocGen a new client engagement and is producing reviewable draft documents within days, not weeks. They spend their effort on what's actually different about this client — domain rules, wording conventions, and the end-user UX that fits this client's workflow. They inherit everything else from the framework: the synthesis loop, section generation, regenerate/edit primitives, version history, exports, the UI component library, and a catalog of ready-to-use data-gathering tools they wire to each section.
Their solution lives in its own repo that the builder owns. In the default consumption pattern, the solution submodules altaml/altaforge-docgen and imports the docgen Python package directly — heavy lifting (DB, LLM gateway via altaforge-llm, versioning, exports) happens in-process. For AltaML-managed deployments, the solution's organization data lives on the shared AltaForge Postgres server (multi-tenant via organization_id discriminator). For client-cloud (sovereignty) deployments (Odyssey Phoenix, and any future sovereignty-bound engagement), AltaML deploys the same libraries — engine, content units, tools, UX — into the client's cloud account as a single-organization install that AltaML still operates. A hosted SaaS HTTP API is available as a deferrable later option for prospects who want to consume DocGen without managing the library themselves. In all modes, builders code against the same public surface; the only difference is where the deployment runs and whether infra is shared.
The data-gathering catalog is curated and growing: generic tools (web search, page scrape, file ingestion, MCP client) ship out of the box, and domain-specific tools (e.g., the JBRK PDF-to-graph extractor, the Phoenix screen scraper) graduate into the catalog as they prove reusable across engagements.
The framework exposes its capabilities as building blocks — APIs and UI components — so each client's UX can range from "fixed templates, no user choices" (JBRK) to "configurable output, fixed sections" (Phoenix) to whatever the next engagement needs, without forking the engine. Among those building blocks are end-user controls — prompt editing, regenerate-with-variant, section locking — that DocGen ships as functionality; each client UX decides whether to expose them to its end users.
The DocGen Demo app is itself a real AltaML product — hardened, supported, and built entirely from the public component library. It proves the framework can produce production-grade output, gives solution builders a copy-paste starting point, and is the artifact AltaML demos in sales conversations.
Killer demo for v1 [B-killer]¶
A solution builder creates a new document type on the DocGen Demo app — configures sections, data sources, and prompts — and within an hour an end user is reviewing, editing, regenerating sections, and exporting to .docx, all in a polished UX nobody had to build from scratch.
Success metrics [B-success-N]¶
| ID | Metric | Target | Measured how |
|---|---|---|---|
| B-success-1 | Time from "new engagement kicked off" to first reviewable draft document | < 1 week (baseline today: 6-8 weeks) | Tracked per engagement; compared against JBRK / Phoenix baselines |
| B-success-2 | DocGen Demo app runs end-to-end on a freshly configured document type with no framework code changes | 100% of demo paths working | Smoke-tested per release against the canonical demo document type |
Acceptance:
B-success-1: Given a new client engagement of comparable complexity to JBRK or Phoenix, when a solution builder applies DocGen with default scaffolding, then a reviewable draft is produced within 5 working days of kickoff.
B-success-2: Given the DocGen Demo app and a newly configured document type, when a builder runs the app against it, then every demo path (configure → generate → review → edit → regenerate → version → export) works without requiring framework code changes.
Out of scope (v1) [B-not]¶
- Mandating the Demo app as every client's UX. DocGen ships the Demo app as a real product, but each engagement is free to compose its own UX from the same component library. Why: clients differ — Phoenix wants underwriter-configurable output, JBRK wants locked templates — and forcing a shared UX would re-create the bespoke-build problem at a different layer.
- Per-client deployment as the default. Library import (consuming
altaforge-docgenvia submodule in each solution repo) is the default consumption path; shared-infra multi-tenant is the deployment default for AltaML-managed clients; client-cloud single-organization deployment is the exception, reserved for clients with binding data-sovereignty requirements. Why: every client-cloud deployment is a parallel production environment AltaML has to operate — worth it for Odyssey, not the path of least resistance for everyone else. - Domain-specific business logic baked into the framework engine. Tax law, reinsurance underwriting rules, etc. stay in client-specific configuration and prompts. Why: generality is the whole point. Note: domain-specific data-gathering tools are a different matter — those are explicitly in scope as catalog entries that other engagements can opt into.
- CI / CD configuration in this repo. Pipeline standards, runners, secret management, and build-orchestration are owned by the central platform team and applied uniformly across all AltaML repos. Why: avoid per-repo drift; framework teams should focus on framework code, not pipeline plumbing.
Constraints [B-constraint]¶
| Type | Detail |
|---|---|
| Timeline | v1 must be usable for the Odyssey Phoenix UAT (R3 in the 6-release plan). Subsequent rollouts in R5 (multi-solution-rollout): VerticalScope Sales, DealDesk, JBRK to UAT + prod; Odyssey to prod; RWDI / Site / NAIT paced. Framework scope and sequencing are gated by what these engagements need — see ./epic.md for the full release decomposition. |
| Mandatory tech | Python framework (docgen — engine running the gather → synthesize → render pipeline; data tools; section-owned actions — data_needs (gather) + content_outputs (output, prompt text inline); texts / snippets / risk_blocks in R2). Built on the AltaForge dependency set: altaforge-core (runtime base — AltaForgeGraph is the project's LangGraph wrapper that ships standard Arize tracing; multi-tenant OrganizationScopedSession; shared SQLAlchemy 2.x + Alembic conventions), altaforge-llm (LiteLLM wrapper; 4 auth modes — oauth_anthropic / oauth_openai / oauth_gemini for dev via vendor-CLI tokens, litellm for CI/prod via AltaML's gateway; LLM-agnostic at the model layer, Claude default), altaforge-logging (structured logging with Arize trace correlation). Orchestration: LangGraph (via AltaForgeGraph) — engine functions stay pure; nodes are thin wrappers. React UX components (docgen-ui co-located in this repo, building on altaforge-ui for org-wide design-system primitives). |
| Deployment modes | (a) Library import (default) — solutions submodule altaforge-docgen and call docgen Python directly. AltaML-managed clients land on shared AltaForge Postgres (one docgen DB, multi-tenant via organization_id); observability flows to Arize AX (cloud). (b) Client-cloud single-organization (sovereignty exception) — same libraries deployed into client cloud account with their own Postgres + colocated Arize Phoenix (Odyssey is the first). (c) SaaS HTTP API — deferred to R6 (deferrable); not a gate for any client. |
| Public surface | The DocGen library/API is the only contract client-facing code may build against. Same semantics whether libraries are imported locally (lib-import or client-cloud mode) or eventually called via SaaS HTTP (R6). Deployments must not reach into internals. |
| Domain model | Canonical vocabulary lives in ./reference/domain-model.md (split out of the R1 PRD). R1 entities: organizations (tenants), documents (a reusable template or a stamped instance), sections (each owning two ordered action lists), tools (first-class data sources with 4 R1 built-ins + Mock + env-var credentials), data_needs (gather actions), content_outputs (output actions — prompt text inline; no shared prompts table; carries sources_json for audit traceability — list of Source records pinning each output back to the gathered_data rows it consumed, surfaced as URL/path/snippet in the UI), gathered_data (per-document audit), synthesized_context (per-document shared understanding). A template is deep-copied to instantiate. Texts / snippets / risk_blocks defer to R2 (span-level pinning rides on texts). Every downstream PRD/TDD + consuming solution repo references this doc. |
Open questions [B-q-N]¶
None at the brief level — remaining unknowns surface in the Epic / PRD decomposition.
Sign-off¶
| Role | Name | Date |
|---|---|---|
| Author | Mark Ly | 2026-05-20 |
| Reviewer | ||
| Approver | Mark Ly (CTO, sponsor) |
Reviewer's checklist¶
- [x] TL;DR fits in ≤80 words and reads as a complete thought
- [x] At least one success metric is measurable (number + unit + target)
- [x] Out-of-scope list contains at least one item
- [x] Constraints honestly captured - no hidden deadlines or budget assumptions
- [x] Open questions are truly unresolved (no "Resolved:" annotations)
- [x] Cross-ref IDs (
B-*) assigned where the PRD will need to trace back