PRD - DocGen Framework v0 Core (R1)¶
status: draft
date: 2026-05-26
author: Mark Ly
brief: ../../explanation/brief.md
epic: ../roadmap.md
parent-release: E-release-1
written-by: AIA + TechLead
stakeholders: Mark Ly (CTO/sponsor)
Written by: AIA + TechLead together. This PRD doubles as the foundational concept doc for all of DocGen — it defines the vocabulary every downstream release uses.
How to read this¶
| Audience | Read | Skip |
|---|---|---|
| Anyone new to DocGen | Domain Model section first (vocabulary + walkthroughs) | Implementation detail rows |
| Product reviewer | TL;DR + Domain Model + Success Metrics + Functional Requirements + Reviewer Checklist | Data Model details |
| Engineering reviewer | TL;DR + Domain Model + Functional Requirements + System Requirements + Data Model + Open Questions | User personas (skim) |
| TDD writer (human or agent) | Everything | — |
| Sponsor signing off | TL;DR + Success Metrics + Open Questions + Sign-off | Implementation detail |
Length budget: longer than the template's 3-7 pages (R1's nature is foundational; the Domain Model section is the deliverable, not boilerplate).
Change History¶
| Date | Author | Change |
|---|---|---|
| 2026-05-23 | Mark Ly | Initial draft. R1 absorbs the foundational domain model (was a separate domain-model.md) and ships the framework v0 core. |
| 2026-05-26 | Mark Ly | Rearchitect to gather → synthesize → render pipeline. Each section now declares data_needs; the engine gathers data once per document, synthesizes a shared context, then fans out per-section render. R1 ships prompts only as the render recipe; texts / snippets / risk_blocks defer to R2. Tools elevated to first-class data sources with 4 built-ins (web_search, page_scrape, file_ingest, MCP client) + credential handling via env vars (.env for dev, Azure Key Vault for staging+). R1 phase count goes 9 → 8. |
| 2026-05-29 | Mark Ly | Source traceability folded into R1.6 (render). Every content_output records the sources it consumed: content_outputs gains a sources_json column (list of Source records pinning each output to gathered_data IDs + per-fragment selectors). Recording is from the input side at render time (never trust LLM-emitted citations); optional per-output, if the schema declares a citation field, the LLM emits AND the engine validates against context. Bridge denormalizes selectors to URL/title/snippet for one clickable citation per output. New [P-feat-13] + [P-eval-6]; [P-entity-5] updated; span-level pinning explicitly defers to R2 texts. See docs/design/specs/2026-05-29-source-traceability.md. |
TL;DR [P-tldr]¶
- R1 is the foundational release. Ships the core DocGen framework — domain model, schema, engine with
gather → synthesize → renderpipeline, multi-tenant data layer, LLM access viaaltaforge.llm, observability viaaltaforge.core'sAltaForgeGraph(LangGraph base) — plus an extensive pytest suite that generates real markdown documents intest_outputs/. - The Domain Model section below is the canonical vocabulary every downstream PRD, TDD, and conversation cites. Organizations, documents, sections, tools, prompts, data needs, gathered data, synthesized context — defined in
../../reference/domain-model.md. - R1 ships prompts as the sole render recipe. Texts, snippets, and risk blocks land in R2. The R1 schema is designed so adding them is purely additive.
- R1 ships no deployment infra and no client UAT. Deployment infrastructure and production features land in R2; first client UAT (Odyssey) lands in R3. R1's proof is its own test suite producing eyeball-able document outputs that any reviewer can open.
- Solutions can start consuming R1 libraries immediately — Odyssey, JBRK, VS Sales, DealDesk, RWDI, Site, NAIT — but no client ships to UAT or production until R3+ (Odyssey UAT in R3, others in R5).
- Sponsor: Mark Ly (CTO).
Domain Model¶
The canonical vocabulary for DocGen lives in a separate doc that all releases reference:
See
../../reference/domain-model.mdfor the full Domain Model + walkthroughs.
R1 ships the canonical domain model as a foundational deliverable. Every downstream PRD, TDD, and stories doc cites ../../reference/domain-model.md for terminology. If a term in this PRD is unclear, look there first.
Product Vision [P-vision]¶
← see parent [E-release-1] and Brief [B-tldr] / [B-outcome].
In short: ship the framework foundation that every downstream client solution will rest on. R1 doesn't aim at a paying customer; it aims at the internal capability to assemble client document-generation solutions from a common substrate. The proof that R1 succeeds is the test suite — running end-to-end, producing real markdown documents that demonstrably exercise the gather → synthesize → render pipeline, multi-tenant isolation, and LLM access.
Success Metrics [P-metric-N]¶
| ID | Metric | Baseline | Target | Type | Notes |
|---|---|---|---|---|---|
| P-metric-1 | R1 test suite produces real document outputs that an engineer can open and visually verify | n/a (new) | 100% of R1 walkthroughs (cooking, Odyssey-shape, render-time-drift) have a passing test in tests/integration/ that writes a .md file to test_outputs/ |
byproduct | The framework is real if the tests produce documents. No documents = no framework. |
| P-metric-2 | Multi-tenant isolation test passes | n/a (new) | 100% — two organizations in the same DB cannot see each other's documents via the OrganizationScopedSession |
lever | Per [P-risk-1]; the single most damaging bug class |
| P-metric-3 | A new solution builder can clone the repo, follow the runbook, and generate their first document in under 30 minutes | n/a (new) | < 30 minutes from clone to first output | byproduct | The framework is usable if onboarding is fast. Measured via a "fresh-clone smoke test" runbook validated by one engineer outside the R1 team. |
Acceptance:
P-metric-1: Given a fresh checkout, when
pytest tests/integration/runs, then every walkthrough test passes and writes a.mdfile totest_outputs/that an engineer can open in any text editor and verify the structure (sections present, content per the test's fixture, etc.).P-metric-2: Given two organizations
organization_aandorganization_bin the same SQLite DB, each with one document titled "Memo", when a session scoped toorganization_aqueries documents, then it sees exactly one (its own). When scoped toorganization_b, it sees exactly one (the other one). Cross-organization query attempts raise no documents.P-metric-3: Given a new engineer + the R1 onboarding runbook (in
releases/r1-framework-v0-core/README.md), when they clone the repo and follow the runbook, then within 30 minutes they have a passing local test that generated a document intest_outputs/.
Evals [P-eval-N]¶
R1 ships framework-correctness evals only; full eval observability lands in R2 (Arize AX / Phoenix wiring).
| ID | What it checks | Methodology | Pass threshold |
|---|---|---|---|
| P-eval-1 | Gather phase invokes every data need exactly once (dedup correctness) | Fixture document with two sections referencing the same (tool, args); assert gathered_data has one row, not two; both sections see the same value |
100% |
| P-eval-2 | Synthesis output conforms to the configured shape (raw_text vs structured Pydantic) |
Two fixture documents, one per mode; assert synthesized_context.payload_json parses as expected shape |
100% |
| P-eval-3 | Render fidelity — prompt's LLM call returns structurally-conforming output | Fixture prompt + interpolated synthesized context; structural conformance scorer (output is non-empty, has expected sections, no error markers) | ≥ 95% on a 20-prompt synthetic dataset |
| P-eval-4 | Multi-tenant isolation — OrganizationScopedSession filters at the data layer |
Two organizations, overlapping document titles; query scoped to one; assert other is invisible | 100% (zero leakage) |
| P-eval-5 | Render-time tool call flagging — when a prompt invokes a tool mid-render, the engine emits an Arize span tagged render_time_tool_call=true |
Fixture prompt with tools_available=[AddTool] invoked during render; assert OTel span attribute is set |
100% |
| P-eval-6 | Source-correctness — when the LLM emits a citation in a citation-required output_schema, render rejects citations not present in context |
Fixture content_output with output_schema.citation declared; force the LLM via prompt to emit (a) a valid gathered_data_id + resolvable selector — assert recorded; (b) a gathered_data_id not in context — assert CitationValidationError and status='failed'; (c) a selector that doesn't resolve against gathered_data.result_json — assert CitationValidationError. Per [P-feat-13] + docs/design/specs/2026-05-29-source-traceability.md [3]. |
100% (zero hallucinated citations persisted) |
R2 wires these (plus solution-specific evals) into Arize for production observability.
Scope & Non-Goals [P-scope]¶
Prerequisite (R0, shipped):
- altaforge.llm library (altaforge-libraries/altaforge/) — Python package wrapping LiteLLM with four auth modes: oauth_anthropic / oauth_openai / oauth_gemini (dev via vendor-CLI tokens) and litellm (CI via AltaML's gateway). Path-installed from vendor/altaforge-libraries/.
- altaforge.core runtime — AltaForgeGraph LangGraph wrapper that auto-wires OTel + OpenInference for Arize tracing; multi-tenant OrganizationScopedSession conventions.
- altaforge.logging — structured logging with Arize trace correlation.
In scope (R1) — built in 8 interleaved phases (Python + UI + demo together):
Python framework (docgen/):
- Domain model documentation — see ../../reference/domain-model.md for the canonical vocabulary
- Schema: organizations, documents, sections, prompts, tools, data_needs, gathered_data, synthesized_context
- Multi-tenant data layer — OrganizationScopedSession factory using SQLAlchemy with_loader_criteria
- Tools — DataTool ABC + 9 built-ins (AddTool, AppendTool, WebSearchTool, ClaudeWebSearchTool, OpenAIWebSearchTool, PageScrapeTool, WebPageTool, FileIngestTool, MCPTool) + test-only StubTool (in tests/): ClaudeWebSearchTool uses altaforge.llm.anthropic_web_search (Anthropic server-side web_search_20250305, direct SDK — sidesteps the LiteLLM gateway's web-search streaming/citation bugs). OpenAIWebSearchTool uses altaforge.llm.web_search(mode=OPENAI_FAST|AGENTIC|DEEP_RESEARCH) over the Azure-OpenAI Responses API. Both web-search tools accept fetch_pages=True to attach full page bodies (title, text, status_code, truncated) to each citation in one call — backed by altaforge.llm.web_page. WebPageTool wraps the same web_page helper as a standalone single-URL fetcher (category='page_scrape') for agentic follow-up reads. Solution-specific tools subclass DataTool in solution repos.
- Tool credentials — env-var driven; .env.example shipped in the repo; Azure Key Vault for staging+ (per [D-9])
- Gather phase — gather_data_node unions section data_needs, dedupes, invokes (tool or prompt), persists to gathered_data
- Synthesize phase — synthesize_node produces SynthesizedContext; supports raw_text, structured (Pydantic), and none (skip) modes; templated mode defers to R2
- Render phase — fan-out per output (content_outputs); prompt or tool kind in R1; render-time tool calls allowed but Arize-flagged
- engine.generate_document(organization_slug, document_spec) — top-level orchestration via AltaForgeGraph: collect_data_needs → gather → synthesize → fan-out render → persist
- Local SQLite + Alembic migrations (multi-tenant via organization_id)
- Markdown export + pytest suite producing real markdown documents in test_outputs/
React UX library (docgen-ui/):
- Vite-built npm package consuming altaforge-ui (Radix Themes-based) via pnpm workspace
- Components: <SectionCard> (display + status badge + edit/regenerate buttons + data-needs panel), <DocumentView> (composes SectionCards + synthesis context preview), <ExportButton> (triggers markdown download)
- Per-feature evolution alongside Python: each new framework feature surfaces in a matching UI affordance
Demo Next.js app (demo/):
- Next.js 14 App Router consuming docgen-ui + altaforge-ui via workspace; calls Python docgen via subprocess shim (R1) — HTTP API path is R6
- User-testable from phase 1.2 onward — each phase grows the demo's surface so humans can click around and validate
- Walkthrough showcases in phase 1.8 — canonical document types (cooking, Odyssey-shape, render-time-drift) live in demo routes
Non-goals (R1) — deferred to later releases (per 6-release plan):
- Texts (deterministic Jinja content units) — deferred to R2. Brief's promise of richer rendering primitives lands then.
- Snippets (composition of texts + risk_blocks — deterministic content only; prompts never live inside a snippet) — deferred to R2 alongside texts
- Risk blocks + workflow gate — deferred to R2
- Templated synthesis mode (synthesis output produced via texts / snippets instead of an LLM call) — deferred to R2
- Per-section version history (sections rolling back to prior content) — deferred to R2
.docxexport — deferred to R2- Arize eval adapter (AX + Phoenix dashboards) — deferred to R2. R1 wires OTel + OpenInference spans via
AltaForgeGraph, so trace data is exported; the eval dashboards land in R2. - Per-organization encrypted credential storage — deferred to R3 if multi-tenant shared infra needs it; R1 + R2 use env vars only
- Client-cloud deployment template + Phoenix self-host recipe — deferred to R2
- Odyssey UAT as a deliverable — deferred to R3 (odyssey-uat). Odyssey solution work starts against R1 libraries — its UX is built in R1 against fixture data per the cross-team plan; real adapters + UAT in R3.
- Version-pinning columns on content units — deferred to R4 (framework-scale)
.pdf/ Google Doc export expansion — deferred to R4 (.mdis R1's default test output format)- Shared-infra deployment template — deferred to R4
- Other solution UATs (VS / DealDesk / JBRK) + Odyssey prod + paced rollouts (RWDI / Site / NAIT) — deferred to R5 (multi-solution-rollout)
- Demo app productized for sales + SaaS HTTP API — deferred to R6 (saas-api-plus-demo, deferrable)
- Domain-specific business logic in the engine — per Brief
[B-not]; lives in solution prompts and tools - CI workflows — owned by the AltaForge CI team org-wide; stories never include
.github/workflows/*files
User Personas [P-persona-N]¶
P-persona-1 (primary): AltaML Solution Builder (engineer or AIA) Type: primary. The R1 deliverable's user. Consumes the framework from each solution repo. Goal (in their voice): "Give me a Python library I can import, a clear vocabulary I can reason about, and a test suite that proves the framework works. I'll build solution-specific prompts and tools on top." Pain today: every engagement reinvents engine + data layer + LLM gateway + data-gathering plumbing. Three weeks of plumbing before the first prompt. Patterns and constraints: will read source if docs are thin; expects to debug their own integration, not the framework.
P-persona-2 (secondary): DocGen Framework Maintainer Type: secondary. The engineers who own the framework code (R1+). Goal: keep the framework's API stable so downstream solutions don't break on minor bumps. Pain today: none yet — R1 is the first release.
User Flows [P-flow-N]¶
P-flow-1: Solution builder integrates DocGen into a new solution repo Actor: Solution Builder (P-persona-1). Starting state: Empty solution repo (e.g., a new
altaml/<solution>-apprepo). Steps: 1. Addaltaml/altaforge-docgenas a submodule atvendor/altaforge-docgen. 2. Configureuvworkspace to installdocgenfrom the submodule path. 3. Set the required env vars listed indocgen/.env.example(tool credentials, LLM access). 4. Importdocgenandaltaforge.llmin solution code. 5. Register solution-specificDataToolsubclasses + define prompts. 6. Seed them into the organization's tables on first run. 7. Generate a test document. End state: Solution-specific code generating documents via the framework, all data stored under the solution'sorganization_id. Edge case: Solution forgets to scope queries by organization_id →OrganizationScopedSessionprevents the unsafe pattern at the data layer. Acceptance: Given a fresh solution repo with the DocGen submodule, when the builder follows the onboarding runbook, then their first document generates and the test suite confirms multi-tenant isolation.P-flow-2: R1 test suite generates documents and writes them to
test_outputs/Actor: R1 test runner (pytest, exercised by humans + CI). Starting state: Fresh checkout, env configured (LLM access reachable; tool credentials set per.env.example). Steps: 1.pytest tests/integration/runs. 2. Each integration test sets up an organization + fixtures + sections withdata_needs+ content prompts, callsengine.generate_document(...), writes the rendered document totest_outputs/<test_name>.md. 3. Test asserts content matches expectations (golden file or structural conformance). End state:test_outputs/contains one.mdper integration test. Humans open them to eyeball. Edge case: LLM call fails (rate limit, transient) → test retries N times, then fails loud (no fabrication on retry exhaustion). Acceptance: Given healthy LLM access, when the integration test suite runs, then all R1 walkthrough tests pass and write.mdfiles an engineer can open in any text editor.P-flow-3: A document is generated via gather → synthesize → render Actor:
engine.generate_document(...)invoked from solution code or pytest. Starting state: An organization + a document spec (title + sections, each owningdata_needs(gather actions) +content_outputs(output actions, prompt text inline)) + an optionalvariablesdict for top-level inputs. Steps: 1. collect_data_needs node — walks the document's sections, unions theirdata_needs, dedupes on(kind, tool_ref-or-prompt_body, args_hash). 2. gather_node — invokes each unique gather action.kind='tool'calls theDataToolnamed bytool_ref;kind='prompt'runs an LLM call viaaltaforge.llmwith the inlineprompt_body(may itself invoke tools). Every result persists togathered_data. 3. synthesize_node — depending on the document's configured synthesis mode (none/raw_text/structured), either skips, produces a single text blob, or extracts typed Pydantic facts. Result persists tosynthesized_context. 4. fan-out per output — for eachcontent_output, the engine renders it against the synthesized context (kind='prompt'→ LLM, may calltools_available;kind='tool'→ deterministic tool result). Render-time tool calls emit an Arize span taggedrender_time_tool_call=trueso reviewers can promote them todata_needsif they prove expensive. 5. persist — each output'sresultpersisted; the section's content is the ordered concatenation of its outputs; documentstatetransitions if applicable. End state: Document with all sections rendered;gathered_data+synthesized_contextrows persist as the audit trail. Edge case 1: Adata_needinvocation fails — the section that needed it surfaces a failure state; other sections still generate. Edge case 2: Synthesis mode isnone— sections render directly againstgathered_data(key-addressed). Acceptance: Given the Odyssey-shape walkthrough fixture (5 prompts, structured synthesis), whengenerate_documentruns, then gather invokes 2 unique data needs (with dedup), synthesize produces oneOdysseyDealFactsrow, render produces 5 sections, and the document writes totest_outputs/odyssey_walkthrough.md.
Scoring & Business Logic [P-rule-N]¶
n/a — the framework executes templates and prompts defined by solutions; it has no domain business logic of its own. Solution PRDs own their [P-rule-N] entries (e.g., Odyssey's approval-detection logic, JBRK's tax-math computations).
Functional Requirements [P-feat-N]¶
R1's functional surface is the Python framework + tests. Each feature row maps to TDD phases + acceptance tests.
| ID | Feature | TDD phase | Description (one line) | Acceptance lives at |
|---|---|---|---|---|
| P-feat-1 | Schema: organizations, documents, sections + multi-tenant data layer |
R1.1 | SQLAlchemy models + Alembic baseline + OrganizationScopedSession factory enforcing organization_id filter |
P-flow-1, P-metric-2 |
| P-feat-2 | Document CRUD + <DocumentView> + demo doc list |
R1.2 | Documents create / list / get / update_state / delete; <DocumentView> component renders title + state badge; demo homepage lists docs |
P-flow-1 |
| P-feat-3 | Section CRUD + ordering + <SectionCard> + demo section list |
R1.3 | Sections create / list / update / delete with ordinal management; data_needs slot on the section spec; <SectionCard> with status badge |
P-flow-1 |
| P-feat-4 | Tools — DataTool ABC + 6 built-ins (incl. AddTool, AppendTool) + test-only StubTool + gather phase + gathered_data persistence |
R1.4 | DataTool ABC; WebSearchTool / PageScrapeTool / FileIngestTool / MCPTool + StubTool; gather_data_node unions section data_needs, dedupes, invokes, persists |
P-flow-3, P-eval-1 |
| P-feat-5 | Synthesize phase — synthesize_node + SynthesizedContext table + 2 modes (raw_text, structured) + none (skip) |
R1.5 | Single-call synthesis over gathered_data; per-document configurable mode; structured mode uses solution-supplied Pydantic schema |
P-flow-3, P-eval-2 |
| P-feat-6 | Render phase — per-output render (content_outputs: prompt or tool) + render-time tool-call Arize flagging |
R1.6 | render_node renders each content_output over synthesized_context (prompt → LLM, may call tools_available; tool → deterministic result); render-time tool calls emit OTel span tagged render_time_tool_call=true |
P-flow-3, P-eval-3, P-eval-5 |
| P-feat-7 | engine.generate_document(organization_slug, document_spec) + markdown export |
R1.7 | Top-level orchestration via AltaForgeGraph (collect_data_needs → gather → synthesize → fan-out render) + markdown renderer |
P-flow-2 |
| P-feat-8 | Walkthrough tests + onboarding runbook | R1.8 | Three integration tests (cooking, Odyssey-shape, render-time-drift), each producing a real .md document in test_outputs/; README.md runbook; .env.example checklist |
P-eval-1, P-eval-2, P-eval-3, P-eval-5, P-metric-1, P-metric-3 |
| P-feat-9 | docgen-ui React component library — <SectionCard>, <DocumentView>, <ExportButton> |
R1.2-R1.7 | Vite-built npm package built incrementally as Python features land; consumed by demo via pnpm workspace | P-flow-2 |
| P-feat-10 | demo Next.js app — user-testable surface for every R1 phase |
R1.1-R1.8 | Next.js 14 App Router; calls Python docgen via subprocess shim; grows per phase; walkthrough showcase pages in R1.8 |
P-flow-2, P-metric-3 |
| P-feat-11 | Action-ownership refactor — section-owned data_needs + content_outputs (prompt text inline); drop the shared prompts table + sections.content_recipe_*/current_content |
R1.3.5 (before R1.4) | Reshape per [D-4]/[D-5] + [T-decision-3]/[T-decision-4]; sections own both action lists; content derived from content_outputs[].result; foundation for templates |
P-flow-3 |
| P-feat-12 | Document templates — is_template + created_from_template_id + instantiate_template (deep-copy snapshot) + list_templates |
R1.3.6 (before R1.4) | Configure once, stamp instances ("Proposal — Client A/B"); snapshot deep-copy per [D-13]/[T-decision-16] |
P-flow-1 |
| P-feat-13 | Source traceability — output-level source pinning on every content_output |
R1.6 | Every content_output records the sources it consumed (gathered_data_id + selector); audit chain reaches URL/path/snippet via gathered_data.result_json. Input-side recording at render time; optional validated LLM citations narrow the selector. Bridge denormalizes to one clickable URL/title/snippet per output. Per [D-14] + [T-decision-17] + docs/design/specs/2026-05-29-source-traceability.md. R1.6 scope. |
P-flow-3, P-eval-6 |
P-feat-1 acceptance: Given two organizations on the same DB with overlapping document titles, when either organization queries its documents via
OrganizationScopedSession, then it sees only its own.P-feat-4 acceptance: Given a document with two sections each declaring the same
(WebSearchTool, args={query:'X'})data need, when the gather phase runs, thenWebSearchToolis invoked exactly once,gathered_datahas one row, and both sections receive the sameoutput_keyvalue.P-feat-5 acceptance: Given a document configured with
synthesis_mode='structured'and anOdysseyDealFactsPydantic schema, when synthesize_node runs, thensynthesized_context.payload_jsonparses intoOdysseyDealFactsand every required field is populated.P-feat-6 acceptance: Given a
content_output(kind='prompt') withtools_available=[AddTool]invoked during the render phase, whenrender_noderuns, then the resulting OTel span carries the attributerender_time_tool_call=trueand is exported to Arize.P-feat-8 acceptance: Given a fresh checkout and healthy LLM access, when
pytest tests/integration/runs, then each of the three walkthrough tests passes and writes a markdown document totest_outputs/.P-feat-13 acceptance: Given a
content_output(kind='prompt') whose prompt interpolates onegathered_datarow produced byWebSearchTool(5 hits), whenrender_noderuns, then the output'ssources_jsonhas oneSourcerecord (gathered_data_idset,used_as='prompt_context',selector=None), and the bridge resolves it to an enriched payload withurl/title/snippetfrom the underlying gathered_data row. Additionally, given the same setup with anoutput_schemadeclaring acitation: {gathered_data_id, selector}field, when the LLM returnsselector={"result_index": 2}, then the engine validates the citation against context and persistsselector={"result_index": 2}on the Source; a citation referencing agathered_data_idnot in context raisesCitationValidationErrorand the output'sstatus='failed'.
System Requirements [P-sys]¶
| Concern | Requirement |
|---|---|
| Performance | Tool call: < 5s p95 for built-in network tools (web_search, page_scrape). Synthesis (LLM): ≤ 60s p95. Render per section (LLM): ≤ 30s p95. Full-document render with 5 sections + synthesis: ≤ 3 min p95. |
| Security | LLM credentials never logged. Tool credentials in env vars only, never persisted by the framework. Organization secrets sourced from env / secret store. |
| Tenancy / isolation | Logical multi-tenant isolation: shared DB, organization_id discriminator on every row, enforced at the session layer via with_loader_criteria. |
| Reliability | A data-tool failure never crashes the engine; the section that needed it surfaces the failure state and the rest of the document still generates. |
| Observability | Every node in the graph (collect_data_needs, gather, synthesize, render_section, persist_section) emits an OTel span via AltaForgeGraph. Render-time tool calls add render_time_tool_call=true attribute. Spans export to Arize AX (shared infra) or Arize Phoenix (sovereignty). |
| Determinism | Tool calls are deterministic (same args → same result, modulo external state). Synthesis and prompt rendering are LLM-mediated and not deterministic; tests use structural-conformance scoring. |
| Idempotency | Re-running generate_document with unchanged inputs hits cached gathered_data rows (keyed by args_hash). force_refetch=true bypasses cache. |
Technical Requirements [P-tech-req]¶
| Type | Constraint | Because |
|---|---|---|
| Language | Python 3.11 | AltaML default; matches downstream releases |
| Tooling | uv + uv_build + ruff + mypy + pytest + pre-commit |
AltaML defaults; lock for consistency |
| ORM | SQLAlchemy 2.x + Alembic | AltaML default; supports SQLite + Postgres dialects |
| Orchestration | LangGraph via altaforge.core.runtime.AltaForgeGraph |
Single graph wiring; OTel + OpenInference instrumentation for free; standard tracing to Arize |
| LLM access | altaforge.llm.client(auth_mode=...).completion() from altaforge-libraries/altaforge (path dep) |
Four auth modes (3 OAuth + 1 LiteLLM gateway); centralized LLM access org-wide |
| Tool credentials | Env vars; .env.example in repo; Azure Key Vault for staging+ |
12-factor app pattern; no secrets in DB; portable to client-cloud sovereignty |
| Templating | Jinja2 (prompt body variables + tool args interpolation) | Standard Python templating |
| Multi-tenant pattern | organization_id discriminator + SQLAlchemy session-level filter (with_loader_criteria) |
Enforce at the data layer so call sites cannot accidentally write unscoped queries |
| Test runner | pytest + pytest-asyncio | AltaML default |
| LLM-backed tests | Real LLM calls via the dev's vendor-CLI token in dev, via the LiteLLM gateway in CI | Per memory rule: tests for agent behavior use real LLM, not mocks |
| Test outputs | Markdown files in test_outputs/ (gitignored except .gitkeep) |
Eyeball-able artifacts; reviewers open them in any text editor |
Assumptions & Constraints [P-assume]¶
altaforge-libraries/altaforgeis available — shipped (R0); path-installed viavendor/altaforge-libraries/.altaforge.coreruntime module is available — shipped; providesAltaForgeGraphLangGraph base + multi-tenant session.- Tool credentials provided via env vars — solution code is responsible for setting them before
generate_documentruns; framework reads viaos.environ. - Claude 4.5 Sonnet is the default LLM — selectable per-organization via
organizations.llm_model. - Developer SSO works for OAuth — devs sign in via Claude Code / Codex CLI / Gemini CLI to populate vendor-CLI credential files.
Risks [P-risk-N]¶
| ID | Risk | Frame | Mitigation | Owner |
|---|---|---|---|---|
| P-risk-1 | Multi-tenant data isolation leak — a query forgets to filter by organization_id |
feasibility | Organization-scoping enforced at the data-access layer (OrganizationScopedSession + with_loader_criteria), not at every call site; multi-tenant integration test in CI (P-eval-4); production deployments add a DB-level safety net (RLS or schema-per-org) only if logical isolation proves insufficient under load. |
TechLead |
| P-risk-2 | Domain model proves wrong for one of the downstream solutions, requiring R1 revision | feasibility | R1 PRD walkthroughs cover the cooking sanity check, Odyssey-shape (5 prompts, structured synthesis), and render-time-drift. R2 adds JBRK-shape (texts + snippets). Each solution team reviews this PRD before R1 ships. If a shape is missing, surface it before R1 close, not in R2. | Mark Ly + solution leads |
| P-risk-3 | Render-time tool calls go unflagged (drift to expensive render runs) | observability | [P-feat-6] mandates the render_time_tool_call=true Arize span attribute. [P-eval-5] asserts it fires. Reviewers can scan Arize traces and promote calls to gather-phase data_needs. |
TechLead |
| P-risk-4 | Test suite takes too long because of real LLM calls (each test is 5-30s of latency) | usability | Tag tests with @pytest.mark.llm; CI runs them; local dev runs them on-demand. Pure tool-call tests (no LLM) stay fast and run every save. |
TechLead |
| P-risk-5 | Tool credential leakage (env vars accidentally logged or committed) | security | Repo ships only .env.example (never .env); .gitignore excludes .env; framework never logs full credential values; pre-commit hook scans for common secret patterns (TBD). |
TechLead |
| P-risk-6 | The [T-issue-N.M] decomposition in the TDD isn't tight enough for agents to execute atomically |
feasibility | TDD's phase decomposition is explicit about files, signatures, and acceptance tests per issue. Issues that turn out to be too coarse get split during R1 development. Owner reviews any issue that takes longer than expected. | TechLead |
Release Plan [P-release-plan]¶
R1 ships the framework v0 core. No solution UAT, no deployment templates, no eval adapter.
| Milestone | Target date | Owner | Customer-side dependency |
|---|---|---|---|
altaforge.llm package shipped + altaforge.core runtime + altaforge.logging |
✓ (R0 done 2026-05-25) | Platform team | — |
altaforge-docgen repo scaffolded (R1.1 foundation) |
✓ (#13) | DocGen TechLead | — |
Document CRUD + <DocumentView> + demo (R1.2) |
✓ (#14) | DocGen engineers | — |
Section CRUD + <SectionCard> + data_needs slot (R1.3) |
TBD | DocGen engineers | — |
| Tools + gather phase (R1.4) | TBD | DocGen engineers | — |
| Synthesize phase (R1.5) | TBD | DocGen engineers | — |
| Render phase + Arize-flagging (R1.6) | TBD | DocGen engineers | — |
engine.generate_document + markdown export (R1.7) |
TBD | DocGen engineers | — |
| Walkthrough tests + onboarding runbook (R1.8) | TBD | DocGen TechLead + AIA reviewer | — |
| Solution teams begin consuming R1 libraries | When R1.8 ships | Each solution lead | Solution repos scaffolded with submodule wiring |
Open Questions [P-q-N]¶
- P-q-1: Forward-compat for texts / snippets — when R2 adds them, the
content_outputs.kindenum grows from('prompt', 'tool')to add'text','risk_block', and snippet composition arrives as new tables. Locked: additive enum extension; no breaking change. SQLite + Postgres both support this. - P-q-2: Versioning — should prompts have a
versioncolumn from R1 (even if unused) so R4 doesn't require a schema migration? Locked: defer to R4 migration; R1 doesn't have version columns. - P-q-3: Synthesis mode default per organization — should the default be
none,raw_text, orstructured? Default:none(sections render directly againstgathered_data); a solution opts intoraw_textorstructuredwhen it has cross-section context to extract. - P-q-4: Per-developer cost cap on
altaforge.llmOAuth modes — does the provider enforce a default, or does R1 specify one? Default: vendor CLI / provider decides; R1 doesn't specify. - P-q-5: Gather-phase parallelism — should data needs run in parallel (asyncio) or sequentially in R1? Default: sequential in R1; parallel via asyncio is an R1.4 follow-up if measured latency hurts.
Next Steps:
| Action | Owner | Date |
|---|---|---|
| Solution leads review R1 PRD's rearchitect, flag missing shapes | All solution leads | TBD |
| Lock the R1 phase decomposition into atomic issues that agents can execute | TechLead | TBD |
Data Model [P-entity-N]¶
R1 ships the core entities. Schema details (column types, indexes, FK constraints) live in the TDD. Texts, snippets, and risk blocks defer to R2 (see Domain Model [D-11]).
[P-entity-1] Organization¶
- One per engagement.
- Holds: id, slug, display name, LLM provider/model config, default synthesis mode, eval-backend choice (used in R2+), per-org configuration JSON.
[P-entity-2] Document¶
- A template (
is_template=true) or an instance (is_template=false): a template is reusable, generatable config; an instance is a deep copy of a template (or ad-hoc), one per generated output. See[D-13]. - Holds: id,
organization_id, title, state (draft/under-review/finalized),is_template,created_from_template_id(provenance; null for templates + ad-hoc), synthesis mode override (optional), timestamps.
[P-entity-3] Section¶
- One per named region within a document. Owns two ordered action lists:
data_needs(gather,[P-entity-6]) +content_outputs(output,[P-entity-5]). - Holds: id,
organization_id(denormalized), parent document id,ordinal(immutable creation order;(document_id, ordinal)unique),position(mutable display order; sort key forlist_sections; rewritten byreorder_sections), title, status (aggregate of its outputs), timestamps. No content-recipe column and no stored content — the section's content is the ordered concatenation of itscontent_outputs[].result, composed on read.
[P-entity-4] Tool (registration)¶
- Per-organization registration of a
DataToolPython class. - Holds: id,
organization_id, name, category (web_search/page_scrape/file_ingest/mcp/ solution-specific),class_path(Python dotted path), non-secret config JSON. - Credentials are NOT stored — see
[D-9].
[P-entity-5] ContentOutput (output action)¶
- One output action on a section —
'prompt'(LLM render; may call tools) or'tool'(emit a tool's result deterministically, no LLM); R2 adds'text'/'risk_block'. Section-owned, prompt text inline so it deep-copies with a template. Replaces the old shared Prompt entity — there is no shared prompt table. - Holds: id,
organization_id(denormalized), section_id, ordinal,kind,prompt_body(inline, when kind=prompt),tools_available(JSON array of tool names the prompt may call),tool_ref(when kind=tool),args_json,result, status,sources_json(list ofSourcerecords —gathered_data_id+selectorpin — populated at render time; the audit chain that proves which gathered_data + URL/path/snippet fed this output; see[P-feat-13]+[D-14]),output_schema(optional JSON Schema / Pydantic dotted path; when it declares acitationfield, render validates the LLM-emitted citation against context).
[P-entity-6] DataNeed (gather action)¶
- One gather action on a section — what data it needs before rendering. Section-owned; same action core as
[P-entity-5]. - Holds: id,
organization_id(denormalized), section_id, ordinal,kind('tool'|'prompt'),prompt_body(inline, when kind=prompt),tools_available(when kind=prompt),tool_ref(when kind=tool),args_json,output_key.
[P-entity-7] GatheredData¶
- Per-document persisted result of one gather-action invocation. Deduped across sections.
- Holds: id,
organization_id(denormalized), document_id, source_category, source_ref (tool name, or the gather action's id for a prompt-kind need),args_hash,args_json,result_json, fetched_at.
[P-entity-8] SynthesizedContext¶
- Per-document synthesized payload. One row per document per run.
- Holds: id,
organization_id(denormalized), document_id, mode ('none'/'raw_text'/'structured'), payload_json (text blob for raw_text; serialized Pydantic for structured), model (LLM model used), created_at.
Sign-off¶
| Role | Name | Date |
|---|---|---|
| Author | Mark Ly | 2026-05-26 |
| Engineering reviewer | ||
| Approver | Mark Ly (CTO) |
Reviewer's checklist¶
- [ ] Domain Model
../../reference/domain-model.mdis the canonical vocabulary; every term used elsewhere is defined there - [ ] Walkthroughs cover cooking (sanity), Odyssey-shape (5 prompts + structured synthesis), and render-time tool drift
- [ ] Functional Requirements map 1:1 to TDD phases (1.1–1.8)
- [ ] Success Metrics each have an acceptance Given/When/Then
- [ ] Risks have mitigations and owners
- [ ] Open Questions are real (not resolved-but-listed)
- [ ]
P-feat-Nmapping toT-phase-Nis consistent - [ ] Text / snippet / risk_block scope is explicitly deferred to R2 in non-goals
When approved, the TDD (./r1-tdd.md) decomposes each [T-phase-N] into atomic [T-issue-N.M] stories that agents can execute.