Skip to content

Stories - DocGen Framework v0 Core (R1)

status: draft
date: 2026-05-26
author: AltaML TechLead
prd: ./r1-prd.md
tdd: ./r1-tdd.md
domain-model: ../../reference/domain-model.md
brief: ../../explanation/brief.md
epic: ../roadmap.md
parent-release: E-release-1
written-by: TechLead

Written by: TechLead. This doc decomposes the R1 TDD's architecture into atomic stories that an agent can execute in a single focused pass. Phases are interleaved Python + UI + demo — each phase ships a vertical slice that's both unit-testable and user-testable. Each story names files, signatures, and an acceptance test. Stories cite the [T-decision-N] rows in r1-tdd.md they implement.

How to read this

Audience Read
Dev agent (implementer) The story you're executing. Each [T-issue-N.M] is self-contained.
Architect agent Everything; this doc is the output of decomposition
Engineering reviewer TL;DR + skim phase boundaries; trust the architecture in r1-tdd.md
Sponsor signing off Skip — read r1-tdd.md instead

Length budget: longer than the TDD; this is the execution plan.

Change History

Date Author Change
2026-05-23 AltaML TechLead Initial draft — Python-only phase decomposition (10 phases, ~30 issues).
2026-05-24 AltaML TechLead Rewritten for interleaved Python + UI + demo phasing. 9 phases, each ships a vertical slice that's unit-testable AND user-testable via the demo app. altaforge.llm lifted to R0 prerequisite.
2026-05-26 AltaML TechLead Rearchitected to gather → synthesize → render pipeline. R1 phase count goes 9 → 8. Phases 1.4 (Texts) and 1.7 (Snippets) retired to R2. New 1.4 (Gather), 1.5 (Synthesize), 1.6 (Render — prompt-only). Old 1.5 (Prompts as content unit) absorbs into 1.6. Old 1.6 (Tools) reframed as the gather phase implementation. Old 1.8 (Engine + export) renumbers to 1.7; old 1.9 (Walkthroughs) renumbers to 1.8. Walkthroughs collapse from 4 to 3 (JBRK + mixed defer to R2 alongside texts / snippets).
2026-05-29 AltaML TechLead R1.6 picks up source traceability. New [T-issue-1.6.4] (sources_json fill + LLM-citation validation) and [T-issue-1.6.5] (bridge + demo source panel). Render stories now explicitly call out content_outputs.sources_json population (tool-kind: deterministic tool_input recording; prompt-kind: all-of-context unless output_schema.citation enforces narrowing). Bridge surfaces denormalized URL/title/snippet; <SectionCard> adds a "Sources" panel. Per [P-feat-13] + [T-decision-17] + docs/design/specs/2026-05-29-source-traceability.md.
2026-06-04 Mark Ly R1.4 / 1.5 / 1.6 / 1.7 marked shipped based on actual code state (mega-synth, doc-owned actions, HTTP bridge, all 5 generic tools landed: AddTool / AppendTool / ClaudeWebSearchTool / OpenAIWebSearchTool / WebPageTool; plus the fetch_pages opt-in on the two web-search tools backed by altaforge.llm.web_page). R1.8 reshaped to 3 real-flavor walkthroughs (Phoenix firm-order memo, candidate reference check report, DealDesk discount-approval memo) replacing the prior 4 toy walkthroughs (cooking / odyssey-shape / render-time-drift / template-instantiate). Every walkthrough doc ships as is_template=true and gets one instantiated client instance per template, so the template→preview→instantiate lifecycle is exercised by all three. Each walkthrough uses only the 5 generic tools (no client-specific adapters — those land in R3/R5).

TL;DR

  • R0 prerequisite (shipped in altaforge-libraries): altaforge.llm + altaforge.core runtime (incl. AltaForgeGraph) + altaforge.logging. Path-installed via vendor/altaforge-libraries/.
  • 10 phases (1.1, 1.2, 1.3, 1.3.5, 1.3.6, 1.4, 1.5, 1.6, 1.7, 1.8), ~47 atomic issues. Phases 1.1 through 1.7 shipped; 1.8 in flight as of 2026-06-04. Each phase ships a vertical slice (Python + UI + demo + tests). Stop at any phase and the demo runs against reduced functionality. User testing rides every phase alongside unit tests.
  • Pipeline shape: every R1 document flows collect_data_needs → gather → synthesize → fan-out render → persist. Phases 1.4 / 1.5 / 1.6 build the three pipeline stages; phase 1.7 wires them into one graph.
  • Each issue names files, signatures, and an acceptance test. Most issues sized for 1-3 hours of focused agent work.
  • Phases are mostly sequential; inner-phase Python and UI work can parallelize across two agents.

R0 Prerequisite — altaforge.llm + altaforge.core + altaforge.logging (in altaforge-libraries) — SHIPPED

Not altaforge-docgen stories. Tracked here because R1.4–R1.7 depend on them.

Owned by: AltaML (in altaml/altaforge-libraries/).

altaforge.llm — verified end-to-end 2026-05-25

Python package wrapping LiteLLM. Four auth modes:

  • oauth_anthropic — reads ~/.claude/.credentials.json (Claude Code's local token)
  • oauth_openai — reads ~/.codex/auth.json (Codex CLI's token); routes to OpenAI's Codex backend (public API rejects consumer JWTs)
  • oauth_gemini — reads ~/.gemini/oauth_creds.json (Gemini CLI's token); routes to Google's Code Assist endpoint
  • litellm — static API key + gateway URL (for CI); ALTAFORGE_LLM_API_KEY + ALTAFORGE_LLM_BASE_URL

The lib reads each vendor CLI's local credentials file on every call — it does not run OAuth, store its own tokens, or refresh. The vendor CLI owns its token lifecycle.

Surface: client(auth_mode=..., token=..., api_key=..., base_url=..., extra_headers=...).completion(model=..., messages=[...], ...). Async via acompletion. Other LiteLLM methods pass through via __getattr__.

altaforge.core runtime — AltaForgeGraph

Subclass of langgraph.graph.StateGraph that auto-wires OpenTelemetry + OpenInference's LangChainInstrumentor for Arize AX / Phoenix tracing on construction. Engine ships standard OrganizationScopedSession conventions. Used by R1.7 to build the document generation graph.

altaforge.logging

Stdlib logging setup + Grafana Loki forwarder (toggled by AF_LOKI_URL). Trace-correlation with Arize spans.

Distribution: path install (uv add ./vendor/altaforge-libraries/<pkg>) — no PyPI feed.

Phase Decomposition [T-phase-N]

R1 has 8 phases. Each phase ships Python + UI + demo + tests together. Phase exit criteria includes: code compiles, tests pass, AND demo runs end-to-end against the new feature.

[T-phase-1.1] Foundation — repo + workspaces + base schema — ✓ SHIPPED (#13)

Goal: clone-and-run works for a new dev in < 30 min. Repo has both Python and JS workspaces wired. Empty schema + multi-tenant isolation test passing.

Out of scope for this phase (and all subsequent R1 phases): GitHub Actions workflows under .github/workflows/. The AltaForge CI team owns those org-wide. Stories should never create or modify workflow files. Local pre-commit hooks (ruff, mypy, etc. via .pre-commit-config.yaml) are in scope — they're dev-machine tooling, not CI.

Story What shipped
1.1.1 Python project scaffold (uv + ruff + mypy + pytest + pre-commit); path deps to altaforge[core,llm,logging] (unified v0.2 package)
1.1.2 JS workspace (pnpm-workspace.yaml, root package.json, .npmrc); altaforge-ui submodule visible as workspace member
1.1.3 docgen-ui package skeleton (Vite library mode + vitest)
1.1.4 demo Next.js 14 app skeleton with /api/health
1.1.5 Base schema migration (organizations, documents, sections) per TDD [T-entity-1..3]
1.1.6 OrganizationScopedSession + multi-org isolation tests

Merged 2026-05-26 in PR #13.

[T-phase-1.2] Documents — CRUD + DocumentView + demo drill-in — ✓ SHIPPED (#14)

Goal: Document CRUD via Python API; <DocumentView> component; demo shows the document list and a drill-in shell.

Story What shipped
1.2.0 Shared conftest.py fixtures (engine, organization, second_organization, session)
1.2.1 Document CRUD (create / list / get / update_state / delete) scoped via OrganizationScopedSession
1.2.2 <DocumentView> React component — title + state badge + section placeholder
1.2.3 Demo wiring — docgen.bridge (Python JSON-stdout CLI) + demo/lib/docgen-bridge.ts (Node subprocess) + API routes + homepage + /docs/[id]/page.tsx

Merged 2026-05-26 in PR #14.

[T-phase-1.3] Sections — CRUD + <SectionCard> + data_needs slot — ✓ SHIPPED (88c5767 / e672a9d / 5fd9551)

Goal: CRUD sections inside an existing document; <SectionCard> renders one; the section spec gains a data_needs slot (the table lands here; population happens in 1.4); demo's document drill-in shows the section list with status badges.

Story What shipped
1.3.1 Section CRUD (create_section / list_sections / get_section / update_section / delete_section) in docgen/src/docgen/sections.py with auto-increment ordinal; org_id inherited from parent Document; 12 integration tests in test_sections_crud.py
1.3.2 DataNeed model + data_needs table per TDD [T-entity-6] (migration 20260526_1200_add_data_needs.py); DataNeedSpec Pydantic in docgen/src/docgen/specs.py; 4 schema round-trip tests in test_data_needs_crud.py (raw inserts with synthetic source ids until R1.4/R1.6)
1.3.3 Migration 20260526_1300_section_content_recipe.py — drops inline_text_id / inline_prompt_id / snippet_id / role_filter columns + the ck_sections_one_content_source CHECK; adds content_recipe_type (NOT NULL, default 'prompt') + content_recipe_id (NOT NULL); SQLite handled via batch_alter_table. Downgrade restores R1.1.5 shape. FK constraint on content_recipe_id deferred to R1.6 per [T-decision-3].
1.3.4 <SectionCard> in docgen-ui/src/SectionCard/SectionCard.tsx (props: title, content, status, dataNeeds, onEdit, onRegenerate; status badge across all 5 states; collapsible data-needs panel; conditional action buttons); <DocumentView> updated to map sections to <SectionCard> by default (custom renderSection prop still wins); 9 vitest cases + 5-section ordering test in DocumentView.test.tsx
1.3.5 docgen.bridge extended with list_sections / get_section + fixture-section seeding (3 sections per fixture doc with mixed statuses); demo/lib/docgen-bridge.ts adds typed listSections / getSection; new demo/app/api/sections/route.ts; demo/app/docs/[id]/page.tsx fetches sections server-side and passes to <DocumentView>

42 Python tests + 22 docgen-ui tests pass; ruff + mypy + tsc clean. Browser-rendered drill-in verified — 3 <SectionCard> instances with status badges, content, and proper ordering.

Merged 2026-05-26 directly to dev (no PR).

[T-phase-1.3.5] Action-ownership refactor — section-owned actions, no shared prompts — ✓ SHIPPED

Goal: sections own their actions — data_needs (gather) + content_outputs (output), prompt text inline; drop the shared prompts table and sections.content_recipe_* / current_content. Foundation for templates (1.3.6) and the rest of the pipeline (1.4+). Lands before 1.4 because gather/render build on this shape. Reshapes the just-shipped 1.3 schema. Per [D-4]/[D-5]/[D-13] + [T-decision-3]/[T-decision-4].

Story What
1.3.5.1 Migration: sections drop content_recipe_type / content_recipe_id / current_content (status stays as aggregate); data_needs + organization_id, replace source_type/source_id with kind / prompt_body / tools_available / tool_ref; new content_outputs table ([T-entity-5])
1.3.5.2 Models + specs: Section (no recipe cols), DataNeed (action core + output_key), new ContentOutput; Pydantic ContentOutputSpec, reshaped DataNeedSpec; errors.py hierarchy (DocGenError/NotFoundError/ValidationError/ConflictError)
1.3.5.3 OrganizationScopedSession scopes ContentOutput; section content derived from content_outputs[].result (composed on read); action CRUD (data_needs + content_outputs)
1.3.5.4 engine/render.py render_output (prompt→LLM with tools_available; tool→deterministic), replacing the prompt-only path; tests
1.3.5.5 Bridge + demo + <SectionCard> updated to the action shape (per-section outputs list); tests

[T-phase-1.3.6] Document templates — is_template + instantiate_template — ✓ SHIPPED

Goal: configure once, stamp instances ("Proposal — Client A" / "— Client B"). Snapshot deep-copy; nothing shared. Per [D-13] + [T-decision-16]. Depends on 1.3.5 (the owned subtree).

Story What
1.3.6.1 Migration: documents + is_template (bool, default false) + created_from_template_id (nullable self-FK); index (organization_id, is_template)
1.3.6.2 templates.py: instantiate_template(template_id, title) deep-copies document → sections → (data_needs + content_outputs), prompt text verbatim, clears result/status; list_templates; ValidationError on non-template; tests (deep-copy correctness, result cleared, provenance set, org inherited, cross-org not-found)
1.3.6.3 Bridge + demo: template list + "New from template" → instantiate → drill into the fresh instance

[T-phase-1.4] Gather — DataTool ABC + 5 generic built-ins + gather node — ✓ SHIPPED

Built-ins as landed (broader than the original 4): AddTool (math), AppendTool (text), ClaudeWebSearchTool (Anthropic server-side via altaforge.llm.anthropic_web_search), OpenAIWebSearchTool (Azure OpenAI Responses-API, modes fast/agentic/deep_research), WebPageTool (single-URL fetcher via altaforge.llm.web_page). The two web-search tools accept fetch_pages=True to attach WebPage bodies to citations in one call. WebSearchTool / PageScrapeTool / FileIngestTool / MCPTool source files stay on disk but are deferred (not exported from docgen.tools) until a consumer needs them.

Goal: the engine can invoke a document's data_needs, dedupe across sections, and persist results to gathered_data. Demo shows gathered data per section.

[T-issue-1.4.1] DataTool ABC + StubTool + tool registry

Files: docgen/src/docgen/tools/{__init__,base,mock}.py (DataTool ABC, StubTool, register_tool/get_tool registry); extend docgen/src/docgen/data/models.py with Tool per TDD [T-entity-4]; migration <ts>_add_tools.py; docgen/tests/unit/test_tools.py.

Implementation: Per TDD [T-api-6] and [T-decision-9]. DataTool ABC declares name, category (free-form tool-category descriptor, distinct from DataNeed.kind action discriminator), args_schema (Pydantic), output_schema (Pydantic), invoke(args) -> result. StubTool (test-only, under tests/) returns fixture data. register_tool(tool) adds to an in-process Python dict; get_tool(name) looks up by name. The tools DB table stores registration metadata (name, category, class_path, non-secret config) — credentials never live in the DB.

Acceptance [T-test-1.4.1]: StubTool(name='foo', fixture={'result': 'bar'}); register_tool(mock); get_tool('foo').invoke({}) == {'result': 'bar'}. Tools table created with unique (org_id, name).

[T-issue-1.4.2] Four built-in DataTool implementations

Files: docgen/src/docgen/tools/web_search.py (WebSearchTool), page_scrape.py (PageScrapeTool), file_ingest.py (FileIngestTool), mcp.py (MCPTool); docgen/.env.example listing required env-var names; unit tests for each (mocked external IO).

Implementation: Per Brief [B-outcome] and TDD [T-decision-9]. Each tool's constructor takes credentials as kwargs (e.g. WebSearchTool(api_key=...)). Solutions inject credentials from env vars at register_tool time; the tool itself reads no env vars internally. Minimal depth — happy-path implementation; R2 deepens each.

  • WebSearchTool — HTTP call to a search provider (Tavily / Brave / etc., configurable via provider kwarg)
  • PageScrapeTool — httpx.get(url) + BeautifulSoup parse; returns text + metadata
  • FileIngestTool — local/remote file read; auto-detects .pdf / .docx / .md / .txt via extension
  • MCPTool — JSON-RPC call to an MCP server (server_url + optional token kwargs)

Acceptance [T-test-1.4.2]: Each tool's unit test exercises the happy path against a mocked external endpoint; failure modes (timeout, auth fail) raise typed exceptions. .env.example documents every env-var name.

[T-issue-1.4.3] gathered_data table + engine.gather pure function

Files: Extend data/models.py with GatheredData per TDD [T-entity-7]; migration; docgen/src/docgen/engine/gather.py (pure gather(document, variables, *, organization) -> dict); docgen/tests/integration/test_gather_phase.py.

Implementation: Per TDD [T-api-2] and [T-decision-12]. The pure function: 1. Loads the document's sections + their owned data_needs (ordinal order) via OrganizationScopedSession. 2. Resolves each data_need's args_json against variables (Jinja). 3. Dispatches on the action kind: tool → get_tool(tool_ref).invoke(args); prompt → run prompt_body against the gather context (one altaforge.llm call, which may itself invoke tools listed in tools_available mid-call). 4. Dedupes by args_hash = sha256(canonical_json(kind, tool_ref or prompt_body, args)) so two sections declaring the identical invocation run it once. 5. Persists each result to gathered_data (keyed by (document_id, args_hash)). 6. Returns {output_key: result, ...} keyed by every data_need's output_key.

Pure function — no graph involvement; nodes wrap it in 1.7.

Acceptance [T-test-1.4.3]: Document with two sections each declaring the same kind='tool' data_need (tool_ref='mock_search', args={'q':'X'}) → StubTool.invoke called exactly once; gathered_data has one row; both sections receive the same output_key value. A second section declares a kind='prompt' data_need (prompt_body + tools_available) → its result lands under its own output_key. Per [P-eval-1].

[T-issue-1.4.4] Demo: gathered data visible per section

Files: Extend demo/app/docs/[id]/page.tsx to show gathered data summary per section; extend docgen.bridge with a gather_for_document shim; demo seeds a fixture doc whose sections use StubTool + FileIngestTool.

Acceptance [T-test-1.4.4]: Demo's document drill-in shows a "Gathered data" panel listing each output_key and its action kind (tool → tool_ref; prompt → prompt_body excerpt) + args.

[T-phase-1.5] Synthesize — mega-synth structured call + SynthesizedContext + 2 modes — ✓ SHIPPED

As landed: synthesize collapsed to a single structured LLM call ("mega-synth") that fills a dynamic Pydantic schema built from every section's render_mode='from_synth' field. none mode still supported as a no-op skip; raw_text retained as a fallback. Per the 2026-05-31 doc-owned-actions refactor.

Goal: the engine can produce a SynthesizedContext from gathered_data. Two R1 modes: raw_text (LLM produces a text blob) and structured (LLM produces a Pydantic facts object). none is also supported as a no-op skip. Demo shows the synthesized context preview.

[T-issue-1.5.1] synthesized_context table + mode columns on documents

Files: Extend data/models.py with SynthesizedContext per TDD [T-entity-8]; extend Document model with synthesis_mode_override + structured_schema_path columns per [T-entity-2]; extend Organization with default_synthesis_mode; migration.

synthesized_context is persisted — one row per document — as a per-generation runtime artifact (audit, fault-localization, regenerate-without-re-synthesizing). It is not part of the template deep-copy in 1.3.6. Per [T-decision-13].

Acceptance: Schema additions round-trip; document creation defaults to org's default_synthesis_mode='none'; synthesizing a document persists exactly one synthesized_context row.

[T-issue-1.5.2] engine.synthesize(document, gathered, *, organization) — pure function

Files: docgen/src/docgen/engine/synthesize.py (pure synthesize(document, gathered, *, organization) -> SynthesizedContext); docgen/tests/integration/test_synthesize_phase.py (@pytest.mark.llm for the LLM-backed modes).

Implementation: Per TDD [T-api-3] and [T-decision-10]. Resolve mode from document.synthesis_mode_override or organization.default_synthesis_mode. Dispatch:

  • mode='none' → returns SynthesizedContext(mode='none', payload=gathered) without persisting.
  • mode='raw_text' → one altaforge.llm call with a system prompt that summarizes gathered into a coherent narrative; persists.
  • mode='structured' → loads the Pydantic class at document.structured_schema_path; one altaforge.llm call with response_format set to that class; parses + validates the response; persists.

Pure function — no graph involvement.

Acceptance [T-test-1.5.2]: Two fixture documents — one raw_text, one structured against a fixture DemoFacts(client_name, top_topics) schema. Each runs end-to-end against the real LLM. synthesized_context.payload_json parses correctly per mode. Per [P-eval-2].

[T-issue-1.5.3] <SynthesizedContextPreview> component

Files: docgen-ui/src/SynthesizedContextPreview/SynthesizedContextPreview.tsx — collapsible JSON viewer; mode badge; "regenerate synthesis" button. Extend <DocumentView> to render it above the section list.

Acceptance [T-test-1.5.3]: Snapshot tests for each mode (none / raw_text / structured); JSON preview renders.

[T-issue-1.5.4] Demo: synthesis visible in drill-in

Files: Extend demo/app/docs/[id]/page.tsx to render <SynthesizedContextPreview>; extend docgen.bridge with a synthesize_for_document shim. Fixture: one document configured mode='structured' against a tiny Pydantic schema.

Acceptance [T-test-1.5.4]: Demo drill-in shows synthesized context preview matching the configured mode.

[T-phase-1.6] Render — per-section dispatch by render_mode + source traceability — ✓ SHIPPED

As landed: the content_outputs table was removed in the 2026-05-31 doc-owned-actions refactor in favor of each section owning a single prompt + render_mode + output_template + output_type. Render dispatches per-section by render_mode (from_synth reads the mega-synth field; template uses Jinja over texts/snippets; prompt calls the LLM with the section's prompt + gathered context). Source traceability is still output-level — every rendered section records the gathered_data IDs it consumed; bridge surfaces denormalized URL/title/snippet.

Goal: the engine can render a section by filling each of its owned content_outputs over the synthesized context; the section's content is the composed concatenation of those content_outputs.result. Prompt outputs may invoke tools during render — those calls emit a flagging Arize span. Every rendered output also records its sources — the list of gathered_data rows it consumed, with optional fragment selectors — to content_outputs.sources_json, recorded from the INPUT side at render time (per [T-decision-17]). Demo shows section content rendered live + a "Sources" panel per section with one clickable URL/path/snippet per output.

No prompts table: prompt text lives inline on each content_outputs action (prompt_body), owned by the section and deep-copied with it (the shared prompts table was removed in 1.3.5). There is therefore no standalone "prompts table" story here, and no sections.content_recipe_* FK to wire up.

[T-issue-1.6.1] engine.render_section(section, context, *, organization) — fill content_outputs

Files: docgen/src/docgen/engine/render.py (extend render_output from 1.3.5.4 into render_section); docgen/tests/integration/test_render_phase.py (@pytest.mark.llm).

Implementation: Per TDD [T-api-4], [T-decision-7], [T-decision-11]. Iterate the section's content_outputs in ordinal order and dispatch on each action's kind:

  • kind='prompt' → Jinja-interpolate context into prompt_body; construct an LLM client via altaforge.llm.client(auth_mode=organization.llm_auth_mode); call .completion(model=organization.llm_model, messages=[...], temperature=0, tools=tools_available); three-attempt retry on transient failure.
  • kind='tool' → get_tool(tool_ref).invoke(args_json) — deterministic, no LLM round-trip; the result is the output verbatim.

Write each action's result + per-output status back to its content_outputs row. The section's rendered content is derived = the ordered concatenation of content_outputs.result (composed on read; no stored current_content).

Render-time tool-call flagging: each kind='prompt' LLM call wraps its execution in an OTel span. If any tool is invoked during this call, the engine sets the span attribute render_time_tool_call=true and includes the tool name + args. Reviewers see the flag in Arize and can promote the call to a data_need if it bloats cost.

Pure function — no graph involvement; the node wraps it in 1.7. Source recording (population of content_outputs.sources_json) is added in [T-issue-1.6.4].

Acceptance [T-test-1.6.1]: Section with two content_outputs: one kind='prompt' with tools_available=['mock_search'] invoked during render, one kind='tool' (tool_ref='mock_search'). Assert: (a) the prompt action's OTel span has render_time_tool_call=true; (b) each action's result is populated + status='generated'; (c) the composed section content concatenates both results in ordinal order. Per [P-eval-3] + [P-eval-5].

[T-issue-1.6.2] SectionCard generating/failed states

Files: Extend docgen-ui/src/SectionCard/SectionCard.tsx — generating spinner; failed UI with retry button; rendered markdown content with react-markdown (the composed content_outputs.result).

Acceptance: Each status state renders distinctly; markdown content (headings, lists) renders visibly.

[T-issue-1.6.3] Demo: section content rendered live by Claude

Files: Extend demo/app/docs/[id]/page.tsx to trigger render per section; extend docgen.bridge with a render_section shim. Fixture: Odyssey-shape document.

Acceptance [T-test-1.6.3]: Demo shows a section rendered by Claude live (generating → generated transition) — its content_outputs fill and compose into the section's content. @pytest.mark.llm.

[T-issue-1.6.4] Source recording — fill content_outputs.sources_json + optional LLM-citation validation

Files: Migration <ts>_add_content_outputs_sources_json.py (adds sources_json text/JSON NOT NULL default '[]', and output_schema text/JSON nullable, to content_outputs); docgen/src/docgen/specs.py adds Source Pydantic + UsedAs literal; extend docgen/src/docgen/engine/render.py with record_sources (per [T-api-9]) + CitationValidationError (in errors.py); extend render_output to call record_sources before commit; docgen/tests/integration/test_render_phase.py adds source-recording cases (@pytest.mark.llm for the citation-validation case).

Implementation: Per [T-api-9], [T-decision-17], and docs/design/specs/2026-05-29-source-traceability.md. INPUT-side recording — for kind='prompt', walk Jinja-referenced output_keys in prompt_body and emit one Source(used_as='prompt_context', selector=None) per matched gathered_data row in the index; for kind='tool', do the same against args_json with used_as='tool_input'. Render-time tool calls (per [T-decision-11]) add Source(used_as='tool_result', selector=None) entries on the triggering output. When content_output.output_schema declares a citation: {gathered_data_id, selector} field, the LLM's emitted citation is parsed against the schema, validated (gathered_data_id must be in the recorded Source list; selector must resolve against gathered_data.result_json), and narrows the matching Source's selector in place. An unmatched id or unresolvable selector raises CitationValidationError; the output's status='failed' and no partial citation persists. Selector resolver is per source_category ({result_index: N} for web_search; {page, char_range} for file_ingest; {json_path: ...} for mcp).

Acceptance [T-test-1.6.4]: (a) One web_search source, no citation validation — a kind='prompt' content_output whose prompt interpolates one gathered_data row (5-hit web_search result) renders → sources_json has exactly one Source with gathered_data_id set, used_as='prompt_context', selector=None. (b) Citation validation happy path — same output with output_schema declaring citation; LLM returns selector={"result_index": 2} → engine validates against gathered_data.result_json[2] and persists the narrowed selector on the existing Source (no extra Source added). (c) Hallucinated citation — LLM returns a gathered_data_id not in the context → CitationValidationError raised, status='failed', sources_json not partially persisted (transactional). (d) Wide prompt many-sources — prompt interpolates 4 rows, no citation validation → sources_json has 4 Source records all with selector=None. (e) Render-time tool call — kind='prompt' output triggers a tool mid-render → an additional Source(used_as='tool_result') lands. (f) Multi-tenant — two orgs, each with a rendered output; loading either org's output under the other's scoped session returns not-found; Source records never reference cross-org gathered_data_ids. Per [P-feat-13] + [P-eval-6].

[T-issue-1.6.5] Bridge + <SectionCard> sources panel — denormalize selector → URL/title/snippet

Files: docgen.bridge adds sources-for-output <output_id> command (loads content_outputs.sources_json, resolves each Source's selector against its gathered_data.result_json, returns the enriched payload per source-traceability spec [5]); demo/lib/docgen-bridge.ts adds getOutputSources(outputId): Promise<EnrichedSource[]>; demo/app/api/sections/[id]/sources/route.ts (or output-keyed equivalent); extend docgen-ui/src/SectionCard/SectionCard.tsx with a collapsible "Sources" panel rendering one row per output's sources — clickable URL for web_search, file/page label for file_ingest, MCP json_path + snippet for mcp, fallback "context" badge when selector=None; vitest snapshot tests.

Acceptance [T-test-1.6.5]: Demo's document drill-in shows, beneath each rendered section, a "Sources" panel that lists each output's sources as URL/path/snippet (denormalized from the underlying gathered_data row, not from a JOIN at the UI). Click a web_search source → opens the article URL in a new tab. A wide-prompt output shows all its sources as a list (no false single-citation). A render-time tool-result source renders with a "tool-result" badge.

[T-phase-1.7] Engine — generate_document + LangGraph wiring + markdown + .docx export — ✓ SHIPPED

Extended at landing: .docx export via python-docx shipped alongside .md (originally a R2 deliverable; pulled forward because the demo needed a downloadable artifact). HTTP bridge (long-lived FastAPI service on 127.0.0.1:8765) replaced the per-call subprocess pattern; locked in by [T-decision-15].

Goal: top-level generate_document(org_slug, spec, variables) API orchestrates gather → synthesize → render across a document's sections end-to-end via an AltaForgeGraph StateGraph. Works on both document instances and templates — a template (is_template=true) is generated against sample variables to preview/validate that its prompts work, before instantiation. Markdown export downloads a .md file. Per-node Arize spans visible in traces.

[T-issue-1.7.1] DocGenState + node wrappers + build_document_graph

Files: docgen/src/docgen/engine/state.py (DocGenState TypedDict — organization_id, document_id, variables, data_needs, gathered, synthesized_context, pending_sections, rendered_sections); docgen/src/docgen/engine/nodes.py (thin wrappers per node — collect_data_needs_node, gather_node, synthesize_node, render_section_node, persist_section_node); docgen/src/docgen/engine/graph.py (build_document_graph(organization) -> CompiledGraph); docgen/tests/unit/test_nodes.py + docgen/tests/unit/test_graph_construction.py.

Implementation: Per TDD [T-decision-1] + [T-decision-8]. Pure functions stay in their files (engine.gather, engine.synthesize, engine.render). Nodes are 3-5 line wrappers that translate between DocGenState and pure-function args. Graph wiring: collect_data_needs → gather → synthesize → fan-out (per pending section: render_section → persist_section) → END. render_section_node fills the section's content_outputs; persist_section_node writes their result/status (section content composed on read). AltaForgeGraph's constructor auto-wires OTel + OpenInference; every node + LLM call becomes an Arize span. No new LLM calls in this story — the graph just orchestrates existing pure functions.

Acceptance [T-test-1.7.1]: Unit test builds the graph with a fake organization, inspects compiled-graph node names + edges. Separate unit test invokes the graph with a synthetic state (no DB) and verifies each node moves the right field.

[T-issue-1.7.2] engine.generate_document(org_slug, spec, variables=None) — top-level API

Files: docgen/src/docgen/engine/generate.py (the public function per TDD [T-api-1]); docgen/src/docgen/specs.py (Pydantic DocumentSpec, SectionSpec); docgen/tests/integration/test_generate_document.py (@pytest.mark.llm).

Implementation: Loads the organization, calls build_document_graph(organization), builds the initial DocGenState from the spec + variables, invokes graph.invoke(initial_state), persists each section's filled content_outputs, returns the populated Document. Runs identically whether the target document is an instance or a template (is_template=true) generated against sample variables for preview/validation — same graph, same nodes.

Acceptance [T-test-1.7.2]: A DocumentSpec with 3 sections (each with data_needs + one or more content_outputs) → returned Document with all sections' content_outputs filled and composed content, status='generated'. A template generated against sample variables likewise populates content_outputs (preview), and the template stays is_template=true. Tracing disabled in test env (ALTAFORGE_TRACING_DISABLED=true).

[T-issue-1.7.3] engine.export_markdown(document_id)

Files: docgen/src/docgen/engine/export.py — pure export_markdown(document_id) -> str; tests.

Acceptance: Generate doc, export → markdown string with # Title + ## Section Title + content per section.

[T-issue-1.7.4] <ExportButton> component

Files: docgen-ui/src/ExportButton/ExportButton.tsx; tests.

Acceptance: Click triggers download (mocked in test).

[T-issue-1.7.5] Demo: end-to-end configure → generate → export

Files: demo/app/configure/page.tsx (section spec editor: title, data_needs, content_outputs with inline prompt_body); extend demo/app/docs/[id]/page.tsx (generate + export buttons); demo/app/api/documents/[id]/{generate,export}/route.ts.

Acceptance [T-test-1.7.5]: Demo flow: /configure → fill spec → Generate → drill in → see gathered data + synthesized context + rendered sections → Export → download .md.

[T-phase-1.8] Walkthroughs + onboarding — 3 real-shape templates + runbook

Goal: three real-shape templates ship as seeded is_template=true documents in the demo org, each demonstrating a different client engagement pattern using only the 5 generic tools (no client-specific adapters — those land in R3/R5). Every template gets one instantiated "Client A" instance on first reset, so the template→preview→instantiate lifecycle is exercised by every walkthrough. Demo walkthrough pages render each template; an onboarding runbook lets a fresh dev clone-to-first-document in < 30 min.

Templates (each ships with texts, snippets, data_needs, sections, sample variables):

  1. Phoenix Firm Order Memo — Odyssey reinsurance underwriting; 5 sections (Approval Source, Deal Type, Layer Detail, Pricing/Cap, Special Terms). Stubbed Phoenix/EFS data via kind='prompt' data_needs with hardcoded sample text (real adapters are R3). Demonstrates structured 5-paragraph format with hard rules (no fabricated dates/approvals).
  2. Reference Check Report — pre-employment candidate reference check; 4 sections (Candidate Summary, Referee Quotes, Strengths, Concerns/Risks). Uses claude_web_search to research candidate's public profile + kind='prompt' data_needs to synthesize sample referee responses. Demonstrates web-research-driven generation.
  3. DealDesk Discount Approval Memo — sales-ops discount/exception pricing approval; 4 sections (Customer Snapshot, Deal Economics, Risk Factors, Recommendation). Uses openai_web_search for customer research + AddTool for deal math + kind='prompt' for synthesis. Demonstrates numeric-tool integration alongside research.

[T-issue-1.8.1] Phoenix template fixture

Files: docgen/src/docgen/fixtures/phoenix.py (ensure_phoenix_template(engine, org) -> Document, register_phoenix_texts_snippets); docgen/tests/unit/test_phoenix_fixture.py.

Spec: Document(is_template=True, title='Phoenix Firm Order Memo'). 5 sections, all render_mode='from_synth'. Texts: cedant_name, broker_name, deal_id, effective_date. Snippets: signature_block, confidentiality_footer. Data needs:

  • prompt need phoenix_program_summary — hardcoded sample Phoenix program-summary prose in prompt_body ("system message" pattern: tells the LLM to echo back the supplied program-summary text verbatim into the output_key). Mimics a future phoenix_tool adapter call shape.
  • prompt need phoenix_efs_approval — hardcoded sample EFS approval comment + email body; output_key efs_approval_text.
  • tool need claude_web_search query '<cedant_name> reinsurance program 2026' for cedant-context grounding (real network call when refreshed).

Acceptance: ensure_phoenix_template(engine, org) is idempotent; returns the template Document with 5 sections + 3 data_needs + the 4 texts + 2 snippets registered. Re-call is a no-op.

[T-issue-1.8.2] Reference Check template fixture

Files: docgen/src/docgen/fixtures/reference_check.py; docgen/tests/unit/test_reference_check_fixture.py.

Spec: Document(is_template=True, title='Candidate Reference Check Report'). 4 sections, all render_mode='from_synth'. Texts: candidate_name, role_applied, referee_names (comma-separated list). Snippets: reference_check_disclaimer. Data needs:

  • tool need claude_web_search query '<candidate_name> <role_applied> linkedin' for candidate public profile.
  • prompt need sample_referee_responses — hardcoded sample referee response transcripts (3 referees, 5 questions each) in prompt_body; output_key referee_transcripts.
  • prompt need risk_lens — instructs the LLM to extract risk flags from the transcripts.

Acceptance: Fixture seeds 4 sections + 3 data_needs + 3 texts + 1 snippet. Idempotent.

[T-issue-1.8.3] DealDesk template fixture

Files: docgen/src/docgen/fixtures/dealdesk.py; docgen/tests/unit/test_dealdesk_fixture.py.

Spec: Document(is_template=True, title='DealDesk Discount Approval Memo'). 4 sections, all render_mode='from_synth'. Texts: customer_name, list_price (numeric), requested_discount_pct (numeric), deal_size_usd (numeric). Snippets: dealdesk_signature_block. Data needs:

  • tool need openai_web_search query '<customer_name> revenue funding employee count' for customer research.
  • tool need add with {a: <list_price>, b: -<deal_amount>} (or appropriate arithmetic) for net-price math.
  • prompt need risk_lens — instructs the LLM to enumerate deal-risk factors from the research + texts.

Acceptance: Fixture seeds 4 sections + 3 data_needs + 4 texts + 1 snippet. Idempotent.

[T-issue-1.8.4] Wire templates into bridge reset_demo

Files: Extend docgen/src/docgen/bridge.py (cmd_reset_demo) to call the three new ensure_*_template functions after the existing ensure_document_1; for each template, also instantiate one client instance ("Phoenix — Sample Deal", "Reference Check — Jane Doe", "DealDesk — Acme Corp") via instantiate_template. Update docgen/tests/integration/test_bridge.py to assert the 4 documents (1 demo + 3 templates + 3 instances = 7 total) exist post-reset.

Acceptance: Post-reset: demo org has 1 demo document + 3 templates + 3 instances. Each instance carries its template's prompts verbatim, with result cleared per instantiate_template contract.

[T-issue-1.8.5] Demo walkthrough showcase pages

Files: demo/app/walkthroughs/page.tsx (index listing all 3 walkthroughs); demo/app/walkthroughs/{phoenix,reference-check,dealdesk}/page.tsx (one per walkthrough); navigation entry in <DocGenSidebar />.

Acceptance: Each walkthrough page shows the template's texts/snippets/data_needs/sections structure + a "New from this template" button that calls instantiate_template and navigates to the new instance. The instance drill-in shows the gather → synthesize → render flow visible per section.

[T-issue-1.8.6] Onboarding runbook (R1.8 P-metric-3)

Files: docgen/README.md (Python-side quickstart: clone → uv sync → .env.local setup → pnpm dev); repo-root README.md updated with: 5-minute clone-to-running steps, link to the 3 walkthroughs as the "what you can do" surface, troubleshooting for the common port-stuck / submodule-missing cases.

Acceptance [T-test-1.8.6]: P-metric-3 verification — a fresh dev clones the repo, follows the runbook, reaches a working demo (with at least one of the 3 walkthrough templates rendered end-to-end) in < 30 min. Verified by walking through the runbook on a clean Windows + clean Mac/Linux env.

Phase summary

Phase Stories Demo capability after
1.1 Foundation ✓ 6 "Hello tenant" page renders; multi-org isolation verified
1.2 Documents ✓ 4 List docs, drill into one
1.3 Sections ✓ 5 View section list inside a document; data_needs slot visible
1.3.5 Action-ownership refactor ✓ 5 Sections own data_needs + content_outputs; prompt text inline
1.3.6 Document templates ✓ 3 is_template + instantiate_template; "New from template" demo flow
1.4 Gather ✓ 4 Gathered data panel per section; dedup visible. As-built: 5 generic built-ins (Add / Append / ClaudeWebSearch / OpenAIWebSearch / WebPage), the two web-search tools with fetch_pages= opt-in
1.5 Synthesize ✓ 4 Synthesized context preview per document; mega-synth structured-call collapsed schema
1.6 Render ✓ 5 LLM-rendered section content visible; per-output "Sources" panel; per-section render dispatch by render_mode (from_synth/template/prompt)
1.7 Engine + Export ✓ 5 End-to-end via AltaForgeGraph: configure → gather → synthesize → render → export .md / .docx; HTTP bridge replacing per-call subprocess
1.8 Walkthroughs + onboarding 6 3 real-shape templates (Phoenix / Reference Check / DealDesk) seeded as is_template=true with one instance each; demo showcase pages; < 30 min onboarding runbook

~47 atomic stories total; ~41 shipped (1.1 + 1.2 + 1.3 + 1.3.5 + 1.3.6 + 1.4 + 1.5 + 1.6 + 1.7); 6 remaining in 1.8. Each phase is shippable / demoable at its boundary.

Cross-references

Cross-ref check (PRD [P-feat-N] → stories)

  • P-feat-1 (schema + multi-tenant) → [T-issue-1.1.5], [T-issue-1.1.6] ✓
  • P-feat-2 (document CRUD + DocumentView + demo doc list) → [T-issue-1.2.1..3] ✓
  • P-feat-3 (section CRUD + data_needs slot + SectionCard) → [T-issue-1.3.1..5]
  • P-feat-4 (tools + gather phase) → [T-issue-1.4.1..4]
  • P-feat-5 (synthesize phase) → [T-issue-1.5.1..4]
  • P-feat-6 (render phase + Arize-flagging) → [T-issue-1.6.1..3]
  • P-feat-13 (source traceability — output-level) → [T-issue-1.6.4] (record_sources + LLM-citation validation) + [T-issue-1.6.5] (bridge + SectionCard sources panel); eval P-eval-6 lives under [T-test-1.6.4]
  • P-feat-7 (generate_document + markdown export) → [T-issue-1.7.1..5]
  • P-feat-8 (walkthroughs + runbook + .env.example) → [T-issue-1.8.1..6] (.env.example shipped in 1.4.2)
  • P-feat-9 (docgen-ui) → distributed across 1.2.2 / 1.3.4 / 1.5.3 / 1.6.2 / 1.7.4
  • P-feat-10 (demo Next.js) → distributed across 1.1.4 / 1.2.3 / 1.3.5 / 1.4.4 / 1.5.4 / 1.6.3 / 1.7.5 / 1.8.5

Sign-off

Role Name Date
Author AltaML TechLead 2026-05-26
Engineering reviewer
Approver Mark Ly (CTO)