TDD - Odyssey Phoenix Firm Order Memo Generator — UAT (R3)¶

status: draft
date: 2026-05-20
author: Mark Gleason (Odyssey tech lead)
prd: ./r3-prd.md
brief: ../../explanation/brief.md
epic: ../roadmap.md
parent-release: E-release-3
written-by: TechLead
source-spec: ../reference/Phoenix_Firm_Order_Memo_Spec.docx

STALE — needs revalidation before R3 work starts. This TDD was written against the pre-2026-05-26 R1 architecture (texts / snippets / inline_ routing on sections, separate engine.py render dispatch). R1 was rearchitected to a gather → synthesize → render pipeline with prompts only; texts / snippets / risk_blocks moved to R2. What stays valid: Odyssey-specific tables (firm_order_memos, approval_records, readme_versions), the FastAPI routes, the Next.js app layout, the approval-detection algorithm (rapidfuzz on EFS comments + pdfplumber for typed PDF dates + lookup-table phrase selection — [T-decision-1]), the read-me-in-Postgres pattern ([T-decision-2]), the Phoenix self-host deployment shape, and Odyssey SSO auth. What needs revalidation:* how the Odyssey solution invokes DocGen primitives (it'll be data_needs + a content_recipe=prompt per paragraph in the new world, not inline_text/inline_prompt), the eval adapter wiring (R2-deferred from R1), and the Compose recipe (R2-deferred from R1). When R3 starts, rewrite the framework-integration sections against the then-current R1+R2 state; preserve everything else.

Update (2026-06-10) — divergence, not just staleness. The framework shipped as a single-repo showcase, not the separate odyssey-firm-order-memo/ monorepo this TDD describes. The Phoenix memo is a seeded template inside the demo/ app + docgen package (see ../reference/demo-app.md for the as-built shape); auth is altaforge.auth bearer tokens, not Odyssey SAML; and the framework capabilities live in R2 (../releases/r2-framework-production-features/r2-tdd.md — render modes [T2-render-modes], computed-fields [T2-computed], readiness gate [T2-readiness], per-doc model/temp [T2-llm-config], version history). This document is retained as the historical Odyssey production-solution design (sovereignty deployment, Odyssey-specific tables, SSO) for when that engagement is real — it is intentionally not rewritten to match the demo. What carried through verbatim: the deterministic approval-detection principle ([T-decision-1]) and the no-fabrication / eval intent, both embodied by the Phoenix computed layer + P4/P5 grounding.

Originally written by: TechLead (Mark Gleason). Reviewed by AltaML TechLead before the pod's architect agent picks it up.

How to read this¶

Audience	Read	Skip
Engineering reviewer	TL;DR + Tech Stack + Data Model + Key Decisions + Reviewer Checklist	Detailed acceptance tests on first pass
Architect agent (issue decomposer)	Everything	-
Dev agent (implementer)	Everything; especially Tech Stack, Data Model, Component Structure, current phase's Acceptance Tests	Phases not currently in flight
Sponsor / business owner signing off	TL;DR + Phase Decomposition + Reviewer Checklist	Implementation detail

Length budget: 5-15 pages.

Change History¶

Date	Author	Change
2026-05-20	Mark Gleason	Initial draft
2026-05-22	Mark Ly	UI stack: docgen-ui builds on altaforge-ui (Radix Themes); Tailwind removed
2026-05-22	Mark Ly	Framework package boundary update: `docgen-engine` + `docgen-tools` merged into single `docgen` package; `docgen-ui` moves to own repo (`altaml/altaforge-docgen-ui`) consumed as submodule
2026-05-23	Mark Ly	Simplified: docgen-ui stays co-located in `altaforge-docgen` (no separate repo); Odyssey submodules `altaml/altaforge-docgen` and consumes both Python and React from one fetch
2026-05-23	Mark Ly	Rescoped to new R2 in 4-release plan. Odyssey UAT now lands in R2 (was R1 in old 5-release plan). Engineering content below is unchanged — same FastAPI backend, same Next.js UI, same Phoenix + EFS adapters, same Arize Phoenix self-host deployment. Cross-refs to the framework PRD/TDD now resolve to the R2 framework docs.

TL;DR `[T-tldr]`¶

Next.js + FastAPI Odyssey solution app built on DocGen v0 libraries (docgen Python package + @altaml/docgen-ui React package, both consumed by submoduling altaml/altaforge-docgen and resolving via pnpm workspace during active co-dev); deployed inside Odyssey's cloud account.
Five DocGen Sections (one per firm-order-memo paragraph) wired to two Odyssey-specific data tools (phoenix_tool, efs_tool) and a prompt template per paragraph drawn from the prompt read-me.
The prompt read-me is a Markdown file in Postgres (versioned by organization_id, version_hash); the web UI exposes a Monaco-based editor; engine renders Jinja2 over it at generation time.
Approval-detection logic (the highest-risk area, PRD [P-rule-1/2/3]) lives in efs_tool as pure Python — fuzzy regex match on EFS comments + typed date extraction. Persisted as an ApprovalRecord row per generation for debugging.
Arize Phoenix colocated with the deployment; all evals (PRD P-eval-1 through P-eval-5) authored against the framework's eval adapter.
4 phases: adapter scaffolding + read-me v0 → approval-detection + evals → web UI + read-me editor → Phoenix deployment + UAT.

System Overview `[T-system]`¶

   Underwriter (Stamford)
   ┌────────────────────────────────────────────────┐
   │  Browser (Chrome/Safari/Firefox/Edge)          │
   └──────────────────────┬─────────────────────────┘
                          │  HTTPS
                          ▼
   Odyssey cloud (sovereignty zone)
   ┌────────────────────────────────────────────────┐
   │  Next.js UI (Odyssey-branded)                  │
   │   • imports @altaml/docgen-ui                  │
   │   • Monaco editor for prompt read-me           │
   │              │                                  │
   │              ▼ (server actions)                │
   │  FastAPI backend (Odyssey solution layer)      │
   │   • POST /generate/{pid}                       │
   │   • PATCH /readme                              │
   │   • POST /sections/{id}/regenerate, /edit      │
   │   • POST /documents/{id}/export                │
   │              │                                  │
   │              ▼ uses docgen                     │
   │  docgen (in-process)                           │
   │   • orchestrate(5 sections)                    │
   │   • LiteLLM gateway → Claude (Odyssey-routed)  │
   │   • Arize adapter → Phoenix                    │
   │              │            │                    │
   │              ▼            ▼                    │
   │  ┌──────────────┐  ┌──────────────┐            │
   │  │  Postgres    │  │  Arize       │            │
   │  │  (DocGen +   │  │  Phoenix     │            │
   │  │   Odyssey    │  │  (self-host) │            │
   │  │   tables)    │  └──────────────┘            │
   │  └──────────────┘                              │
   │              │                                  │
   │              ▼ (data tools)                    │
   │  ┌──────────────┐  ┌──────────────┐            │
   │  │ phoenix_tool │  │ efs_tool     │            │
   │  │ (Phoenix DB  │  │ (EFS DB/API; │            │
   │  │  or API; TBD)│  │  approval    │            │
   │  │              │  │  detection)  │            │
   │  └──────┬───────┘  └──────┬───────┘            │
   └─────────┼─────────────────┼────────────────────┘
             │                 │
             ▼                 ▼
        Phoenix systems    EFS systems (Odyssey-owned)

Everything in the dashed box runs in Odyssey's cloud account. No outbound traffic to AltaML cloud at runtime (per [P-sys] sovereignty).

Tech Stack `[T-stack]`¶

Layer	Choice	Rationale (only if non-default)
Language (backend)	Python 3.11	(default; matches framework)
Language (UI)	TypeScript 5	(default)
UI framework	Next.js 14 (App Router)	(default for AltaML web work)
UI components	`@altaml/docgen-ui` + `altaforge-ui` (Radix Themes) + local Odyssey-branded components	(default; framework UX components consumed directly)
Code editor (read-me)	Monaco Editor (`@monaco-editor/react`)	Battle-tested for in-browser Markdown editing; underwriters comfortable with code-editor-like surface
Backend framework	FastAPI	(default)
Engine integration	`docgen` (Python import)	Framework R1 library
Data tools	`docgen.tools` catalog + Odyssey-specific `phoenix_tool`, `efs_tool` (live in solution repo, implement `docgen.tools.DataTool` ABC)	Per framework P-feat-6: domain-specific tools start solution-local
Prompt templating	Jinja2 (engine-supplied)	(default)
Approval-comment fuzzy match	`rapidfuzz` (Levenshtein distance)	Tolerates typos per PRD `[P-rule-1]`; faster than `fuzzywuzzy`; pure Python
PDF date extraction	`pdfplumber` for text-layer; explicit reject (placeholder) for image-only PDFs	Per PRD `[P-rule-2]` — never guess from handwritten dates; pdfplumber surfaces text-layer cleanly
LLM access	`litellm` via framework gateway → Claude 4.5 Sonnet	(default; framework decision)
Temperature	0 (mandatory per `[P-sys]` determinism)	Spec §9.2
DB	Postgres 15 (bundled in deployment Compose for v0; `[T-q-1]`)	Per framework client-cloud recipe
ORM	SQLAlchemy 2.x (framework-supplied) + Odyssey-only tables via Alembic in solution repo	(default; matches framework)
Eval backend	Arize Phoenix (self-host, colocated)	Per framework `[T-decision-3]`; Odyssey is sovereignty-bound
Deployment	Docker Compose recipe from framework `deployments/client-cloud/`, extended with Odyssey solution containers	Framework P-feat-9; Odyssey is the proving case
Region	Odyssey's cloud region (Odyssey IT picks)	Sovereignty
Auth	Odyssey SSO (SAML; details from Odyssey IT)	Odyssey-owned identity provider; outside DocGen scope per framework `[T-auth]` (n/a in R1)
Test runner	pytest + pytest-asyncio (backend), vitest (UI)	(default)
Browser support	Chrome, Safari, Firefox, Edge — latest 2 versions	Per PRD `[P-sys]`

Data Model `[T-entity-N]`¶

Odyssey reuses DocGen's organizations (one row: odyssey), documents, sections, versions (R2). The Odyssey-specific tables sit alongside.

`[T-entity-1]` `firm_order_memos` ← references PRD `[P-entity-1]`¶

Column	Type	Nullable	Default	Notes
id	text (uuid)	No	`gen_random_uuid()`	PK
organization_id	text (uuid)	No	(always `odyssey`)	FK → `organizations.id`
document_id	text (uuid)	No	-	FK → `documents.id`; one Document per memo
deal_pid	text	No	-	Phoenix deal identifier
program_name	text	Yes	-	Optional contextual input (PRD §5.2)
treaty_year	int	Yes	-	Optional contextual input
underwriter_name	text	Yes	-	Optional contextual input
readme_version_hash	text	No	-	SHA-256 of the prompt-read-me text used at generation; allows reproducing the prompt that produced this output
generated_at	timestamptz	No	`now()`

Indexes: (organization_id, deal_pid) for lookups; (organization_id, generated_at DESC) for "recent memos" list.

`[T-entity-2]` `approval_records` ← references PRD `[P-entity-2]`¶

Column	Type	Nullable	Default	Notes
id	text (uuid)	No	`gen_random_uuid()`	PK
organization_id	text (uuid)	No	(always `odyssey`)	FK → `organizations.id`
memo_id	text (uuid)	No	-	FK → `firm_order_memos.id`, cascade delete
artifact_type	text	No	-	`email` / `pdf` / `none-found`
approver_initials	text	Yes	-	e.g. `JG`, `BQ`, `CO`; null when `none-found`
artifact_date	date	Yes	-	The actual approval date per `[P-rule-2]`; null = placeholder needed
efs_file_ref	text	Yes	-	Opaque EFS file identifier for cross-reference
match_confidence	numeric(3,2)	Yes	-	Fuzzy-match confidence 0.00-1.00; null on exact match
match_reasoning	text	Yes	-	Why we picked this artifact (e.g., "Most recent matching `{initials} approval`; comment exact match `BQ approval`")
resolved_at	timestamptz	No	`now()`

Indexes: (memo_id).

`[T-entity-3]` `readme_versions`¶

Column	Type	Nullable	Default	Notes
id	text (uuid)	No	`gen_random_uuid()`	PK
organization_id	text (uuid)	No	(always `odyssey`)	FK → `organizations.id`
version_hash	text	No	-	SHA-256 of `text`; unique per organization
text	text	No	-	Full Markdown read-me content
created_by	text	No	-	Email of underwriter who saved this version
created_at	timestamptz	No	`now()`
is_current	boolean	No	`false`	Exactly one row per organization has `true`; transactional swap on save

Indexes: unique (organization_id, version_hash); partial unique (organization_id) WHERE is_current.

API Contracts `[T-api-N]`¶

FastAPI endpoints; all authenticated via Odyssey SSO (session-cookie based).

`[T-api-1]` `POST /api/generate` ← references PRD `[P-feat-3]`, `[P-flow-1]`¶

Request:

{
  "deal_pid": "PID-12345",
  "program_name": "Program X",
  "treaty_year": 2026,
  "underwriter_name": "Matthew Nicolini"
}

Response 200:

{
  "memo_id": "uuid",
  "document_id": "uuid",
  "approval_record": { "artifact_type": "pdf", "approver_initials": "BQ", "artifact_date": "2026-03-07", "match_confidence": 1.0 },
  "sections": [
    { "id": "uuid", "ordinal": 1, "title": "Reconfirmation", "content": "...", "status": "generated" },
    ...
  ]
}

Errors: - 404: PID not found in Phoenix - 502: Phoenix or EFS unreachable - 500: unexpected — payload { "error": "internal", "memo_id": "uuid" } so the underwriter can reference

Notes: Synchronous; ≤ 30s p95 (PRD [P-sys] latency). Trace emitted to Phoenix via framework adapter.

`[T-api-2]` `POST /api/sections/{id}/regenerate` ← references PRD `[P-feat-7]`¶

Wraps docgen.regenerate_section. Returns updated section.

`[T-api-3]` `PATCH /api/sections/{id}` ← references PRD `[P-feat-7]`¶

Body: { "content": "..." }. Wraps docgen.edit_section.

`[T-api-4]` `GET /api/readme` / `PUT /api/readme` ← references PRD `[P-feat-8]`, `[P-flow-2]`¶

GET returns current read-me text + version_hash. PUT body { "text": "..." } creates a new readme_versions row, atomically flips is_current. Returns new hash.

`[T-api-5]` `POST /api/documents/{id}/export.docx` ← references PRD `[P-feat-9]`¶

Wraps docgen.export_docx. Returns the .docx as Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document.

Component / Module Structure `[T-files]`¶

odyssey-firm-order-memo/
  apps/
    backend/                              # FastAPI app
      pyproject.toml
      src/firm_order_memo/
        app.py                            # FastAPI entry; routes
        config.py                         # Env: Phoenix/EFS endpoints, secrets
        deps.py                           # FastAPI dependencies (DB session, organization)
        api/
          generate.py                     # POST /api/generate
          sections.py                     # regenerate, edit
          readme.py                       # GET/PUT readme
          export.py                       # POST .docx export
        tools/
          phoenix_tool.py                 # docgen DataTool: reads Phoenix screens
          efs_tool.py                     # docgen DataTool: reads EFS + approval detection
          approval/
            selector.py                   # [P-rule-1] artifact selection
            date_extractor.py             # [P-rule-2] date extraction
            phrase.py                     # [P-rule-3] phrase selection
        readme/
          renderer.py                     # Jinja2 over the read-me Markdown
          version_store.py                # readme_versions table operations
        evals/                            # [P-eval-1..5] definitions
          approval_source.py
          approval_date.py
          structural_conformance.py
          no_fabrication.py
          end_to_end.py
        db/
          models.py                       # FirmOrderMemo, ApprovalRecord, ReadmeVersion
          migrations/                     # Alembic
        observability/
          arize.py                        # Wraps framework adapter; sets eval_backend=phoenix
      tests/
        unit/
        integration/
          test_approval_detection.py      # T-test-2.1, 2.2
        e2e/
          fixtures/                       # Sample EFS comments, PDFs, emails
    frontend/                             # Next.js app
      package.json
      app/
        layout.tsx
        page.tsx                          # Deal input + generate
        memo/[id]/page.tsx                # Memo review/edit/regenerate/export
        readme/page.tsx                   # Monaco editor for read-me
      components/
        ParagraphCard.tsx                 # Wraps @altaml/docgen-ui SectionCard with Odyssey labels
        DealInput.tsx
        ApprovalBadge.tsx                 # Shows resolved approval source/date or placeholder
        ReadmeEditor.tsx                  # Monaco wrapper
      lib/
        api.ts                            # Typed client for /api/*
  deployments/
    odyssey-cloud/
      docker-compose.yml                  # Extends framework recipe with backend + frontend + tools
      .env.example
      README.md                           # Runbook for AltaML DevOps deploying into Odyssey cloud
  readme/
    odyssey-firm-order-memo-prompt-readme.md  # The initial read-me (Matthew co-authored)

Auth Model `[T-auth]`¶

Provider: Odyssey SSO (SAML 2.0); IdP details from Odyssey IT.
Session: server-side session in Postgres (cookie-based, HTTPS-only, SameSite=Lax). Sessions expire after 8 hours of inactivity.
Authorization: all routes require an authenticated underwriter; no fine-grained RBAC in v1 (every authenticated user can generate, edit, export, and edit the read-me). Read-me edits log created_by for accountability.
Protected routes: /api/* (all); UI routes redirect to SSO if unauthenticated.

Key Design Decisions `[T-decision-N]`¶

T-decision-1: Approval detection is pure Python in efs_tool, not an LLM call Decision: approval.selector, approval.date_extractor, approval.phrase are deterministic Python — fuzzy regex over EFS comments, pdfplumber for typed PDF dates, lookup table for the phrase. The LLM is only used for synthesis of the five paragraphs. Because: Per PRD [P-sys] no-fabrication contract — the approval source/date is the single most-watched correctness item ([P-metric-2] ≥ 95%). LLM-based detection would introduce nondeterminism the spec explicitly forbids. Deterministic code is easier to evaluate, audit, and fix. Cost: harder to handle truly novel approval-comment formats — mitigated by P-eval-1 catching drift.

T-decision-2: Prompt read-me in Postgres, not git Decision: The read-me lives as rows in readme_versions. Underwriters edit via Monaco; save creates a new row + flips is_current. Initial seed is checked into the repo (readme/odyssey-firm-order-memo-prompt-readme.md) for first deploy. Because: Per PRD [P-metric-3] — underwriters edit without engineering involvement; round-tripping through git would require GitHub access and merge conflict knowledge they shouldn't need. Postgres gives us auditable history, atomic save, and version_hash traceability into firm_order_memos.readme_version_hash. Cost: lose IDE diff/PR review — acceptable because the read-me changes are wording, not structural.

T-decision-3: Sync generation, no background queue Decision: POST /api/generate runs end-to-end inside the request. No Celery. Because: Per PRD [P-sys] latency: ≤ 30s per deal. Single-deal-per-run is the v1 interaction (PRD [P-scope] out-of-scope: batch). Spec §9.2 corroborates. Framework engine is sync (framework [T-decision-1]). Queue would only matter at batch scale, which is explicitly v1+.

T-decision-4: Persist ApprovalRecord per generation, not just compute on the fly Decision: Every POST /api/generate writes a row to approval_records with the selected artifact, fuzzy-match confidence, and reasoning string. Because: Per PRD [P-risk-1] — convention adherence is fragile in production; we need a debug surface to investigate "why did Paragraph 1 say X for deal Y." Persisting the chosen artifact + reasoning lets us re-investigate without re-querying EFS (which may have changed). Storage cost is negligible (~1 row per generation, small payload).

T-decision-5: PDF date extraction via pdfplumber; image-only PDFs return null (placeholder) Decision: pdfplumber extracts the text layer. If the PDF has no text layer (scan/image), date_extractor returns null and the UI shows {placeholder — underwriter to complete} per PRD [P-rule-2] worked example 2. Because: Per PRD [P-scope] out-of-scope item — OCR for handwritten/scanned PDFs is explicitly deferred. Per [P-eval-2] failure mode — fabricated dates are unacceptable. Returning null is the safe path. Cost: degrades perceived value when image-only PDFs are common ([P-risk-2]); we measure during UAT and decide if OCR investigation is warranted.

T-decision-6: All 5 evals live in the solution repo, not the framework Decision: evals/ directory in apps/backend/ defines the eval suite, datasets in tests/e2e/fixtures/. Run via docgen eval (framework CLI). Because: Per framework PRD — solution-specific evals live in the solution, not the framework. Keeps the Odyssey eval suite versioned alongside the prompts and tools that affect its outputs. Run in CI on every PR to the read-me or tool code.

Phase Decomposition `[T-phase-N]`¶

`[T-phase-1]` Adapter scaffolding + read-me v0 + happy-path generation¶

Issues: 1. Repo setup (monorepo: apps/backend, apps/frontend); pyproject/package.json; CI skeleton 2. FastAPI app with /api/generate returning a hardcoded 5-section response (no real data yet) 3. Odyssey-only tables migration (firm_order_memos, approval_records, readme_versions) 4. phoenix_tool against a Phoenix sandbox with stubbed data; efs_tool against an EFS sandbox with stubbed data 5. Read-me v0 markdown checked in; rendered via Jinja2; produces a memo for a fixture deal 6. Wire DocGen engine, 5 Sections, sync to Postgres

Acceptance tests:

T-test-1.1: Given a Phoenix sandbox with one fixture deal and an EFS sandbox with one fixture approval email, when POST /api/generate runs, then a firm_order_memos row is created with five sections filled and an approval_records row. Maps to PRD [P-feat-3], [P-flow-1].

T-test-1.2: Given the seeded read-me v0, when a memo is generated, then every section's content references the rules from the read-me (paragraph headings match; mandatory Paragraph 1 opening is present). Maps to [P-feat-5].

`[T-phase-2]` Approval-detection logic + evals P-eval-1, P-eval-2, P-eval-4¶

Issues: 1. approval.selector — fuzzy regex on EFS comments with rapidfuzz; pick most-recent matching artifact ([P-rule-1]) 2. approval.date_extractor — email send-date for emails; pdfplumber text-layer for PDFs; return null for image-only PDFs ([P-rule-2]) 3. approval.phrase — lookup table mapping (artifact_type, date?) to opening sentence ([P-rule-3]) 4. Wire into efs_tool; persist ApprovalRecord per generation 5. Eval datasets: ~30 EFS scenarios for P-eval-1; ~30 artifacts for P-eval-2; a fixture set for P-eval-4 6. Eval definitions in evals/ using framework adapter; run target: hosted-AX for dev runs, Phoenix once deployed

Acceptance tests:

T-test-2.1: Given an EFS deal with files [email "JG approval" 2026-03-05, pdf "BQ approval" 2026-03-08, email "initial review" 2026-03-10], when the selector runs, then the chosen artifact is the PDF with approver_initials="BQ". Maps to [P-rule-1] worked example.

T-test-2.2: Given a PDF with typed date 2026-03-07 near the signature, when the date extractor runs, then approval_date == "2026-03-07". Given an image-only PDF, when the date extractor runs, then approval_date IS NULL. Maps to [P-rule-2].

T-test-2.3: Given (artifact_type=pdf, date="2026-03-07"), when the phrase selector runs, then the opening sentence is exactly "Please reconfirm as agreed per our office meeting on 2026-03-07." and not "...per our meeting...". Maps to [P-rule-3] and spec §8.5.

T-test-2.4: Given the P-eval-1 dataset (30 EFS scenarios) run through docgen eval against the framework adapter, when the suite completes, then approval-source classification accuracy ≥ 95% on standard cases, with all none-found cases surfaced (not promoted to a guess). Maps to [P-eval-1].

T-test-2.5: Given the P-eval-4 fixture set (deals + their generated memos) run through the LLM-as-judge no-fabrication scorer, when the suite completes, then zero fabricated approvals/dates/negotiations are detected. Maps to [P-eval-4], spec §13.

`[T-phase-3]` Web UI + read-me editor¶

Issues: 1. Next.js app skeleton with Odyssey SSO integration 2. Deal-input page (/); on submit calls /api/generate, redirects to memo view 3. Memo view (/memo/[id]) — five ParagraphCards using @altaml/docgen-ui SectionCard; per-paragraph edit + regenerate + ApprovalBadge for Paragraph 1 4. Export button → POST /api/documents/{id}/export.docx; trigger browser download 5. Read-me page (/readme) — Monaco editor with Markdown syntax highlighting; Save calls PUT /api/readme; preview pane shows rendered Markdown 6. Smoke test against the phase-2 backend

Acceptance tests:

T-test-3.1: Given an underwriter logged in, when they enter a PID and click Generate, then within 30s the memo view shows five paragraphs with content (or clear failure states for sections whose tool failed). Maps to [P-flow-1], [P-sys] latency.

T-test-3.2: Given a generated memo, when the underwriter edits Paragraph 2's content inline and saves, then the section content updates and the section is marked edited in the UI. When they click regenerate on Paragraph 4, then Paragraph 4 re-runs and the new content replaces the old. Maps to [P-flow-3], [P-feat-7].

T-test-3.3: Given an underwriter on the /readme page, when they change a fixed phrase in Paragraph 2's rules and save, then a new readme_versions row is created with is_current=true and the previous version's is_current=false. When they then generate a new memo, then Paragraph 2 reflects the new phrase. Maps to [P-flow-2], [P-metric-3].

T-test-3.4: Given a generated memo on screen, when the underwriter clicks Export, then a .docx file downloads whose content is byte-equivalent to the on-screen text (modulo .docx-native rendering). Maps to [P-flow-3], [P-feat-9].

`[T-phase-4]` Phoenix deployment in Odyssey cloud + production-data evals (P-eval-3, P-eval-5) + UAT¶

Issues: 1. Extend framework deployments/client-cloud/docker-compose.yml with Odyssey backend + frontend + tools 2. Deploy into a representative sandbox in Odyssey's cloud account 3. Wire phoenix_tool and efs_tool to production Phoenix/EFS read paths (DB / API / scrape per [P-q-4]) 4. Validate Phoenix instance reachability from the engine; first eval run against Phoenix 5. Gather Matthew's gold-standard memos for the P-eval-5 dataset (~10-20 real deals) 6. UAT with Matthew per spec §12.1 cadence (Thursdays for 2 weeks) 7. Tighten read-me based on UAT findings

Acceptance tests:

T-test-4.1: Given the deployment running in the Odyssey sandbox, when POST /api/generate runs against a real PID, then the response returns within 30s with a five-paragraph memo and a Phoenix trace exists for the generation. Maps to [P-flow-1], [P-sys], framework [T-test-4.1].

T-test-4.2: Given the deployment, when network traffic is captured during a generation, then zero outbound traffic to Arize AX domains (*.arize.com) or to AltaML cloud is observed; all eval/trace traffic stays inside Odyssey's cloud. Maps to [P-sys] sovereignty, framework [T-test-4.2].

T-test-4.3: Given 50 representative deals generated through the system, when P-eval-3 (structural conformance) runs, then 100% of memos have all 5 paragraphs present and in order. Maps to [P-eval-3].

T-test-4.4: Given Matthew's gold-standard set for 10-20 real deals, when P-eval-5 (end-to-end acceptance) runs via LLM-as-judge + manual review, then the majority of generated memos are judged "acceptable with minor edits." Maps to [P-eval-5], [P-metric-1].

Deployment `[T-deploy]`¶

Owned by AltaML DevOps + Odyssey IT collaboratively.

Compose recipe: deployments/odyssey-cloud/docker-compose.yml extends the framework client-cloud recipe with Odyssey backend + frontend containers.
Environment: runs entirely inside Odyssey's cloud account (sovereignty); Odyssey IT provisions the VM/AKS-equivalent host; AltaML DevOps deploys and operates.
Secrets: Odyssey's secret store (e.g., Azure Key Vault inside Odyssey's organization); env-mounted into the containers at start.
Phoenix: colocated container (arize-phoenix:latest); persistent volume for eval data.
Postgres: bundled in Compose for v0 ([T-q-1]); migration path to managed Postgres if Odyssey IT prefers.
Migrations: alembic upgrade head on backend container start (both framework migrations and Odyssey-specific migrations).

Env vars (passed by Odyssey IT / AltaML DevOps): - LITELLM_BASE_URL, LITELLM_API_KEY (AltaML LiteLLM endpoint accessible from Odyssey cloud — coordinate networking) - DATABASE_URL - EVAL_BACKEND=phoenix, EVAL_BACKEND_URL=http://phoenix:6006 - PHOENIX_DB_DSN / PHOENIX_API_URL (per [P-q-4] once access mechanism is locked) - EFS_DB_DSN / EFS_API_URL - ODYSSEY_SSO_METADATA_URL, ODYSSEY_SSO_CERT

Open Questions `[T-q-N]`¶

T-q-1: Bundle Postgres in Compose vs. require Odyssey-managed Postgres? Default: bundle for v0 to minimize Odyssey IT setup; revisit if Odyssey requires managed DB for backup reasons.
T-q-2: LiteLLM endpoint accessibility from Odyssey cloud — does Odyssey allow outbound HTTPS to AltaML's LiteLLM, or do we need a private endpoint / VPN peering? Coordinate with Odyssey IT before phase 4.
T-q-3: EFS PDF access pattern — does EFS return raw PDF bytes via API, or do we have to download files via SFTP/blob store? Drives pdfplumber invocation pattern.
T-q-4: Audit-log retention period — resolving PRD [P-q-2]. Spec §9.2 + §10 frame this as operational logs for review and regression testing, not formal regulatory audit logs (no retention requirements, tamper-evidence, or compliance framework cited in the spec). Default: retain firm_order_memos, approval_records, and Phoenix trace records for 90 days rolling window. That's enough for: (a) UAT-era regression testing against gold-standard outputs, (b) debugging "why did Paragraph 1 say X" within a quarter of the run, (c) bounded storage cost on the deployment Postgres. Revisit if Odyssey surfaces a regulatory retention requirement or if Matthew wants longer-term comparison across releases.

(Note: PRD [P-q-1] (input modality) resolved — web UI only for R1. [P-q-2] (audit-log retention) resolved here at T-q-4. [P-q-4] (Phoenix/EFS access mechanism) is the same as T-q-2 and T-q-3 — bubbles up to engineering coordination with Odyssey IT.)

Sign-off¶

Role	Name	Date
Author	Mark Gleason	2026-05-20
Engineering reviewer (AltaML TechLead)
Business owner	Matthew Nicolini
Approver	Mark Ly (AltaML CTO)

Reviewer's checklist¶

[x] TL;DR fits in ≤6 bullets and conveys the engineering shape
[x] Every PRD [P-feat-N] has a TDD home — see cross-ref below
[x] Every PRD [P-metric-N] has a [T-test-N.M] — P-metric-1 (acceptance) → T-test-4.4; P-metric-2 (Paragraph 1 correctness) → T-test-2.4 + T-test-2.5; P-metric-3 (self-service) → T-test-3.3
[x] Every [T-test-N.M] annotates its parent inline
[x] Tech Stack table is complete; non-default choices (Monaco, rapidfuzz, pdfplumber) have a because
[x] Every [T-decision-N] has a because
[x] Every phase has at least one Given/When/Then acceptance test
[x] Sections that don't apply are marked "n/a" — none (Odyssey solution has auth)
[x] Open questions are truly unresolved
[x] Cross-ref IDs (T-*) assigned

Cross-ref check (PRD [P-feat-N] → TDD home): - P-feat-1 (Phoenix adapter) → apps/backend/.../phoenix_tool.py, T-test-1.1 - P-feat-2 (EFS adapter) → apps/backend/.../efs_tool.py, approval/*, T-test-2.1 through T-test-2.3 - P-feat-3 (5-paragraph orchestration) → T-api-1, T-test-1.1 - P-feat-4 (Paragraph 1 phrase selector) → approval/phrase.py, T-test-2.3 - P-feat-5 (read-me v0) → readme/odyssey-firm-order-memo-prompt-readme.md + readme/renderer.py, T-test-1.2 - P-feat-6 (deal input + generate UI) → apps/frontend/app/page.tsx, T-test-3.1 - P-feat-7 (memo review UI) → apps/frontend/app/memo/[id]/page.tsx + ParagraphCard, T-test-3.2 - P-feat-8 (read-me editor) → apps/frontend/app/readme/page.tsx + ReadmeEditor, T-test-3.3 - P-feat-9 (.docx export) → T-api-5, T-test-3.4 - P-feat-10 (eval suite) → apps/backend/.../evals/, T-test-2.4, T-test-2.5, T-test-4.3, T-test-4.4

When approved, the architect agent decomposes each [T-phase-N] into atomic ≤30-min issues; each issue inherits its parent [T-test-N.M] as the assertion to verify.

TDD - Odyssey Phoenix Firm Order Memo Generator — UAT (R3)¶

How to read this¶

Change History¶

TL;DR [T-tldr]¶

System Overview [T-system]¶

Tech Stack [T-stack]¶

Data Model [T-entity-N]¶

[T-entity-1] firm_order_memos ← references PRD [P-entity-1]¶

[T-entity-2] approval_records ← references PRD [P-entity-2]¶

[T-entity-3] readme_versions¶

API Contracts [T-api-N]¶

[T-api-1] POST /api/generate ← references PRD [P-feat-3], [P-flow-1]¶

[T-api-2] POST /api/sections/{id}/regenerate ← references PRD [P-feat-7]¶

[T-api-3] PATCH /api/sections/{id} ← references PRD [P-feat-7]¶

[T-api-4] GET /api/readme / PUT /api/readme ← references PRD [P-feat-8], [P-flow-2]¶

[T-api-5] POST /api/documents/{id}/export.docx ← references PRD [P-feat-9]¶

Component / Module Structure [T-files]¶

Auth Model [T-auth]¶

Key Design Decisions [T-decision-N]¶

Phase Decomposition [T-phase-N]¶

[T-phase-1] Adapter scaffolding + read-me v0 + happy-path generation¶

[T-phase-2] Approval-detection logic + evals P-eval-1, P-eval-2, P-eval-4¶

[T-phase-3] Web UI + read-me editor¶

[T-phase-4] Phoenix deployment in Odyssey cloud + production-data evals (P-eval-3, P-eval-5) + UAT¶

Deployment [T-deploy]¶

Open Questions [T-q-N]¶

Sign-off¶

Reviewer's checklist¶

TL;DR `[T-tldr]`¶

System Overview `[T-system]`¶

Tech Stack `[T-stack]`¶

Data Model `[T-entity-N]`¶

`[T-entity-1]` `firm_order_memos` ← references PRD `[P-entity-1]`¶

`[T-entity-2]` `approval_records` ← references PRD `[P-entity-2]`¶

`[T-entity-3]` `readme_versions`¶

API Contracts `[T-api-N]`¶

`[T-api-1]` `POST /api/generate` ← references PRD `[P-feat-3]`, `[P-flow-1]`¶

`[T-api-2]` `POST /api/sections/{id}/regenerate` ← references PRD `[P-feat-7]`¶

`[T-api-3]` `PATCH /api/sections/{id}` ← references PRD `[P-feat-7]`¶

`[T-api-4]` `GET /api/readme` / `PUT /api/readme` ← references PRD `[P-feat-8]`, `[P-flow-2]`¶

`[T-api-5]` `POST /api/documents/{id}/export.docx` ← references PRD `[P-feat-9]`¶

Component / Module Structure `[T-files]`¶

Auth Model `[T-auth]`¶

Key Design Decisions `[T-decision-N]`¶

Phase Decomposition `[T-phase-N]`¶

`[T-phase-1]` Adapter scaffolding + read-me v0 + happy-path generation¶

`[T-phase-2]` Approval-detection logic + evals P-eval-1, P-eval-2, P-eval-4¶

`[T-phase-3]` Web UI + read-me editor¶

`[T-phase-4]` Phoenix deployment in Odyssey cloud + production-data evals (P-eval-3, P-eval-5) + UAT¶

Deployment `[T-deploy]`¶

Open Questions `[T-q-N]`¶