Skip to content

Phoenix UC2 — firm-order memo: requirements + framework gap analysis

status: analysis
date: 2026-06-06
source: "Google Drive — Phoenix Memo Generator Sample Data (Badri Venkataraman / AltaML × Odyssey Re)"
source_folder: https://drive.google.com/drive/folders/1IUAZlcGV0yLRFVGrrfwr8epPVa4gm7ZW
relates_to: docs/reference/Phoenix_Firm_Order_Memo_Spec.docx ; epic [E-release-3] Odyssey UAT ; epic [E-release-2] framework features

Incorporates the Phoenix sample-data folder (UC2 of the AltaML × Odyssey Re engagement). It is engineering-grade sample data + contract for the firm-order memo pipeline. This doc records what the use-case needs, maps it onto DocGen, and flags the three framework features we're missing so we build them deliberately rather than discover them in R3.

All source documents (spec, sample data, generators, mockup) are catalogued in ../reference/README.md — the full input set behind the Phoenix work.

1. What Phoenix UC2 is

Generate a 5-paragraph reinsurance firm-order memo for a deal (Phoenix Program ID / "PID"), for two deal types: CAT XOL and Per Risk. Three layers:

  1. Ingestion — pull a deal's phoenix block (program summary, per-layer detail, cat modeling, pricing, authorization) + efs block (approval artifact, referral commentary, underlying-book narrative) for a PID.
  2. Assembly (P1, P2, P3) — deterministic: template + data + computed values, no LLM.
  3. LLM generation (P4, P5) — temperature-0 model call, faithful to EFS sources, no fabrication, conservative fallback when sources are sparse.

The 7-deal dataset (deals.json) is the input contract (the shape ingestion must produce); expected-memos.json is the regression target for assembly + LLM. Both are provisional until Matthew Nicolini (Odyssey) signs off.

The five paragraphs

Content How produced
P1 reconfirm/approval line — approval date + approver + per-layer authorized shares deterministic (from approval artifact + layer authShare)
P2 outcome totals — signed limit / premium across layers deterministic (sum over layers)
P3 per-layer structure + metrics (CAT: ROL, MLR, MSD, RARC; Per Risk: rate, modeled/expiring composite) + total deterministic, iterates over N layers
P4 rationale / pricing position LLM (temp 0) from EFS referralCommentary
P5 underlying-book narrative (exposure, losses) LLM (temp 0) from EFS underlyingBook

Formulas (computed during assembly — Phoenix does NOT export these)

DevPremium = SubjBasePML × Rate / 100
ROL        = DevPremium / Limit × 100
MLR        = OccEV / DevPremium × 100     ← computed; NOT a Phoenix field
OdyPremium = DevPremium × SignShare / 100
OdyLimit   = Limit × SignShare / 100

Insufficient-data gating (pre-generation)

Generation is blocked with a structured gap (reason + ui_state) — not generated — when approval can't be established. Edge cases in the dataset:

Deal Gate
619055 no approval comment at all → block
620998 comments exist but none match strict "{initials} approval" → block (don't best-guess)
621557 PDF approval, handwritten date, OCR fails → block, require manual date (no substituting upload date)
618015 multiple approvals → pick the most recent matching by artifactDate (not a block — a selection rule)

2. Gap analysis — Phoenix need vs DocGen

Phoenix need DocGen feature Status
Ingestion of phoenix/efs blocks per PID gather phase + DataTool connectors ✅ pattern exists; Phoenix/EFS connectors are solution code (R3)
Deterministic P1/P2/P3 (template + data) templated render — render_mode='template' over gathered.deal (+ Snippet composition for JBRK) ✅ implemented — Phoenix P1–P3 match gold via the engine (fixtures/phoenix.py, test_phoenix_seeded.py)
LLM P4/P5, temp 0, faithful, fallback custom_render prompt sections (R1) ✅ exists; fallback = prompt rules
Per-layer iteration in P3 (loop over N layers) Jinja {% for %} over a list-valued render variable ✅ GAP-1 met — gathered.deal.layers iterated in the P3 template
Computed fields (MLR, ROL, DevPremium, …) engine/computed.py hook — solution registers pure fns → computed.* ✅ GAP-2 implemented (#116). Connectors stay raw; Phoenix math lives in fixtures/phoenix_compute.py, not the connector
Pre-generation readiness gate (block + structured gap) engine/readiness.py + connector _readiness + pipeline gate ✅ GAP-3 implemented (#117)
Manual-completion loop (type the missing value in, re-run) satisfiable_by_field + check_readiness re-arm ✅ GAP-4 implemented (#126, 621557)
Approval extraction (pattern, initials registry, date OCR, most-recent) gather/tool logic 🟡 solution code (R3); framework provides plumbing
Status ready/attention/draft + action-required banner Document.state (draft/under-review/finalized) 🟡 partial — "attention" is GAP-3's output surfaced as state

3. The framework gaps — what we're doing

These are R2 framework features (not Odyssey-specific) — Phoenix is the forcing function, but JBRK needs the same primitives, so they belong in the framework, built before R3. (GAP-1/2/3 surfaced in the first pass; GAP-4 surfaced when the three solution PRD/TDDs were written — see docs/solutions/.)

GAP-1 — Structured (list/dict) render data + iteration

Today Fields are scalar key→value strings and the render context is flat. P3 must loop over a list of layer objects ({% for layer in layers %}…ROL of {{ layer.rol }}%…{% endfor %}). Plan: allow structured (JSON list/dict) values in the render context — either list-valued Fields or, more naturally, the gathered-data rows (already JSON) exposed as iterable objects in the Jinja scope, and confirm the sandboxed env permits {% for %}/attribute access. A Snippet/Text template can then render a repeated per-layer block. (This is also what JBRK's atom bullets[] + {% if is_post_mortem %} imply.)

GAP-2 — Computed / derived fields

✅ Implemented — engine/computed.py (register_computed / run_computed / compute_for_document) merges solution pure-fn outputs into the computed.* render namespace. Connectors stay dumb (raw data only); Phoenix's math is in fixtures/phoenix_compute.py. See r2-tdd [T2-computed].

MLR, ROL, DevPremium etc. are computed from gathered data, not stored. DocGen has no derived-value concept; JBRK solves the same with solution-provided tax_math pure functions feeding the variable dict. Plan: a computed-fields hook — solution code registers pure functions (compute(deal) -> {mlr, rol, …}) whose outputs merge into the render scope (precedence below atoms/fields/variables, or a dedicated computed.* namespace). Framework owns the plumbing + the merge; solutions own the formulas. Deterministic + unit-testable.

GAP-3 — Pre-generation readiness gate (preconditions)

✅ Implemented — engine/readiness.py (evaluate_readiness / document_readiness) + the connector-driven pipeline gate (readiness_from_gathered, run in _run_render_pipeline after gather, before render) + the check_readiness bridge command. See r2-tdd [T2-readiness].

The verify phase is post-hoc (checks citations after synthesis). Phoenix needs a gate before generation: evaluate required inputs (a matched approval with a resolved date); if unmet, do not generate — return a structured { generation_should_proceed: false, reason, ui_state } and set the document to an action-required state. Plan: a precondition / readiness check on the document (declarative required-inputs + a resolver), run ahead of render; on failure it short-circuits with the structured gap the UI renders as the "attention" banner + manual-entry prompt. Distinct from verify (which stays post-hoc).

GAP-4 — Manual-completion loop (gap → writable field → re-arm)

✅ Implemented — each unmet Precondition carries a satisfiable_by_field target; the connector treats a supplied approval_date Field as the resolved date and re-arms. The demo loop: check_readiness (blocked) → upsert_field → check_readiness (proceeds). See test_phoenix_readiness.py.

Surfaced by edge case 621557 (handwritten PDF date, OCR fails) while writing the Phoenix TDD. GAP-3 as scoped is read-only — it blocks and describes the gap. But the use-case requires the underwriter to supply the missing value and re-run: the gap must name a writable target Field, the UI collects it, and supplying it re-arms the readiness check so generation proceeds. No existing or planned primitive covers this loop, and it's generic (any human-satisfiable gate needs it). Plan: extend the GAP-3 readiness result so each unmet precondition carries a satisfiable_by_field target (key + type + prompt); the UI writes that Field and re-runs the gate. Fold into G3's R2 design or track as its own slice. (Reference Check confirms G3/G4 are not universal — its report has no readiness gate at all.)

4. Release mapping

  • R2 (framework): GAP-1, GAP-2, GAP-3 — generic primitives. Built after the composition epic (#87), before R3.
  • R3 (Odyssey UAT): the Phoenix solution — Phoenix/EFS ingestion connectors, the approval-extraction rules (initials registry, pattern match, date OCR + most-recent selection), and the deal-shaped fixtures.
  • Regression target: vendor deals.json (7 deals) + expected-memos.json into docs/reference/phoenix-data/ and drive assembly/LLM tests off them. Provisional until Matthew's sign-off.

5. Open questions (carried from the dataset — resolve with Matthew/Mark before R3 build)

  1. Per Risk actuarial composite — exact location in the Firm Order analysis doc.
  2. Approver-initials registry — canonical list beyond JG/BQ/CO.
  3. EFS comment format — strict "{initials} approval" only, or variants.
  4. Paragraph 2 vs 3 boundary (totals vs per-layer).
  5. Paragraph 5 source priority (EFS narrative vs FOT email vs exhibits).

6. Disclosures (from the source)

Only 2 of 7 deals are GROUNDED (614563 Sutton/Bamboo, 618702 Heritage Per Risk); the rest are DERIVED/EDGE-CASE. EFS narrative is AltaML-written in Matthew's style, not his voice. expected-memos.json is AltaML's interpretation, provisional until Matthew reviews. Every deal carries _numbersOrigin.

7. Built & verified against the authoritative files

The sample folder's generators were read in full and vendored to docs/reference/phoenix-data/ (build_deals.py, build_expected.py — the authoritative formula + assembly source; deals.json, expected-memos.json — regenerated and byte-verified against the real downloaded artifacts). A runnable Phoenix memo generator (phoenix_memo.py) + regression test (docgen/tests/integration/test_phoenix_memo.py) produce the 5-paragraph memo for all 7 deals and match the gold exactly (4 sufficient + 3 blocked; readiness gating, most-recent-approval selection, GAP-4 satisfiable-by-field, Per Risk composite). 13 tests pass.

Reading the primary files (not just the README/samples) corrected/confirmed specifics the first-pass analysis missed: - Formulas confirmed exact: DevPremium=SubjBasePML×Rate/100, ROL=DevPremium/Limit×100, MLR=OccEV/DevPremium×100 (computed), OdyLimit/OdyPremium via SignShare. - P1 also consumes the first sentence of referralCommentary (deterministic slice) + opener varies email ("via email on…") vs PDF ("per our office meeting on…"). - Declined layers: authShare == 0 → the layer renders "Declined" and is excluded from the authorization totals. - Field provenance: MSD←catModeling.occMSD, RARC←pricing.rarc, MLR←catModeling.occEV ÷ layer.devPremium. - P4/P5 in the dataset are hand-authored reviewed gold (the P4_P5 dict), the temp-0 LLM-layer regression target — not generated.