Org → spec seeding & the per-org platform tool set¶

Every seeded example you see in the demo — the Phoenix firm-order memo, the GOA background check, the DealDesk proposal + SOW, the Carbon Media proposal — is compiled from a version-controlled spec JSON into the runtime DB at first use — Postgres (the docgen database, via DATABASE_URL) in normal operation, or the SQLite demo.db fallback when DATABASE_URL is unset. Either way the DB is a rebuildable cache; the JSONs are the source of truth (and, now, the /v1 config API — see authoring-and-config-api.md). This page documents that seeding mechanism and the platform (generic) tool set that every org gets automatically.

The model in one picture¶

authoring/specs/<name>.json        ← source of truth (version-controlled)
  │   (one file = a template + its sections/fields/tools/data_needs + example instances)
  ▼
fixtures/orgs.py :: ORG_SPECS       ← which spec files each org gets
  │
  ▼  bridge :: _demo_session() (first call per DB) → ensure_all_orgs(engine)
  │     for each org:
  │       _ensure_org_row(slug, display_name)            ← Organization row
  │       for spec_file:  seed_from_spec(engine, org, file)
  │         ├─ register_platform_tools(session, org.id)  ← generic tools FIRST
  │         ├─ load_spec(path) → SchemaSpec (Pydantic-validated)
  │         └─ compile_schema(session, org.id, spec)     ← texts/snippets/tools/
  │                                                          fields/sections/data_needs
  │                                                          + example instances
  │       spec.seed(engine, org)  (optional Python seed — JBRK only)
  ▼
demo.db   ← template Documents + instantiated instances (a CACHE; delete to rebuild)

demo.db holds no source-of-truth content — it is materialized from the specs. Delete it (or reset) and the next bridge call rebuilds it identically.

The pieces¶

Layer	Where	Role
Spec JSON	`docgen/src/docgen/authoring/specs/*.json`	The version-controlled source of truth: one file declares a template document, its sections / fields / tools / `data_needs`, and example instances.
`ORG_SPECS`	`docgen/src/docgen/fixtures/orgs.py`	The list of demo orgs and which spec files each one gets (`OrgSpec.spec_files`).
`seed_from_spec`	`docgen/src/docgen/authoring/seeding.py`	Compiles one spec file into an org and returns its template `Document`.
`compile_schema` / `load_spec`	`docgen/src/docgen/authoring/schema_compiler.py`	`load_spec` parses + Pydantic-validates the JSON; `compile_schema` writes the rows.
`register_platform_tools`	`docgen/src/docgen/tools/platform.py`	Ensures the generic tool set exists per org before the compiler validates `data_need.tool_ref`s.
`cmd_seed` / `_demo_session`	`docgen/src/docgen/bridge.py`	The entry point: seeds every org once per DB, then scopes a session to the active org.

Two seed paths, one preferred¶

OrgSpec supports two complementary mechanisms; specs compile first, then the optional Python seed runs:

Declarative specs (spec_files) — the preferred path. The template, its sections/fields/tools, its snippet bindings (template.snippet_bindings), and its example instances all live in one JSON file. GOA, Odyssey, AltaML (DealDesk), Vertical Scope, SITE, and JBRK (tomorrow-law, tomorrow-law-reporting-letter.json + tomorrow-law-inter-vivos.json) are declarative.
Python seed (seed, optional) — a Callable[[Engine, Organization], None] for the steps that MUST be code. JBRK's _seed_tomorrow_law keeps only its tax-math compute-hook registration and the post-compile approval of the lawyer-reviewed Risk Blocks (the compiler always lands blocks draft — specs cannot self-approve).

A fully declarative org leaves seed as None.

Idempotence & the seed guard¶

Seeding is idempotent end-to-end:

The compiler skips-with-warning keys that already exist; instances are only created when no document carries their title.
seed_from_spec reuses the template by (organization_id, title, is_template) rather than re-creating it.
The bridge tracks which DB paths it has already seeded this process in _seeded_dbs (a lock-guarded set), so the 4-org seed runs once per DB per process rather than on every bridge call. The guard is an optimization, not correctness — re-seeding would be safe, just wasteful. It resets on process restart.

Where the DB lives & how to rebuild it¶

Concern	Value
DB	`DATABASE_URL` (Postgres — e.g. the `docgen` database) when set; else SQLite at `DOCGEN_DEMO_DB` (default `./demo.db`). Resolved in `docgen/src/docgen/data/db_url.py` (bare `postgresql://` is pinned to `+psycopg`).
Active org	`_current_org` contextvar (bound per HTTP request from `X-Org-Slug`); CLI/tests fall back to `DOCGEN_ORG_SLUG`, default `odyssey`.
Rebuild (in-process)	`cmd_reset_demo()` — drops every table, recreates the schema, clears the seed guard, re-seeds. Wired to the dev toolbar's "Reseed demo".
Rebuild (wipe the file)	Delete `demo.db` (+ `-wal` / `-shm`). The next bridge call reseeds. The Phoenix eval harness does this with `python -m docgen.scripts.demo_phoenix_eval --fresh` (`_wipe_db()`), so a run reseeds brand-new docs.

After a schema change: - SQLite (dev): delete demo.db or run cmd_reset_demo() — Base.metadata.create_all adds new tables but never retrofits new columns onto an existing table, so a stale DB is the classic "column doesn't exist" failure. - Postgres: run alembic upgrade head (the migrations are additive/nullable, safe on existing rows), or cmd_reset_demo() to rebuild a dev DB from scratch. The dev PG created via reset_demo is create_all-built (no alembic_version); a real deployment should be alembic-managed.

How to add a new template¶

Adding a declarative walkthrough is two edits, no new Python module:

Drop the spec at docgen/src/docgen/authoring/specs/<name>.json. It declares (at minimum) a template, its sections, and any fields / tools / data_needs / example instances. (See an existing spec, e.g. goa-background-check.json or altaml-dealdesk-proposal.json, for the shape; load_spec validates it against the SchemaSpec Pydantic model.)
List it on the org in docgen/src/docgen/fixtures/orgs.py — add the filename to that org's OrgSpec.spec_files:

OrgSpec(
    'my-org',
    'My Org Display Name',
    spec_files=('my-new-template.json',),
),

(Add a whole new OrgSpec(...) entry to ORG_SPECS if the org doesn't exist yet — _ensure_org_row creates the Organization row on first seed.)

Delete demo.db (or hit "Reseed demo") and the new template + its instances appear. Do not declare generic tools (add, claude_web_search, web_page, file_ingest) in the spec's tools[] — they are platform tools, auto-registered (see below). Only custom data-source connectors (the phoenix / hubspot / bigquery MCP sources) belong in tools[].

The platform (generic) tool set¶

Shipped in commit 41e3d75 — "platform tool set auto-registered per org; stop declaring generics in specs."

Generic utilities — web search, page scraping, file ingest, arithmetic — are platform capabilities, not template-specific tools. Before this change every spec had to re-declare them in its tools[]; now they are registered once per org automatically and specs only declare their own data-source connectors.

What's in the set¶

PLATFORM_TOOLS in docgen/src/docgen/tools/platform.py is the canonical list:

Tool name	Category	Class
`add`	`math`	`docgen.tools.add.AddTool`
`claude_web_search`	`web_search`	`docgen.tools.claude_web_search.ClaudeWebSearchTool`
`openai_web_search`	`web_search`	`docgen.tools.openai_web_search.OpenAIWebSearchTool`
`web_page`	`page_scrape`	`docgen.tools.web_page.WebPageTool`
`file_ingest`	`file_ingest`	`docgen.tools.file_ingest.FileIngestTool`

Custom data-source connectors (the phoenix / hubspot / gdrive / fellow / bigquery MCP sources) are not here — those still go in a spec's tools[].

How it's attached¶

register_platform_tools(session, organization_id) ensures each platform tool exists as a DB Tool row for the org, skipping any already present (idempotent). seed_from_spec calls it inside the org-scoped session, before compile_schema:

with OrganizationScopedSession(org.id, engine) as session:
    # Platform (generic) tools must exist BEFORE the compiler validates the
    # spec's data_need.tool_ref's against {DB tools} | {spec.tools}.
    register_platform_tools(session, org.id)
    compile_schema(session, org.id, spec)

Ordering is load-bearing. The compiler resolves each data_need.tool_ref against {DB tools} | {spec.tools}. A tool_ref pointing at a platform tool (e.g. claude_web_search) that wasn't registered yet would fail validation — so registration must run first. Writes use create_tool(commit=False) so they compose into the caller's transaction.

Reference¶

Seeding: docgen/src/docgen/authoring/seeding.py, docgen/src/docgen/fixtures/orgs.py (OrgSpec, ORG_SPECS), docgen/src/docgen/authoring/schema_compiler.py (load_spec, compile_schema), docgen/src/docgen/bridge.py (cmd_seed, cmd_reset_demo, _demo_session, _seeded_dbs, DOCGEN_DEMO_DB).
Platform tools: docgen/src/docgen/tools/platform.py (PLATFORM_TOOLS, register_platform_tools); commit 41e3d75.
Specs: docgen/src/docgen/authoring/specs/*.json.