v5.0 Introduce centralized homepage & portrait photo management

Adds .github/workflows/quality-gate.yml, running the gate on push and pull
request. CI invokes `pre-commit run --all-files` rather than restating the
`ruff check` and `ty check` commands, so the checks keep a single definition
in .pre-commit-config.yaml and local and CI cannot drift (plan task 2).

The workflow writes a .env file rather than exporting an environment
variable. The two are not equivalent here: Settings reads the .env file,
while the external-test skip guard reads os.getenv, so an exported variable
un-skips the external tests and sends them to the network. Measured in CI:
no .env gave 115 failures and 18 errors, an exported dummy key gave 3
failures, and a written .env file reproduced the local baseline exactly.

Negative-tested on a scratch branch: a deliberate lint error failed the run
at `ruff check` with exactly the planted errors, confirming the gate blocks
rather than merely reporting (plan task 4). The subsequent clean run passed
ruff and ty and reported 295 passed, 4 skipped, matching local and
confirming the four credential-gated tests skip cleanly (plan task 3).

That first green run caught a real platform-dependent defect. PromptStore
rejected non-direct-child names via `Path(name).name != name`, which is
platform-dependent: on POSIX a backslash is an ordinary filename character,
so "nested\prompt.md" passed the guard and failed later as NOT_FOUND rather
than VALIDATION. Windows cannot reproduce it. No traversal was possible,
since the path.parent != root check still held, so the impact was a wrong
error category and a red gate. Both separators are now rejected explicitly,
matching the ^[^/\\]+$ pattern config.PromptFilename already used.

Also exports docs/ver4.7/review_log_v4_7.md, the working record kept across
all six phases: 50 entries, 1 still open. The open entry is a pre-existing
/ui redirect defect found during the Phase 3 UI walk and deliberately left
unfixed as outside the V4.7 scope boundary.

Co-authored-by: Copilot App <[email protected]>

.env.example

@@ -1,8 +1,62 @@
-PROVIDER=openrouter
-OPENROUTER_API_KEY=sk-or-...
-# PROVIDER_MODEL=          # optional: OpenRouter adapter supplies default
-# OPENROUTER_HTTP_REFERER=https://example.com
-# OPENROUTER_APP_TITLE=Historical Transcription MVP
-# DATABASE_URL=sqlite:///./transcription.db
-# UPLOAD_DIR=./uploads
-# PROMPT_DIR=./prompts
+# Canonical settings mirror for src/transcription/config.py (Settings).
+# Any value here overrides the in-code default.
+
+# --- NiceGUI Server ---
+HOST=0.0.0.0
+PORT=8000
+# LOG_LEVEL: critical | error | warning | info | debug | trace
+LOG_LEVEL=info
+RELOAD=false
+LOG_DIR=./data/logs
+LOG_FILE_NAME=transcription.log
+LOG_FILE_MAX_BYTES=10485760
+LOG_FILE_BACKUP_COUNT=5
+
+# --- AI provider ---
+# PROVIDER: openrouter
+PROVIDER=openrouter
+# Required.
+OPENROUTER_API_KEY=your-api-key-goes-here
+PROVIDER_MODEL=google/gemini-2.5-flash
+# PROVIDER_MODELS default: derived from PROVIDER_MODEL when omitted.
+# If provided, use a non-empty JSON array.
+# PROVIDER_MODELS=["google/gemini-2.5-flash","anthropic/claude-sonnet-4"]
+# OPENROUTER_HTTP_REFERER=
+# OPENROUTER_APP_TITLE=
+DEFAULT_PROMPT_NAME=transcribe_document.md
+# TRANSCRIPTION_TEMPERATURE default: unset (optional range 0.0..2.0)
+# TRANSCRIPTION_TEMPERATURE=
+# TRANSCRIPTION_TOP_P default: unset (optional range 0.0..1.0)
+# TRANSCRIPTION_TOP_P=
+
+# --- runtime environment ---
+# ENVIRONMENT: development | test | production
+ENVIRONMENT=development
+# TRANSCRIPTION_COMMIT default: unset (optional build/commit identifier for provenance evidence)
+# TRANSCRIPTION_COMMIT=
+
+# --- persistence ---
+# Use nested keys (env_nested_delimiter="__").
+DATABASE__DRIVER=sqlite
+DATABASE__PATH=./data/transcription.db
+# Postgres example:
+# DATABASE__DRIVER=postgres
+# DATABASE__HOST=localhost
+# DATABASE__PORT=5432
+# DATABASE__DATABASE=transcription
+# DATABASE__USER=postgres
+# DATABASE__PASSWORD=change-me
+BOOTSTRAP_SCHEMA_ON_STARTUP=false
+SQLITE_CHECK_SAME_THREAD=false
+
+# --- filesystem paths ---
+UPLOAD_DIR=./data
+PROMPT_DIR=./prompts
+DATABASE_BACKUP_DIR=./data/backups
+
+# --- worker reliability ---
+WORKER_MAX_RETRIES=0
+WORKER_PROVIDER_TIMEOUT_SECONDS=30.0
+WORKER_MIN_TRANSCRIPTION_CHARS=0
+WORKER_MIN_TRANSCRIPTION_LINES=0
+WORKER_FAIL_ON_FINISH_REASON_LENGTH=false

.github/agents/python-reviewer.agent.md

@@ -0,0 +1,24 @@
+---
+name: Python Architect Reviewer
+description: Evidence-based senior architect reviewer for FastAPI, NiceGUI, and SQLModel codebases.
+tools:
+  - read_file
+  - list_dir
+  - file_search
+  - grep_search
+  - run_in_terminal
+skills:
+  - python-code-reviewer
+---
+
+# Python Architect Reviewer
+
+You are a Senior Python Architect performing an evidence-based, read-only code review.
+
+## Operating Principles
+
+- **Stack Context:** Python 3.12+, FastAPI, NiceGUI, SQLModel, SQLAlchemy (SQLite/PostgreSQL), Pydantic V2, asyncio workers, and OpenRouter adapters.
+- **Evidence-Based:** Always inspect real files. Every finding must reference concrete file paths and line numbers (e.g., `app/services/worker.py:45-78`). Do not speculate.
+- **Tool Verification:** Run linters and tests via the terminal (`ruff check`, `pytest`, `ty`) to verify issues before reporting.
+- **Skill Execution:** Adhere strictly to the review dimensions, duplication analysis, and report scaffolding defined in the `python-code-reviewer` skill.
+- **Report Target:** Output all complete review reports as Markdown files written to `./docs`.

.github/instructions/documentation-sync.instructions.md

@@ -0,0 +1,35 @@
+---
+description: Require documentation updates whenever code changes alter contracts, behavior, or scope.
+applyTo: 'src/transcription/**/*.py'
+---
+
+# Documentation Sync Requirements
+
+Keep docs in sync in the same change whenever implementation alters a documented contract, behavior, or roadmap decision.
+
+## Update documentation when any of these change
+
+1. **Schema/Data contract**
+   - Models, fields, enums, constraints, indexes, relationships, loading semantics.
+   - **Required doc update:** `docs/schema.md`.
+
+2. **Configuration contract**
+   - `Settings` keys, defaults, required/optional environment values.
+   - **Required doc update:** `.env.example` and any directly related setup docs.
+
+3. **User-visible UI behavior**
+   - Page flow, routes, button/action behavior, labels, status wording, empty/error states.
+   - **Required doc update:** relevant `docs/ui/pages/*.md` docs and feature docs when applicable.
+
+4. **Error handling semantics**
+   - Error categories, retry behavior, envelope structure, translation boundaries.
+   - **Required doc update:** `docs/error_handling.md` and `docs/invariant/error_handling.md`.
+
+5. **Roadmap/scope decisions**
+   - Version targets, sequencing, deferrals, and accepted alternatives.
+   - **Required doc update:** `docs/roadmap_plan.md` and related backlog docs (for example `docs/ver4.8/feature_backlog_v4_8.md`).
+
+## Working rule
+
+If none of the categories above changed, documentation edits are optional.
+If any category changed, update docs in the same PR/change set rather than deferring.

.github/instructions/error-handling.instructions.md

@@ -0,0 +1,97 @@
+---
+description: Cross-cutting error handling rules for services, API, and UI.
+applyTo: 'src/transcription/**/*.py'
+---
+
+# Error Handling (Cross-cutting)
+
+Primary references:
+
+- `docs/error_handling.md`
+- `docs/invariant/error_handling.md`
+- `docs/requirements.md`
+
+## Taxonomy and Categories
+
+Use category-driven semantics aligned to canonical policy:
+
+- `validation`
+- `not_found`
+- `conflict`
+- `external`
+- `timeout`
+- `internal`
+
+Do not invent ad hoc categories in user/API-facing envelopes unless canonical docs are updated.
+
+Runtime/internal categories may be more specific for diagnostics and persistence, but they must map
+deterministically to the canonical envelope categories through the centralized mapper in
+`transcription.errors.canonical_error_category`.
+
+Current internal categories:
+
+- `validation_error`
+- `user_input_error`
+- `not_found_error`
+- `conflict_error`
+- `external_provider_error`
+- `external_timeout_error`
+- `processing_error`
+- `infrastructure_transient_error`
+- `infrastructure_persistent_error`
+- `internal_unexpected_error`
+
+Required internal -> canonical mapping:
+
+- `validation_error`, `user_input_error` -> `validation`
+- `not_found_error` -> `not_found`
+- `conflict_error` -> `conflict`
+- `external_provider_error` -> `external`
+- `external_timeout_error`, `infrastructure_transient_error` -> `timeout`
+- `processing_error`, `infrastructure_persistent_error`, `internal_unexpected_error` -> `internal`
+
+## Translation Boundaries
+
+- **Provider/adapters:** raise provider/domain exceptions; do not emit UI text.
+- **Services:** map raw exceptions into internal categories and preserve causal chain (`raise ... from ...`).
+- **UI/API:** map internal category -> canonical envelope category and emit user-safe, actionable messages.
+
+## Retry Rules
+
+- No auto-retry for `validation`, `not_found`, `conflict`.
+- `external`/`timeout` may be retried when operation semantics are safe.
+- Preserve each retry as new evidence where applicable (no history rewrite).
+
+## Job/Page Failure Semantics
+
+- Page-level (`JobSource`): `pending`, `transcribed`, `failed`, `cancelled`.
+- Job terminals: `transcribed`, `partial_success`, `failed`.
+- Cancellation must keep job-level and page-level semantics explicit and consistent.
+- Do not emit legacy terminal state language such as `completed` in active user/API lifecycle contracts.
+
+## User-Safe Messaging
+
+- Never leak stack traces, credentials, auth headers, or local filesystem paths in user-facing output.
+- Include actionable remediation guidance aligned to category.
+- Keep envelope structure consistent across API endpoints.
+
+## Logging and Diagnostics
+
+- Log operation identifiers and error IDs where available.
+- Preserve category + cause-chain context.
+- Distinguish no-response timeout/network failures from returned provider error responses.
+
+## Guardrails
+
+- No broad catch-and-swallow patterns.
+- No success-shaped fallback values after exceptions.
+- Category mapping must remain deterministic and testable.
+
+## Contract Sync Rule
+
+If taxonomy, retries, or envelope semantics change:
+
+1. Update canonical docs (`docs/error_handling.md`, and invariant docs if needed).
+2. Update tests in the same change.
+3. Update related instruction/skill references.
+4. If change affects persisted status/category fields, update `docs/schema.md` when applicable.

.github/instructions/services.instructions.md

@@ -7,29 +7,94 @@ applyTo: 'src/transcription/services/*.py'
 
 ## Structure
 
-- Project core data models defined in [models](../../src/transcription/models.py)
-- 1 service class per data model
-- Only services directly interact with the database, and only through async methods
-- Services are completely independent of one another. Any operation that needs to use more than a single service, which is most of them, needs to have a separate orchestration function.
+- Project core data models are defined in [models](../../src/transcription/db/models.py)
+- One service class per **aggregate**, not per table. An aggregate is a root model plus
+  the models that have no independent lifecycle of their own. `DocumentType` has no
+  meaning without `Document`, so it belongs to `DocumentService`; it does not get its
+  own service. Splitting per table produces services that must reach across each other
+  for every real operation, which is what line 13 forbids.
+- Only services interact with the database, and only through async methods.
+- **A service module must not import another service module.** This is enforced by
+  [test_service_boundaries](../../tests/test_service_boundaries.py). Shared types go in a
+  neutral module that defines no service class (see [errors](../../src/transcription/services/errors.py)).
+- Not every module in this package is a service. Helper modules that define no `*Service`
+  class (`base`, `errors`, `normalization`, `prompts`, `quality`, `media_storage`,
+  `source_media`) are free-function modules and are exempt from the service rules below.
+- Cross-cutting error behavior must follow
+  [error-handling instructions](./error-handling.instructions.md).
+
+## Model Ownership
+
+Every model has exactly one owning service. The owner defines that model's invariants and
+is the only service that may **create or delete** its rows.
+
+| Model | Owner |
+| --- | --- |
+| `Document`, `DocumentType` | `DocumentService` |
+| `Source`, `JobSource` | `SourceService` |
+| `Job` | `JobService` |
+| `Person`, `PersonRole`, `DocumentPerson` | `PeopleService` |
+| `ExecutionAttempt` | `SourceService` |
+
+### Junction tables
+
+A junction table is owned by the service that **creates and deletes its rows** — its
+lifecycle owner. The service on the other side may read through the junction (via
+`selectinload`) but must not create rows in it.
+
+- `document_person` -> `PeopleService`. Every write is there; `DocumentService` only
+  eager-loads through it.
+- `job_source` -> `SourceService`, which creates the row, records each page's outcome,
+  and deletes it.
+
+Two consequences follow, and both are deliberate:
+
+- **Cascade deletion is not a violation.** A service deleting the aggregate root it owns
+  may delete junction rows referencing that root, because they cannot outlive it
+  (`JobService.delete_job_with_guardrails`).
+- **Ownership governs creation and deletion, not every state transition.** `job_source` is
+  both a link and the transcription work queue. `JobService.cancel_job` and
+  `resubmit_failed_sources` transition `job_source.status` across a whole job, because that
+  transition is a Job lifecycle event, not a per-page outcome. They create and delete
+  nothing.
+
+`EvidenceService` is read-focused and projection-focused. It may coordinate selection
+flows, but append-only attempt creation remains in `SourceService` write paths.
+
+If a new operation cannot be expressed within one owner, it belongs in an orchestration
+module, not in a cross-service import.
 
 ## Error Handling
 
-- Service-specific errors defined at the top of the respective module and inherit from `AppError`
-- Use a context manager for large `try/except` blocks like in [transcription](../../src/transcription/services/transcription.py)
+- Errors used by a single service are defined at the top of that module and inherit from `AppError`.
+- Errors shared by more than one service go in [errors](../../src/transcription/services/errors.py),
+  which defines no service class and is therefore importable by any of them.
+- Use a context manager for large `try/except` blocks, like `handle_transcription_errors` in
+  [sources](../../src/transcription/services/sources.py).
+- Category mapping, retry behavior, and translation boundaries are defined in
+  [error-handling instructions](./error-handling.instructions.md).
+- Service-edge exception translation must be deterministic: map to canonical categories and preserve clear provider->service->API/UI boundaries.
 
 ## Checklist
 
 - [ ] Uses `ServiceBase` for common logic
-- [ ] CRUD methods created at the top
-- [ ] Session kwarg for `AsyncSession` to pass in a session object to each method
-- [ ] Services use `self._session_scope` in their methods to pass the session thru.
-    - Multiple operations on the same object(s) require sharing a session between all the methods used.
+- [ ] Session kwarg for `AsyncSession` to pass a session object into each method
+- [ ] Services use `self._session_scope` in their methods to pass the session through
+  - Multiple operations on the same object(s) require sharing a session between all the methods used
+- [ ] Every model the module touches is either owned by it or reached read-only
+- [ ] Evidence writes preserve append-only semantics
 
 ## CRUD Methods
 
-- Create, read, update, and delete, created in that order
-- Name format `<operation>_<model >`, for example `create_document` or `update_job`
-- All services must define these 4 methods first, and in that order
+- Name format `<operation>_<model>`, for example `create_document` or `update_job`.
+- Where a service exposes create/read/update/delete for its root model, define them at the
+  top of the class in that order, before derived reads and workflow helpers.
+- Not every aggregate needs all four. `ExecutionAttempt` is append-only evidence written by
+  `SourceService` workflow-facing methods, so `EvidenceService` deliberately exposes reads and
+  no create or delete.
+  Do not add unused CRUD methods to satisfy symmetry.
+- `RegistryService` is generic across small lookup models and uses `<operation>_entry`
+  naming instead.
 
 ## Transaction Finalization
 
@@ -39,18 +104,9 @@ When a service method accepts an optional `session` kwarg, write methods must us
 - If `session` is provided: the method must **not** commit; it should `flush()` so IDs and FK values are available to the caller's transaction.
 - Use `refresh()` on returned ORM objects when the caller needs DB-populated values (defaults, triggers, merged state).
 
-Recommended helper behavior:
-
-- Inputs: active session object, original `session` arg (or a boolean ownership flag), and an optional list of objects to refresh.
-- Logic: `commit` when service-owned session, `flush` when caller-owned session, then refresh requested objects.
-
-This keeps orchestration functions atomic: they can pass one shared session across multiple services and commit exactly once at the workflow boundary.
-
 ## Workflow Transaction Boundaries
 
-For multi-step job lifecycles (for example queued transcription jobs), orchestration functions must use explicit transaction phases.
-
-Required boundary model:
+For multi-step job lifecycles, orchestration functions must use explicit transaction phases.
 
 - **Transaction A (claim):** transition `JobStatus.QUEUED -> JobStatus.PROCESSING` and commit immediately.
 - Perform provider/network work **outside** database transactions.
@@ -64,14 +120,32 @@ Atomicity rules:
 - Terminal state (`TRANSCRIBED` or `FAILED`) and transcript row changes must succeed or roll back together.
 - Retry persistence (`QUEUED` + retry increment + error detail) must succeed or roll back together.
 
-Separation of concerns:
+## Contract Alignment
 
-- Worker modules should stay lightweight and delegate lifecycle transitions to service/workflow orchestration functions.
-- In `workflows.py`, `process_queued_job` should own one complete attempt lifecycle: `QUEUED -> PROCESSING -> TRANSCRIBED|FAILED`.
-- In `workflows.py`, `advance_job` should coordinate broader status progression around attempts (for example retry scheduling from `FAILED -> QUEUED`).
-- Services should expose session-aware write helpers (flush on caller-owned session) so orchestration controls commit boundaries.
-- Backoff/sleep behavior must run outside transactional scopes.
+- Treat `docs/` as the active architecture and requirements baseline.
+- Legacy revision trees are out of scope for active implementation decisions and must not be referenced as authoritative service guidance.
+- Treat `src/transcription/db/models.py` as runtime schema ground truth and `docs/schema.md` as the field-accurate contract mirror.
+- `Job.status` success path is `TRANSCRIBED`.
+- `JobSource.status` is queue/projection state only (`PENDING`, `TRANSCRIBED`, `FAILED`, `CANCELLED`).
+- Source ingest may normalize media before persistence; persisted bytes/hash are canonical for processing and provenance.
+- `ExecutionAttempt` is append-only evidence history; do not mutate historical attempt rows in runtime code.
+- `Source.raw_transcription` is a projection, not authoritative history.
+- Service/UI read paths that touch relationships must be eager-loaded for `lazy="raise"` compatibility.
+- If model fields, enums, constraints, indexes, or relationship-loading semantics change, update `docs/schema.md` in the same change.
+- If `Settings` fields or defaults change in `src/transcription/config.py`, update `.env.example` in the same change so keys/defaults remain synchronized and no stale settings remain documented.
+
+## Schema Drift and Legacy Compatibility Policy
+
+- Prefer schema migration or startup reconciliation over runtime compatibility paths in service writes.
+- Do not add legacy read/write compatibility code in service workflows by default.
+- If drift is discovered and a migration decision is ambiguous (for example, one-way destructive DDL, uncertain data retention impact, or unknown deployment sequence), pause and ask the user to choose migration vs compatibility before coding.
+- If a temporary compatibility path is explicitly approved, document an expiration/removal plan in the same change.
 
 # Service Composition
 
-Some operations, like uploading a picutre, require modifications to multiple tables, which can be done by composing methods from the service object into a separate function.
+A service method may read across models it does not own, using eager loads from its own
+aggregate root. What it may not do is import another service.
+
+Operations that must **write** models owned by more than one service are composed in an orchestration module
+([store](../../src/transcription/services/store.py),
+[workflows](../../src/transcription/services/workflows.py)).

.github/instructions/ui.instructions.md

@@ -1,6 +1,78 @@
 ---
-description: Copilot rules for modifying the UI
+description: "Use when modifying the NiceGUI application under src/transcription/ui. Defines ownership and dependency boundaries for UI registration, pages, components, services, persistence, state, and static assets."
 applyTo: 'src/transcription/ui/**/*.py'
 ---
 
-The UI is forbidden from directly touching the database. All interactions should be done using the [services](../../src/transcription/services/)
+# UI Conceptual Boundaries
+
+Keep dependencies flowing in this direction:
+
+`ui/__init__.py` -> `pages` -> `components`
+
+Pages may depend on application services and framework-provided dependencies. Components may depend on smaller components and shared presentation helpers. Services and domain modules must never depend on the UI.
+
+Cross-cutting error behavior must follow
+[error-handling instructions](./error-handling.instructions.md).
+
+## Package Root
+
+- Keep `ui/__init__.py` as the UI composition root: register global assets, register pages, and mount NiceGUI on FastAPI.
+- Do not put feature rendering, service calls, persistence, or route-specific state in the package root.
+
+## Pages
+
+- Pages own route registration and route-level orchestration.
+- Resolve request or application dependencies, call [services](../../src/transcription/services/), adapt returned data for presentation when needed, and coordinate refresh, navigation, and notifications here.
+- Do not query, mutate, commit, or roll back the database from a page. Do not import database engines, sessions, operations, or query-building APIs. Persistence belongs to services or workflow functions.
+- Framework dependency types may cross into page handlers only to construct or invoke services; do not pass sessions or session factories into components.
+- Keep business rules, lifecycle transitions, transaction boundaries, and cross-service workflows out of page callbacks.
+
+## Components
+
+- Components own reusable rendering, widget-local state, input normalization, and presentation-only formatting.
+- Expose user actions through typed callback parameters. The calling page decides which service or workflow runs and what refresh or navigation follows.
+- Do not register routes, resolve request/app state, instantiate services, or access persistence from components.
+- Components may accept ORM models returned by services as read-only snapshots. Only use fields and relationships that the service loaded eagerly; never mutate models, trigger lazy loading, or expose session behavior.
+- A component may compose lower-level components, but it must not import from `pages`.
+
+## Shared UI Infrastructure
+
+- Keep app-wide navigation and layout primitives in `components/app_shell.py`.
+- Keep generic table/event adaptation in `components/table/common.py`; feature-specific columns, row read models, and formatting belong in the feature table module.
+- Keep exception normalization and user-facing error display in `components/error_presenter.py`; preserve `AppError` details and operation identifiers at page/component boundaries.
+- Use `components/media_urls.py` for media URL generation; do not hand-build upload/static paths in page code.
+
+## CSS Assets
+
+- Keep all application CSS in `ui/static/theme.css`; do not add page- or component-specific stylesheets or embed style blocks in Python components.
+- Load `theme.css` once from the composition root with `ui.add_css(..., shared=True)`.
+- Read stylesheet text through `importlib.resources.files(...)` so loading works from installed packages and is independent of the working directory.
+- Centralize CSS reading in one typed helper cached by resource path.
+- Do not encode application behavior in CSS or other static assets.
+
+## State and Side Effects
+
+- Limit component state to ephemeral interaction state such as loading flags, form values, dialogs, and expansion state.
+- Application and worker state must be resolved at the page or application boundary and passed through narrow interfaces.
+- Keep filesystem, network, provider, and worker orchestration behind application services or dedicated adapters.
+
+## Media Route Safety Rules
+
+Two patterns are approved:
+
+1. **Record-validated API routes** for print/export contexts.
+2. **Controlled upload URL resolver** (`components/media_urls.py`) for general UI media.
+
+Prohibited patterns:
+
+- Direct `file://` links or exposing local filesystem paths.
+- Manual URL construction from raw `Path` values in pages/components.
+- User-facing payloads containing local absolute paths.
+
+## Contract Alignment
+
+- Treat `docs/` as the active baseline.
+- Resolve lifecycle and status semantics against `src/transcription/db/models.py` and `docs/schema.md`; do not introduce alternate status labels or implied legacy states in UI behavior.
+- Use status vocabulary exactly as modeled (`queued`, `processing`, `transcribed`, `partial_success`, `failed`; and `pending`, `transcribed`, `failed`, `cancelled`).
+- Print/export media flows must use record-validated routes; direct local filesystem paths are prohibited.
+- If lifecycle wording/behavior changes, update corresponding `docs/ui/pages/*.md` contracts in the same change.

.github/prompts/review-python-architecture.prompt.md

@@ -0,0 +1,23 @@
+---
+name: Review Python Architecture
+description: Run an evidence-based architectural code review using the Python Architect Reviewer agent and python-code-reviewer skill.
+agent: Python Architect Reviewer
+---
+
+# Instructions
+
+Execute a comprehensive, evidence-based code review of the target codebase.
+
+## Target Scope
+- **Review Target:** ${{input:target_path:./}}
+- **Source Root:** `src/`
+- **Docs Root:** `docs/`
+- **Focus Areas:** FastAPI endpoints, NiceGUI components, SQLModel persistence, asyncio workers, Pydantic V2 models, and OpenRouter provider adapters.
+
+## Execution Rules
+1. Map repository layout, dependency manifests, and configuration files from the project root before inspecting modules.
+2. Read real code modules under `src/` (or the specified target path); cite exact file paths and line ranges for every finding.
+3. Validate issues using terminal tools (`ruff check`, `pytest`, `ty`) where appropriate.
+4. Check for duplication, divergent implementations, and extractable helpers.
+5. Format the entire review following the standardized 6-section template defined in the `python-code-reviewer` skill.
+6. Write the final report as a Markdown file to `./docs/code-review-${{current_date}}.md`.

.github/skills/evidence-provenance-auditor/skill.md

@@ -0,0 +1,80 @@
+---
+name: evidence-provenance-auditor
+description: Deterministic reviewer for transcription evidence/provenance guarantees. Use when changes touch execution attempts, source storage, retries, transport evidence, artifact provenance, or evidence exports.
+---
+
+# Evidence & Provenance Auditor
+
+Perform focused, deterministic audits of evidence integrity and provenance behavior.
+
+## When to Use
+
+- Reviewing changes in:
+  - `src/transcription/services/sources.py`
+  - `src/transcription/services/store.py`
+  - `src/transcription/services/workflows.py`
+  - `src/transcription/services/evidence.py`
+  - `src/transcription/db/models.py`
+- Auditing evidence exports/imports or evidence-display behavior.
+- Verifying no drift from canonical provenance invariants.
+
+## Normative References (must be used)
+
+1. `docs/invariant/ai_evidence_and_provenance.md`
+2. `docs/schema.md`
+3. `docs/requirements.md`
+4. `docs/error_handling.md`
+
+## Deterministic Pass/Fail Checks
+
+### A. Append-only history
+- Every provider call results in a new `ExecutionAttempt`.
+- Runtime paths do not mutate historical attempts to represent new outcomes.
+- Retry behavior appends attempts rather than rewriting prior rows.
+
+### B. Projection vs authority separation
+- `Source.raw_transcription` and preferred pointers are mutable projection surfaces.
+- Attempt rows remain authoritative historical evidence.
+- Candidate promotion updates projection pointers without rewriting history.
+
+### C. Transport evidence semantics
+- Transport evidence is correctly labeled as application-boundary capture.
+- SDK snapshots/normalized metadata are not mislabeled as native upstream payload.
+- No-response timeout/network states are explicit.
+
+### D. Canonical source identity
+- Canonical stored bytes/hash/size are internally consistent.
+- If ingest normalization is applied, code/docs consistently represent resulting canonical identity.
+- Post-ingest derivatives do not overwrite canonical source bytes.
+
+### E. Secret safety
+- No credentials/auth headers/cookies/unrestricted headers persisted.
+- Header persistence uses explicit allowlist semantics.
+
+### F. Route/path safety
+- Print/export source access is record-validated.
+- UI/media path construction does not expose local filesystem paths.
+
+### G. Schema/docs alignment
+- Evidence-related model fields and semantics align with canonical docs.
+- Evidence model changes require same-change doc updates.
+
+### H. Canonical authority boundaries
+- Active guidance resolves against `docs/*` and current instruction files.
+
+## Review Workflow
+
+1. Read normative references first.
+2. Inspect model + service + workflow write paths.
+3. Inspect evidence read/display/export paths.
+4. Report high-confidence findings with concrete path/line evidence.
+5. Classify each finding by invariant family (A-H).
+
+## Output Format
+
+Use this structure:
+
+- Verdict by invariant family (A-H)
+- Findings with `Location`, `Observed Behavior`, `Risk`, `Recommended Fix`
+- Drift table (`Doc claim` vs `Code reality` vs `Action`)
+- Regression guards needed

.github/skills/python-code-reviewer/skill.md

@@ -0,0 +1,203 @@
+---
+name: python-code-reviewer
+description: Perform an evidence-based, senior architect code review for Python codebases using FastAPI, NiceGUI, SQLModel, SQLAlchemy, Pydantic V2, asyncio, and OpenRouter. Use when asked to review Python repositories, perform architectural or code audits, or evaluate code against Python 3.12+ best practices.
+---
+
+# Python Code Reviewer
+
+Perform thorough, evidence-based code reviews for Python projects. Every finding must cite concrete file paths and line ranges, avoid speculation, and include recommended fixes.
+
+## When to Use
+
+- Performing an architectural or code quality review of a Python codebase.
+- Auditing applications using FastAPI, NiceGUI, SQLModel/SQLAlchemy, Pydantic V2, or asyncio workers.
+- Generating structured Markdown review reports in `./docs`.
+
+## Technical Stack Scope
+
+- **Runtime:** Python 3.12+
+- **Web Application:** FastAPI and NiceGUI
+- **Persistence:** SQLModel, SQLAlchemy (SQLite and PostgreSQL support)
+- **Validation & Settings:** Pydantic V2 and pydantic-settings
+- **Concurrency:** Python asyncio workers
+- **Vision/LLM Integration:** OpenRouter / provider adapters
+- **Quality & Testing:** pytest, pytest-asyncio, Ruff, and ty
+
+## Review Workflow
+
+1. **Map the Repository First:** Inspect entry points, package layout, configurations, dependency manifests, and any project-specific rule files (`AGENTS.md`, `CLAUDE.md`, `.github/instructions/`). Project-specific conventions override generic advice.
+2. **Establish Canonical Authority First:** Read architecture/contracts (`docs/*`, `docs/invariant/*`, UI docs) and active instructions/skills before evaluating source behavior.
+3. **Read Representative Modules:** Sample across all layers (routes/pages, UI components, services, workers, persistence, provider adapters, settings, tests) before drawing conclusions.
+4. **Run Drift Analysis:** Compare documented intended behavior versus repository ground truth; identify both implementation drift and undocumented-but-repeatable conventions that should be formalized.
+5. **Run Dead-Code/Orphan Sweep:** Identify candidate orphan modules/functions/classes with zero inbound references, then verify expected exceptions (entrypoints, framework/plugin registration, dynamic imports/reflection, CLI hooks, test-only utilities) before marking as orphaned.
+6. **Assess Boundary and Coupling Health:** Evaluate UI/service/persistence/provider dependency flow, identify circular dependencies, leaky abstractions, and transaction ownership ambiguity.
+7. **Assess Invariant Placement:** For each hard rule, decide whether it belongs in docs (rationale), instructions (active steering), skills (periodic audit procedure), or deterministic tests (enforcement).
+8. **Verify Claims:** Run or reference project tooling (`ruff check`, `ty`, `pytest`) rather than guessing.
+9. **Prioritize Hot Paths:** Focus deeply on request handling, database sessions, background workers, and external API calls.
+10. **Enforce Read-Only Safety:** Do not modify code unless explicitly instructed.
+11. **Escalate Provenance Audits:** For evidence/provenance-heavy changes, apply invariant checks from `.github/skills/evidence-provenance-auditor/skill.md` and include pass/fail outcomes in the report.
+
+## Repo-Specific Deterministic Checks (Transcription)
+
+When reviewing this repository, always include explicit pass/fail checks for:
+
+1. **Service boundary rule:** no service-to-service imports (`tests/test_service_boundaries.py`).
+2. **UI boundary rule:** pages/components do not perform persistence access (`tests/test_ui_boundaries.py`).
+3. **Status vocabulary conformance:** `JobStatus`/`JobSourceStatus` usage matches current enums in `src/transcription/db/models.py`.
+4. **Evidence ownership conformance:** append-only attempt history is preserved and projection writes are not mistaken for history mutation (`src/transcription/services/sources.py`, `src/transcription/services/evidence.py`).
+5. **Canonical authority:** findings must resolve against `docs/*` first.
+6. **Schema contract fidelity:** when model/persistence behavior changes, `docs/schema.md` remains field-accurate with `src/transcription/db/models.py`.
+7. **Media boundary conformance:** print/export media is record-validated and UI media URL generation uses controlled resolver paths.
+8. **Eager-loading conformance:** service/UI read paths satisfy `lazy="raise"` expectations.
+9. **Cross-cutting error conformance:** service/API/UI translation and retry behavior align with `.github/instructions/error-handling.instructions.md`.
+10. **Orphaned/dead-code conformance:** include a deterministic orphan sweep and report confirmed orphans removed/retained with rationale.
+
+## Core Review Areas
+
+### 1. Python Best Practices (3.12+)
+- **Type Annotations:** Ensure completeness, modern syntax (`X | None`, builtin generics, `Self`, `type` statements), and avoid unparameterized containers or bare `Any`.
+- **Error Handling:** Identify bare/broad `except`, swallowed exceptions, missing `raise ... from`, and exceptions used for control flow.
+- **Resource Management:** Verify context managers for files, DB sessions, HTTP clients, and locks. Check for leaked tasks or connections.
+- **Data Modeling:** Check proper use of dataclasses vs. Pydantic models vs. dictionaries. Eliminate mutable default arguments and stringly-typed payloads.
+- **Idioms & Clean Code:** Verify `pathlib` usage over `os.path`, comprehensions vs manual loops, removal of dead code, and elimination of magic numbers.
+
+### 2. FastAPI
+- **Dependency Injection:** Verify `Depends` is used for shared resources (DB sessions, settings, clients) rather than global singletons.
+- **Route Design:** Validate HTTP verbs, status codes, path/query/body typing, `response_model`, and domain-based router organization.
+- **Lifecycle & Concurrency:** Ensure lifespan handlers are used instead of deprecated `@app.on_event`. Flag blocking synchronous calls in `async def` endpoints.
+
+### 3. NiceGUI
+- **Separation of Concerns:** Ensure UI components delegate business logic and persistence to service layers.
+- **Client State Handling:** Verify correct use of client-scoped state vs global state to avoid state leaks across sessions.
+- **Async Execution:** Check for blocking operations on the UI event loop and unbounded timers/pollers.
+
+### 4. Persistence (SQLModel / SQLAlchemy)
+- **Session Lifecycle:** Enforce one session per request/unit of work with explicit commit/rollback/close boundaries.
+- **Query Optimization:** Detect N+1 patterns, missing eager loads (`selectinload`/`joinedload`), queries inside loops, and unindexed filters.
+- **Cross-Dialect Portability:** Check compatibility for both SQLite (WAL mode, pragmas) and PostgreSQL (JSONB, locking, autoincrement).
+
+### 5. Pydantic V2 & Settings
+- **V2 Migration:** Flag legacy V1 patterns (`@validator`, `Config` class, `.dict()`, `parse_obj`) and use V2 equivalents (`@field_validator`, `model_config = ConfigDict(...)`, `model_dump()`).
+- **Settings Management:** Ensure `BaseSettings` is the single source of truth without scattered `os.getenv` calls or committed secrets.
+
+### 6. Concurrency & Asyncio Workers
+- **Task Lifecycle:** Flag unreferenced `create_task` calls that risk garbage collection, missing cancellation handling, and lack of graceful shutdown.
+- **Backpressure & Synchronization:** Check for appropriate use of `asyncio.Queue`, `TaskGroup`, `Lock`, and backoff retries.
+
+### 7. Provider Adapters (OpenRouter / APIs)
+- **Adapter Encapsulation:** Verify provider-specific details (headers, model names, payload formats) do not leak into UI or business logic.
+- **Client Lifecycle:** Reuse shared `AsyncClient` instances with proper connection pooling and timeouts. Validate API responses using Pydantic schemas.
+
+### 8. Testing & Quality Tooling
+- **Test Isolation:** Verify tests do not rely on live external services, real clocks, or shared global state.
+- **Async Test Setup:** Check `pytest-asyncio` configuration and fixture lifecycle.
+
+### 9. Duplication & Consolidation
+- Identify repeated code blocks, candidate helper abstractions, divergent patterns for identical operations, and duplicated domain constants.
+
+### 10. Orphaned/Dead Code Audit
+- Find candidate orphan modules/functions/classes with no inbound references.
+- Validate each candidate against dynamic wiring exceptions (entrypoints, plugin registration, reflection/dynamic imports, CLI hooks, test utilities).
+- Report outcomes as: removed orphan, retained-with-justification, or uncertain-follow-up.
+
+### 11. Architecture & Governance
+- **Architectural Drift:** Compare intended architecture rules against implementation behavior and cite concrete drift points.
+- **Systemic Health:** Evaluate domain cohesion, dependency direction, lifecycle consistency, and operational reliability seams.
+- **Invariant Routing:** Recommend the correct enforcement layer per rule (docs vs instructions vs skills vs tests).
+- **Meta-Tooling Alignment:** Recommend updates for instruction files and skills when repository patterns or contracts evolve.
+
+## Output Report Structure & Template
+
+Generate Markdown reports in `./docs` following this exact template structure:
+
+```markdown
+# Architecture & Code Review Report
+
+**Repository Target:** `project-root/`
+**Target Stack:** Python 3.12+ | FastAPI | NiceGUI | SQLModel/SQLAlchemy | Pydantic V2 | asyncio | OpenRouter
+
+---
+
+## 1. Executive Summary
+- 5-10 bullets on overall health, top risks, and high-leverage refactors.
+
+---
+
+## 2. Executive Architecture Assessment
+- High-level verdict on domain cohesion, boundary clarity, and architecture fitness.
+- Top 3-5 systemic risks or bottlenecks.
+
+---
+
+## 3. Findings by Severity
+
+### Critical Severity
+#### [CRIT-01] Title
+- **Location:** `path/to/file.py:lines`
+- **Problem & Consequence:** Concrete consequence, not a style opinion.
+- **Recommendation:** Fix with before/after sketch.
+- **Effort:** S / M / L
+
+### High Severity
+#### [HIGH-01] Title
+...
+
+### Medium Severity
+#### [MED-01] Title
+...
+
+### Low Severity
+#### [LOW-01] Title
+...
+
+---
+
+## 4. Architectural Drift & Gap Analysis
+| Area / Component | Documented / Intended Rule | Actual Implementation State | Severity | Recommended Resolution |
+| :--- | :--- | :--- | :--- | :--- |
+
+---
+
+## 5. Invariant Inventory & Routing Recommendations
+| Invariant / Constraint | Current Location | Recommended Target Layer | Rationale |
+| :--- | :--- | :--- | :--- |
+
+---
+
+## 6. Stack-Specific Analysis
+- Python 3.12+ Best Practices
+- FastAPI
+- NiceGUI
+- SQLModel & SQLAlchemy
+- Pydantic V2 & Settings
+- Asyncio Workers
+- OpenRouter / Adapter Boundary
+- Testing & Quality Tooling
+
+---
+
+## 7. Duplication & Consolidation Report
+| Pattern / Duplication | Locations | Proposed Canonical Home | Estimated Lines Removed |
+| :--- | :--- | :--- | :--- |
+
+### Proposed Canonical Abstractions
+- Code signatures and implementation homes.
+
+---
+
+## 8. Meta-Tooling & Instruction Update Recommendations
+- Required updates to docs/instructions/skills/tests to keep enforcement current.
+
+---
+
+## 9. Prioritized Dependency-Ordered Action Plan
+1. **Phase 1: Blocking fixes**
+2. **Phase 2: Enforcement hardening**
+3. **Phase 3: Reliability & concurrency**
+4. **Phase 4: Consolidation & refactoring**
+5. **Phase 5: Non-blocking governance/documentation depth**
+
+---
+
+## 10. Preserved Strengths
+- Existing patterns worth maintaining.

.github/skills/test-effectiveness-auditor/skill.md

@@ -0,0 +1,96 @@
+---
+name: test-effectiveness-auditor
+description: Periodic reviewer for test-suite signal quality. Detects low-value or redundant tests, validates contract coverage, and recommends pruning or strengthening actions.
+---
+
+# Test Effectiveness Auditor
+
+Run a deterministic audit of test usefulness. Focus on whether tests catch real regressions, not whether they merely execute code.
+
+## When to Use
+
+- Monthly/quarterly test-health review.
+- Pre-release hardening when test count grows quickly.
+- After major AI-assisted test generation.
+- When suite runtime is increasing without clear quality gains.
+
+## Primary Objectives
+
+1. Identify tests that are weak, redundant, or non-diagnostic.
+2. Confirm critical contracts are guarded by meaningful assertions.
+3. Produce a prune/strengthen backlog with explicit risk and effort.
+
+## Normative References (Transcription Repo)
+
+1. `docs/*`
+2. `docs/invariant/*`
+3. `.github/instructions/*.instructions.md`
+4. `tests/test_meta_contract_guards.py`
+5. Contract-specific guards (`tests/test_service_boundaries.py`, `tests/test_ui_boundaries.py`, worker/evidence/media/error suites)
+
+## Deterministic Audit Checks
+
+### A. Contract Traceability
+- Each high-risk contract maps to at least one focused regression test file.
+- Missing mapping is a gap.
+
+### B. Assertion Strength
+- Flag tests that only assert status code, non-null, or “no exception” without validating state transitions or persisted outcomes.
+- Prefer assertions on domain effects: DB rows, status changes, error categories, evidence writes, or emitted payload shape.
+
+### C. Failure-Path Coverage
+- Critical paths must include negative-path tests (timeouts, provider errors, validation failures, cancellation paths, retries).
+- Happy-path-only coverage on critical modules is a gap.
+
+### D. Redundancy and Noise
+- Detect near-duplicate tests asserting the same behavior at multiple layers with no extra signal.
+- Recommend canonical location (unit/integration) and prune overlaps.
+
+### E. Mutation/Change Sensitivity
+- Prefer mutation testing for high-risk modules when practical.
+- If not run, identify tests likely to survive meaningful code mutations (low sensitivity).
+
+### F. Drift Guards
+- Verify config/doc/instruction contracts have deterministic guards and are current.
+- Ensure settings/docs synchronization checks remain active.
+
+## Evidence Standards
+
+- Every finding must include concrete file paths and line ranges.
+- No speculative claims.
+- Distinguish clearly between:
+  - **Confirmed ineffective tests**
+  - **Likely weak tests (needs mutation/probe confirmation)**
+
+## Output Format
+
+Produce a Markdown report in `docs/`:
+
+```markdown
+# Test Effectiveness Audit Report
+
+## 1. Executive Verdict
+- Effective / Effective with Conditions / Needs Remediation
+- Top risks to confidence
+
+## 2. Contract Coverage Matrix
+| Contract | Guarding Tests | Signal Quality | Gap | Action |
+| :--- | :--- | :--- | :--- | :--- |
+
+## 3. Weak/Redundant Test Findings
+| Finding ID | Location | Why Low-Signal | Risk | Recommendation |
+| :--- | :--- | :--- | :--- | :--- |
+
+## 4. Prune/Strengthen Backlog
+| Task ID | Goal | Files | Acceptance Criteria | Validation |
+| :--- | :--- | :--- | :--- | :--- |
+
+## 5. Confidence Recommendation
+- Go / Go with Conditions / No-Go for release confidence
+```
+
+## Decision Rules
+
+- Do not recommend deleting a test unless equivalent or stronger coverage is identified.
+- Prefer strengthening assertions before adding more tests.
+- Prioritize deterministic contract guards over broad snapshot-style tests.

.github/workflows/quality-gate.yml

@@ -0,0 +1,45 @@
+name: Quality Gate
+
+# V4.7 Phase 6 / review log [40]. Before this, ruff, ty and pytest were enforced
+# only by .pre-commit-config.yaml, and only for developers who had actually run
+# `pre-commit install`.
+
+on:
+  push:
+  pull_request:
+
+jobs:
+  gate:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out the commit under test
+        uses: actions/checkout@v4
+
+      - name: Install uv
+        run: |
+          curl -LsSf https://astral.sh/uv/install.sh | sh
+          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
+
+      - name: Install dependencies from the lockfile
+        # --locked fails if uv.lock has drifted from pyproject.toml, so a stale
+        # lockfile is caught here rather than producing an untested dependency set.
+        run: uv sync --locked
+
+      - name: Write placeholder configuration
+        # Settings requires openrouter_api_key and 115 tests cannot construct
+        # Settings without it. This is written to a .env file rather than exported
+        # as an environment variable on purpose: the external tests guard on
+        # os.getenv("OPENROUTER_API_KEY"), which reads the process environment and
+        # not the file, so writing the file reproduces the local result exactly -
+        # the 4 external tests skip instead of running against a fake key and
+        # failing. Exporting it instead produces 3 failures.
+        run: echo "OPENROUTER_API_KEY=ci-placeholder-not-a-real-key" > .env
+
+      - name: Lint and type check
+        # Runs the hooks defined in .pre-commit-config.yaml instead of repeating
+        # "ruff check" and "ty check" here. The commands then have one definition,
+        # so the local and CI gates cannot drift apart.
+        run: uv run pre-commit run --all-files --show-diff-on-failure
+
+      - name: Tests
+        run: uv run pytest

.gitignore

@@ -17,3 +17,7 @@ wheels/
 
 # Document images
 uploads/*
+data/*
+
+# Local destructive-test backups
+.test-backups/

.pre-commit-config.yaml

@@ -0,0 +1,18 @@
+# Quality gate for V4.6 [HIGH-06]. Both hooks are blocking: a regression in
+# `ruff check` or `ty check` fails the commit.
+repos:
+  - repo: local
+    hooks:
+      - id: ruff
+        name: ruff check
+        entry: ruff check
+        language: system
+        types_or: [python, pyi]
+        require_serial: true
+      - id: ty
+        name: ty check
+        entry: ty check
+        language: system
+        types_or: [python, pyi]
+        pass_filenames: false
+        require_serial: true

.vscode/launch.json

@@ -8,14 +8,10 @@
             "module": "debugpy",
             "args": [
                 "-m",
-                "uvicorn",
-                "transcription.app:create_app",
-                "--factory",
-                "--host",
-                // "127.0.0.1",
-                "0.0.0.0",
-                "--port",
-                "8080"
+                "transcription",
+                "--host", "127.0.0.1",
+                "--port", "9999",
+                "--database.driver", "sqlite"
             ],
             "justMyCode": true,
             "console": "integratedTerminal",

.vscode/settings.json

@@ -0,0 +1,3 @@
+{
+    "chat.sessionSync.enabled": true
+}

README.md

@@ -22,18 +22,83 @@ uv sync
 
 ### 2) Configure environment
 
-Create a `.env` file in the project root (minimum required setting shown):
+Create a `.env` file in the project root with the required OpenRouter API key:
 
 ```env
 OPENROUTER_API_KEY=your_openrouter_api_key
 ```
 
-Optional settings (defaults shown):
+Settings are read from CLI arguments first, then environment variables, then `.env`, then the defaults below.
+
+### Configuration Source Precedence
+
+When the same setting is provided in multiple places, the value is chosen in this order (highest priority first):
+
+1. CLI arguments (for example `--port 8000`)
+2. Settings constructor arguments (used mainly in tests)
+3. Environment variables
+4. `.env` file values
+5. Model defaults in code
+
+Practical examples:
+
+- `--port 8000` overrides both `PORT=8000` in the shell and `PORT=7000` in `.env`.
+- `DATABASE__PATH=prod.db` in the shell overrides `DATABASE__PATH=dev.db` in `.env`.
+
+#### Server and runtime
+
+| Environment variable | Default | Description |
+| --- | --- | --- |
+| `HOST` | `0.0.0.0` | Address on which the server listens. |
+| `PORT` | `8000` | Server port. |
+| `LOG_LEVEL` | `info` | Uvicorn and application log level. |
+| `RELOAD` | `false` | Restart the development server when source files change. |
+| `ENVIRONMENT` | `development` | Runtime environment: `development`, `test`, or `production`. |
+
+#### Provider
+
+| Environment variable | Default | Description |
+| --- | --- | --- |
+| `PROVIDER` | `openrouter` | Transcription provider. |
+| `OPENROUTER_API_KEY` | Required | OpenRouter API key. |
+| `PROVIDER_MODEL` | Provider default | Optional model override. |
+| `OPENROUTER_HTTP_REFERER` | Unset | Optional OpenRouter attribution URL. |
+| `OPENROUTER_APP_TITLE` | Unset | Optional OpenRouter attribution title. |
+
+#### Database and files
+
+Use nested env vars for database settings (recommended):
 
 ```env
-DATABASE_URL=sqlite:///./transcription.db
+DATABASE__DRIVER=sqlite
+DATABASE__PATH=app.db
+# BOOTSTRAP_SCHEMA_ON_STARTUP=true
+SQLITE_CHECK_SAME_THREAD=false
 UPLOAD_DIR=./uploads
 PROMPT_DIR=./prompts
+DEFAULT_PROMPT_NAME=transcribe_document.md
+# TRANSCRIPTION_TEMPERATURE=0.2  # range: 0.0-2.0
+# TRANSCRIPTION_TOP_P=0.9       # range: 0.0-1.0
+```
+
+For PostgreSQL:
+
+```env
+DATABASE__DRIVER=postgres
+DATABASE__HOST=localhost
+DATABASE__PORT=5432
+DATABASE__DATABASE=transcription
+DATABASE__USER=postgres
+DATABASE__PASSWORD=change-me
+```
+
+This uses Pydantic nested settings (`env_nested_delimiter='__'`) and avoids JSON blobs in `.env`. A top-level `DATABASE={...}` JSON value is still supported as a fallback, and nested keys such as `DATABASE__PATH` take precedence over conflicting JSON keys.
+
+`BOOTSTRAP_SCHEMA_ON_STARTUP` creates missing tables when the app starts. When unset, it is enabled in `development` and `test`, and disabled in `production`; set it explicitly to override that policy. `SQLITE_CHECK_SAME_THREAD` defaults to `false`.
+
+#### Worker
+
+```env
 WORKER_MAX_RETRIES=0
 WORKER_RETRY_BACKOFF_SECONDS=0
 WORKER_PROVIDER_TIMEOUT_SECONDS=20
@@ -45,13 +110,17 @@ WORKER_FAIL_ON_FINISH_REASON_LENGTH=false
 ### 3) Run the app
 
 ```bash
-uv run uvicorn transcription.app:create_app --factory --reload
+uv run python -m transcription --port 8000 --reload --database.driver sqlite --bootstrap-schema-on-startup
 ```
 
+This starts the development server with SQLite, creates missing tables, and enables automatic reload. Run `uv run python -m transcription --help` for all CLI options; CLI names use kebab case and nested database options use dot notation, such as `--database.path ./data/transcription.db`.
+
 ### 4) Open in browser
 
-- GUI: [http://[IP_ADDRESS]:8000/ui](http://[IP_ADDRESS]:8000/ui)
-- Health check: [http://[IP_ADDRESS]:8000/healthz](http://[IP_ADDRESS]:8000/healthz)
+- GUI: [http://localhost:8000/ui](http://localhost:8000/ui)
+- Health check: [http://localhost:8000/healthz](http://localhost:8000/healthz)
+
+Replace `localhost` with the server's hostname or IP address when connecting from another machine.
 
 ## How to navigate the GUI
 
@@ -72,7 +141,70 @@ uv run uvicorn transcription.app:create_app --factory --reload
 
 ## Prompt artifacts
 
-Prompt files are stored in `prompts/` and loaded from `PROMPT_DIR` (default: `./prompts`).
+Prompt files are stored directly in `PROMPT_DIR` (default: `./prompts`). `DEFAULT_PROMPT_NAME` must be a filename,
+not a path. Each job snapshots the validated prompt text, SHA-256 hash, and sampling values for reproducibility.
 
 The canonical MVP prompt is:
 - `prompts/transcribe_document.md`
+
+## Database migration workflow
+
+Schema upgrades use an explicit export/import rebuild flow (no runtime legacy write compatibility).  
+See `docs/data_migration.md` for commands and cutover steps.
+
+## Destructive test procedure (with data backup)
+
+AI execution policy: before the first unit-test run in a test/fix cycle, create one backup of `./data`. Reuse that same backup for every subsequent test run in the cycle. After tests succeed, always pause and ask whether to restore now.
+
+Use the cross-platform Python wrapper below whenever an AI agent runs tests against this repository.
+
+1. Create one backup of `./data` and mark it as the active test-cycle backup.
+2. Run your test command.
+3. On failure, fix the errors and run the wrapper again; it reuses the active backup and never backs up post-test data.
+4. On success, always prompt whether to restore now (do not auto-restore unless explicitly approved).
+5. Close the cycle only by restoring the active backup or explicitly accepting the current data.
+
+Preflight behavior:
+
+- Backup preflight is warning-only when `data/transcription.db` appears in use.
+- Restore preflight is blocking: the script prompts you to close conflicting applications, then type `retry` to re-check or `cancel` to skip restore.
+
+### Run with confirmation-gated restore (default)
+
+```bash
+uv run python tools/run_destructive_tests.py -- pytest tests/services/test_job_service.py tests/ui/test_jobs_page.py
+```
+
+After tests pass, the script asks whether to restore backup immediately.
+
+This is the required default mode for AI-assisted test runs because it gives time to verify and accept code changes before any restoration happens.
+
+### Run with automatic restore (non-interactive)
+
+```bash
+uv run python tools/run_destructive_tests.py --auto-restore -- pytest
+```
+
+### Run without terminal prompt (decide restore later)
+
+```bash
+uv run python tools/run_destructive_tests.py --skip-restore-prompt -- pytest
+```
+
+This keeps both the current post-test state and the backup, so restore can be decided explicitly later.
+
+Repeated wrapper invocations reuse the backup recorded in `.test-backups/.active-backup`. If that backup is missing, the wrapper stops rather than creating a replacement from potentially destructive post-test data.
+
+### Restore later from a saved backup
+
+```bash
+uv run python tools/run_destructive_tests.py --restore-from data-backup-YYYYMMDD-HHMMSS
+```
+
+To keep the current data and close the active cycle without restoring:
+
+```bash
+uv run python tools/run_destructive_tests.py --accept-current-data
+```
+
+Backups are stored in `.test-backups/` and ignored by git.

docker-compose.yml

@@ -7,7 +7,10 @@ services:
     env_file:
       - .env
     environment:
-      DATABASE_URL: sqlite:////app/data/transcription.db
+      # Database configuration uses nested settings names (env_nested_delimiter="__").
+      # DATABASE_URL is NOT read by the application and must not be used here.
+      DATABASE__DRIVER: sqlite
+      DATABASE__PATH: /app/data/transcription.db
       UPLOAD_DIR: /app/uploads
       PROMPT_DIR: /app/prompts
     ports:

docs/Intent.md

@@ -1,24 +0,0 @@
-# Historical Document Transcription Design Intent
-I have several thousand pages of family history told through letters, postcards, books, and other documents that I want to transcribe to text. 
-
----
-
-## Goals
-1. Preserve our family history
-2. Unburden my family (and descendants) from having to store and care for the physical media. Once the documents have been transcribed and organized, they can be donated (or kept by a family member that wants to retain them).
-3. Make the text easily available and searchable by family members, as well as AI (which may have different requirements).
-4. Ability create timelines or assemble the historical record of the family or specific individuals from across the complete document archive. Perhaps use AI to create the timelines in a more narrative form.
-
----
-
-## Source material
-1. **letters, cards, diaries** - handwritten; mostly stored in tubs, with little organization
-2. **books** - typed or typeset; mostly self-published books 50-100 pages in length. This may be expanded to include selected pages from other publications.
-3. **newspaper clippings, event programs, invitations, and other ephemera**
-
----
-
-## Methodology
-
-1. Follow current best practices per "A Guide to Documentary Editing" by Mary-Jo Kline. (See [Transcription Methodology](transcription_methodology.md))
-

docs/architecture.md

@@ -0,0 +1,169 @@
+# System Architecture (Version 4)
+
+This document defines the current Version 4 architecture baseline.
+
+## Architecture Objectives
+
+- Preserve durable archival records for Documents, Sources, People, and processing runs.
+- Execute page transcription asynchronously with bounded worker behavior.
+- Preserve append-only machine-attempt evidence with request/response provenance.
+- Keep UI, API, service, persistence, and provider boundaries explicit and testable.
+
+## Technical Stack
+
+- **Runtime:** Python 3.12+
+- **Web application:** FastAPI + NiceGUI
+- **Persistence:** SQLModel / SQLAlchemy (SQLite-first, PostgreSQL-compatible model design)
+- **Validation and settings:** Pydantic V2 + pydantic-settings
+- **Concurrency:** asyncio worker loop
+- **Provider integration:** OpenRouter adapter behind provider interface
+- **Quality and tests:** Ruff, ty, pytest, pytest-asyncio
+
+## Runtime Topology
+
+```mermaid
+flowchart LR
+U[Browser User] --> A[FastAPI + NiceGUI App]
+A --> W[Asyncio Worker]
+A --> DB[(SQLite/PostgreSQL Model)]
+W --> P[Provider Adapter]
+W --> DB
+```
+
+## Layered Boundaries
+
+### Interface Layer
+
+- `src/transcription/ui/**`
+- `src/transcription/api/**`
+
+Responsibilities:
+
+- Route registration, page orchestration, presentation adapters.
+- Structured user messaging through shared error presenter.
+- No direct persistence access from pages/components.
+
+### Service and Orchestration Layer
+
+- `src/transcription/services/documents.py`
+- `src/transcription/services/people.py`
+- `src/transcription/services/jobs.py`
+- `src/transcription/services/sources.py`
+- `src/transcription/services/evidence.py`
+- `src/transcription/services/store.py`
+- `src/transcription/services/workflows.py`
+
+Responsibilities:
+
+- Aggregate ownership and invariants.
+- Transaction-aware write helpers.
+- Cross-service workflows in orchestration modules (`store.py`, `workflows.py`).
+
+### Persistence Layer
+
+- `src/transcription/db/**`
+
+Responsibilities:
+
+- SQLModel definitions, async session/engine runtime, registry bootstrap.
+- Loader helpers that enforce explicit eager loading with `lazy="raise"` relationships.
+
+### Provider Layer
+
+- `src/transcription/providers/**`
+
+Responsibilities:
+
+- Provider API encapsulation.
+- Request manifest and transport evidence capture.
+- Normalized transcription result contract.
+
+## Core Domain Model
+
+- `Document` owns archival metadata and links to `Source`, `Job`, and `DocumentPerson`.
+- `Source` is a document page/file record with selected machine projection and human revision.
+- `Job` is an aggregate processing run with status and frozen prompt/runtime settings.
+- `JobSource` is queue/membership state for one `(job, source)` pair.
+- `ExecutionAttempt` is append-only evidence for each provider call.
+- `DocumentType` and `PersonRole` are UUID-backed registries with optional protected `semantic_key`.
+
+## Processing and Evidence Workflow
+
+1. User creates/updates Document metadata and linked People atomically through workflow orchestration.
+2. User creates a Job by uploading one or more Source files or by retranscribing an existing Source.
+3. Source files are validated and stored; orientation normalization may be applied at ingest, and stored bytes become the canonical processing bytes.
+4. Worker claims queued Job, transitions to `processing`, and processes pending pages in deterministic order.
+5. Each provider call writes one immutable `ExecutionAttempt` with:
+   - request manifest + hash
+   - transport evidence (when response exists)
+   - SDK snapshot and normalized metadata
+   - outcome, timing, and error details when applicable
+6. `JobSource` status is updated as queue/projection state; `Source.raw_transcription` is set on first successful attempt and can be explicitly re-pointed by candidate promotion.
+7. Job terminal status resolves to `transcribed`, `partial_success`, or `failed`.
+
+## Status Semantics
+
+- **Job statuses:** `queued`, `processing`, `transcribed`, `partial_success`, `failed`
+  - Operational success path resolves to `transcribed`.
+- **JobSource statuses:** `pending`, `transcribed`, `failed`, `cancelled`
+
+## Security and Path Handling Boundaries
+
+- Print media delivery uses record-validated API route:
+  - `src/transcription/api/print_api.py`
+- General UI media links resolve through:
+  - `src/transcription/ui/components/media_urls.py`
+- Local filesystem paths must never be accepted from user input as trusted media routes.
+
+## Concurrency and Reliability Principles
+
+- Worker loop reuses service bundle/provider resources for pooled calls.
+- Provider-call timeout is explicit and bounded.
+- Non-retriable worker-loop faults are surfaced and stop loop spin.
+- Per-page outcomes are durably persisted before processing next page.
+
+## Design Decisions and Rationale
+
+### Why `transcribed` is the success terminal state
+
+- The worker and job orchestration resolve successful completion to `JobStatus.TRANSCRIBED`, with mixed and failure outcomes represented by `partial_success` and `failed`.
+- This keeps terminal status vocabulary aligned with what the pipeline actually produces: transcribed page content and evidence, not a generic completion marker.
+
+### Why evidence history is append-only while page text is a projection
+
+- `ExecutionAttempt` stores immutable per-call evidence and preserves full attempt history across retries.
+- `Source.raw_transcription` is intentionally a mutable projection so UI and exports can show a selected current machine text without mutating historical evidence.
+- This split keeps auditability and UX both first-class: history is durable, presentation is editable.
+
+### Why orchestration modules own cross-service workflows
+
+- Service modules do not import each other; aggregate ownership remains local to each service.
+- Multi-aggregate writes are coordinated in orchestration modules (`store.py`, `workflows.py`) so transaction boundaries are explicit and testable.
+- This avoids circular dependencies and keeps cross-cutting workflow logic centralized.
+
+### Why explicit eager loading is required
+
+- ORM relationships are configured with `lazy="raise"` in key paths, so code must request needed relationships up front.
+- This prevents hidden query behavior in UI/service code and makes read shape deterministic and reviewable.
+
+### Why canonical source bytes may be ingest-normalized
+
+- Ingest normalization can correct orientation before persistence so provider calls, evidence hashes, and rendered processing source are consistent.
+- The canonical stored bytes, digest, and size become the durable processing identity for that source.
+
+### Why media access uses controlled routes/helpers
+
+- Print/export media uses record-validated API endpoints to avoid direct filesystem path exposure.
+- General UI media URLs are generated through shared resolver helpers to keep path handling consistent and centralized.
+
+## Scope Boundary
+
+Current architecture rules live in `docs/*`.
+
+## Related References
+
+- [System Requirements](requirements.md)
+- [Data Model](schema.md)
+- [Error Handling Policy](error_handling.md)
+- [Error Handling invariant](../invariant/error_handling.md)
+- [AI evidence invariant](../invariant/ai_evidence_and_provenance.md)

docs/architecture_v2.md

@@ -1,136 +0,0 @@
-# System Architecture (Version 2)
-
-This document describes the V2 production architecture of the personal historical-document transcription system.
-
-## Architecture Objectives
-
-* Preserve source material as immutable transcribed text alongside page-level spatial AI metadata.
-* Support batching multi-image and folder uploads cleanly into sequential pages (`page_number`).
-* Leverage asynchronous worker pools (`asyncio`) for parallel single-image API execution bounded by rate limiters (`asyncio.Semaphore`).
-* Migrate persistence to PostgreSQL using native `UUID`, `TIMESTAMPTZ`, and `JSONB` document storage.
-* Standardize all data validation, API parsing, and database models on **Pydantic V2**.
-* Support rich historical attribution (multi-author and multi-recipient relationships).
-
-## Runtime Topology
-
-The V2 runtime operates as an asynchronous Python application:
-
-* FastAPI + NiceGUI web application process.
-* In-process `asyncio` background task orchestrator for parallel API execution.
-* Relational persistence via PostgreSQL (using `asyncpg` or `psycopg3`).
-* Pydantic V2 validation layer wrapping API payloads and PostgreSQL `JSONB` schemas.
-
-```mermaid
-flowchart LR
-U[Browser User] --> A[FastAPI + NiceGUI App]
-A --> W[Asyncio Worker Engine]
-A --> DB[(PostgreSQL Database)]
-W --> P[Vision Provider APIs\nOpenAI / Claude]
-W --> DB
-```
-
-## Lifecycle Ownership
-
-Application lifespan owns runtime setup/teardown:
-
-* Initialize environment logging and Pydantic configuration.
-* Manage asynchronous PostgreSQL connection pools (`asyncpg` / `psycopg3`).
-* Execute database migrations and index initialization.
-* Recover stale processing jobs on startup.
-* Manage graceful shutdown of active `asyncio` worker pools.
-
-## Layered Module Structure
-
-### Interface Layer
-
-* `src/transcription/ui/**` (NiceGUI pages, multi-page renderers, person cards)
-* `src/transcription/api/**` (FastAPI routes and JSON error handlers)
-
-### Application & Async Worker Layer
-
-* `src/transcription/services/workflows.py`
-* `src/transcription/worker.py`
-
-Responsibilities:
-
-* Batch orchestration and status transitions (`queued` -> `processing` -> `completed` | `partial_success` | `failed`).
-* Parallel single-image API execution using `asyncio.gather` bounded by `asyncio.Semaphore`.
-* Pydantic schema parsing (`PageAIMetadata`) and validation prior to database storage.
-
-### Domain & Service Layer
-
-* `src/transcription/models/*.py` (Pydantic V2 schemas and entity definitions)
-* `src/transcription/services/*.py` (Transactional operations for `Document`, `Person`, `Source`, `Job`, and `JobSource`)
-
-### Infrastructure Layer
-
-* `src/transcription/db/**` (PostgreSQL connection pooling and raw parameterized SQL execution)
-* `src/transcription/providers/**` (OpenAI & Anthropic Vision SDK adapters)
-
-## Processing Workflow
-
-1. User uploads a folder or batch of images for a `Document`.
-2. System creates `Document`, `Job(status='queued')`, and ordered `Source` pages (`page_number = 1..N`).
-3. Worker claims job, sets `Job.status = 'processing'`, and spawns parallel `asyncio` tasks bounded by semaphore.
-4. Each task calls Vision API for a **single** `Source` image.
-5. On task completion:
-* Writes a `JobSource` record containing `status='transcribed'`, `raw_transcription`, `ai_metadata` (bounding boxes/confidence), and `raw_api_response`.
-* Caches active text to `Source.raw_transcription`.
-
-
-6. On page failure:
-* Writes `JobSource` record with `status='failed'` and `error_detail`.
-
-
-7. Once all page tasks resolve:
-* Marks `Job.status` as `completed` (100% success), `partial_success` (at least 1 success, 1 failure), or `failed` (all failed).
-
-
-
-## Domain Ownership & Invariants
-
-* **Immutable AI Outputs:** `source.raw_transcription` and `job_source.raw_transcription` store original, point-in-time machine output and are immutable.
-* **Inlined Revisions:** Human corrections occur on `source.revised_text`. UI renders `COALESCE(revised_text, raw_transcription)`.
-* **Sequential Integrity:** Multi-page documents are strictly ordered by `source.page_number ASC`.
-* **Page Execution Isolation:** A failure on one page image does not invalidate successful transcriptions on sister pages in the same batch job.
-
-## Data Model Summary
-
-* `Document` has many `Source` pages, many `Job` runs, and many `Person` records via `DocumentPerson` junction (`author` or `recipient`).
-* `Source` belongs to one `Document` and can be processed across many `JobSource` executions.
-* `Job` has many `JobSource` execution records.
-
-## Test Strategy
-
-* Unit tests for Pydantic V2 schemas, custom validators, and JSONB serialization.
-* Integration tests for async PostgreSQL connection handling and parameterized queries.
-* Async workflow tests using mock AI providers to verify `partial_success` and retry logic.
-* UI integration tests for multi-page rendering and person management.
-
----
-
-## Technology References
-
-- [FastAPI documentation](https://fastapi.tiangolo.com/)
-- [NiceGUI documentation](https://nicegui.io/documentation)
-- [PostgreSQL documentation](https://www.postgresql.org/docs/)
-- [Python asyncio](https://docs.python.org/3/library/asyncio.html#module-asyncio)
-- [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)
-- [Pydantic AI](https://pydantic.dev/docs/ai/overview/)
- 
-
-## Related Local References
-
-- [System Overview](index_v1.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- System Architecture (this document)
-- [System Requirements](requirements_v1.md)
-- [Data model](schema_v1.md)
-- [Error Handling Policy](error_handling_v1.md)
-- [Implementation Plan](implementation_plan_v1.md)
-
-
-
-
-

docs/data_migration.md

@@ -0,0 +1,57 @@
+# Database Rebuild Migration Workflow
+
+This project uses an explicit **export/import rebuild workflow** for schema migration.
+
+Policy:
+- Do not add runtime legacy-compatibility write paths.
+- Rebuild a fresh target database from current models.
+- Export current data/media, then import into the fresh target.
+
+## Commands
+
+### 1) Export current DB + uploads into a bundle
+
+```bash
+uv run python tools/export_import_migration.py export --bundle-dir .migration-bundle
+```
+
+Optional source overrides:
+- `--source-db <path-or-sqlalchemy-url>`
+- `--source-upload-dir <path>`
+
+### 2) Import bundle into a fresh DB + uploads root
+
+```bash
+uv run python tools/export_import_migration.py import --bundle-dir .migration-bundle --target-db .\data\transcription-new.db --target-upload-dir .\data-new
+```
+
+### 3) One-shot export+import
+
+```bash
+uv run python tools/export_import_migration.py migrate --bundle-dir .migration-bundle --target-db .\data\transcription-new.db --target-upload-dir .\data-new
+```
+
+## What gets migrated
+
+- Tables (in dependency order): `document_type`, `person_role`, `tag`, `document`, `person`, `photo`, `document_person`, `document_tag`, `job`, `source`, `job_source`, `execution_attempt`.
+- Media tree under `UPLOAD_DIR`.
+
+The bundle contains:
+- `database.json` (row export)
+- `uploads/` (copied media files)
+
+Path normalization during export/import:
+- `source.file_path` is normalized to `documents/...` (upload-root-relative POSIX).
+- `photo.path` is normalized to `photos/...` (upload-root-relative POSIX).
+
+Legacy V4.x portrait/homepage backfill in the export step:
+- If the source DB has no `photo` table, the exporter synthesizes `photo` rows from legacy `person.portrait_path` values and from legacy homepage image files under `UPLOAD_DIR/homepage`.
+- Legacy portrait and homepage image files are copied into the unified `UPLOAD_DIR/photos/{photo_id}{suffix}` layout in the migration bundle.
+- Legacy homepage markdown is relocated from `UPLOAD_DIR/homepage/homepage.md` to `UPLOAD_DIR/homepage.md`.
+
+## Cutover
+
+After importing to a fresh target:
+1. Stop the app.
+2. Point `DATABASE__*` and `UPLOAD_DIR` to the new targets.
+3. Start the app and run smoke checks (`/healthz`, create/upload/process one job).

docs/ddl_v2.sql

@@ -1,102 +0,0 @@
-## PostgreSQL DDL Specification (Version 2)
-
-```sql
--- Enable pgcrypto for UUID generation if on PostgreSQL < 13
-CREATE EXTENSION IF NOT EXISTS "pgcrypto";
-
--- 1. PERSON TABLE
-CREATE TABLE person (
-    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-    full_name TEXT NOT NULL,
-    display_name TEXT,
-    maiden_name TEXT,
-    birth_date DATE,
-    birth_date_raw TEXT,
-    birth_place TEXT,
-    death_date DATE,
-    death_date_raw TEXT,
-    death_place TEXT,
-    biography TEXT,
-    portrait_path TEXT,
-    metadata JSONB DEFAULT '{}'::jsonb,
-    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-    updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
-);
-
--- 2. DOCUMENT TABLE
-CREATE TABLE document (
-    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-    name TEXT NOT NULL,
-    document_type TEXT,
-    document_date DATE,
-    document_date_raw TEXT,
-    location_created TEXT,
-    notes TEXT,
-    archive_identifier TEXT,
-    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-    updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
-);
-
--- 3. DOCUMENT_PERSON (Junction Table for Multi-Author / Multi-Recipient)
-CREATE TABLE document_person (
-    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-    document_id UUID NOT NULL REFERENCES document(id) ON DELETE CASCADE,
-    person_id UUID NOT NULL REFERENCES person(id) ON DELETE CASCADE,
-    role VARCHAR(20) NOT NULL, -- 'author' or 'recipient'
-    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-    CONSTRAINT unique_document_person_role UNIQUE (document_id, person_id, role)
-);
-
--- 4. JOB TABLE (Batch-level orchestrator)
-CREATE TABLE job (
-    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-    document_id UUID NOT NULL REFERENCES document(id) ON DELETE CASCADE,
-    status VARCHAR(50) NOT NULL DEFAULT 'queued', -- 'queued', 'processing', 'completed', 'partial_success', 'failed'
-    retry_count INTEGER NOT NULL DEFAULT 0,
-    provider TEXT NOT NULL,                        -- e.g., 'openai', 'anthropic'
-    model TEXT NOT NULL,                           -- e.g., 'gpt-4o', 'claude-3-5-sonnet'
-    prompt_name TEXT,
-    date_created TIMESTAMPTZ NOT NULL DEFAULT now(),
-    date_updated TIMESTAMPTZ NOT NULL DEFAULT now()
-);
-
--- 5. SOURCE TABLE (Physical image files & active state)
-CREATE TABLE source (
-    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-    document_id UUID NOT NULL REFERENCES document(id) ON DELETE CASCADE,
-    page_number INTEGER NOT NULL DEFAULT 1,
-    upload_name TEXT NOT NULL,
-    filename TEXT NOT NULL,
-    file_path TEXT NOT NULL,
-    raw_transcription TEXT,                       -- Cached active AI text output
-    revised_text TEXT,                            -- Active human edited text
-    date_uploaded TIMESTAMPTZ NOT NULL DEFAULT now(),
-    date_revised TIMESTAMPTZ
-);
-
--- 6. JOB_SOURCE (Junction Table: Per-Image Execution Record)
-CREATE TABLE job_source (
-    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-    job_id UUID NOT NULL REFERENCES job(id) ON DELETE CASCADE,
-    source_id UUID NOT NULL REFERENCES source(id) ON DELETE CASCADE,
-    status VARCHAR(50) NOT NULL DEFAULT 'pending', -- 'pending', 'transcribed', 'failed'
-    raw_transcription TEXT,                       -- Point-in-time raw AI text output
-    ai_metadata JSONB,                            -- Page-level bounding boxes, tokens, confidence
-    raw_api_response JSONB,                       -- Complete REST response envelope
-    error_detail TEXT,
-    executed_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-    CONSTRAINT unique_job_source UNIQUE (job_id, source_id)
-);
-
--- INDEXES FOR FAST LOOKUPS & QUERY PERFORMANCE
-CREATE INDEX idx_person_full_name ON person(full_name);
-CREATE INDEX idx_document_date ON document(document_date);
-CREATE INDEX idx_document_person_doc ON document_person(document_id);
-CREATE INDEX idx_document_person_per ON document_person(person_id);
-CREATE INDEX idx_source_document ON source(document_id);
-CREATE INDEX idx_source_page_order ON source(document_id, page_number);
-CREATE INDEX idx_job_document ON job(document_id);
-CREATE INDEX idx_job_source_job ON job_source(job_id);
-CREATE INDEX idx_job_source_source ON job_source(source_id);
-CREATE INDEX idx_job_source_ai_metadata ON job_source USING GIN (ai_metadata);  
-```

docs/error_handling.md

@@ -0,0 +1,118 @@
+# Error Handling Policy (Version 4)
+
+This policy defines the active Version 4 error taxonomy, translation boundaries, and retry semantics.
+
+## Error Categories
+
+| Category | Meaning | Typical Origin | User Treatment |
+| :--- | :--- | :--- | :--- |
+| `validation` | Input payload/selection is invalid | UI form parsing, service validators | Inline correction guidance |
+| `not_found` | Target record is missing | ID lookup in service layer | Non-blocking warning or redirect |
+| `conflict` | State prevents requested action | lifecycle transitions, duplicate semantic keys | Explain required precondition |
+| `external` | Provider/network dependency failure | OpenRouter/provider adapter | Retry path and evidence retained |
+| `timeout` | Provider call exceeded configured bound | worker/provider client timeout | Retry path and bounded messaging |
+| `internal` | Unexpected local failure | unhandled service/runtime faults | Safe generic message + diagnostics capture |
+
+## Runtime Taxonomy and Canonical Mapping
+
+Runtime code uses a richer internal taxonomy for diagnostics and persisted evidence, then maps that
+taxonomy to the six canonical categories at the API/UI envelope boundary.
+
+### Internal runtime categories
+
+- `validation_error`
+- `user_input_error`
+- `not_found_error`
+- `conflict_error`
+- `external_provider_error`
+- `external_timeout_error`
+- `processing_error`
+- `infrastructure_transient_error`
+- `infrastructure_persistent_error`
+- `internal_unexpected_error`
+
+### Internal -> Canonical mapping
+
+| Internal category | Canonical envelope category |
+| :--- | :--- |
+| `validation_error` | `validation` |
+| `user_input_error` | `validation` |
+| `not_found_error` | `not_found` |
+| `conflict_error` | `conflict` |
+| `external_provider_error` | `external` |
+| `external_timeout_error` | `timeout` |
+| `infrastructure_transient_error` | `timeout` |
+| `processing_error` | `internal` |
+| `infrastructure_persistent_error` | `internal` |
+| `internal_unexpected_error` | `internal` |
+
+`ExecutionAttempt.error_category` stores the internal category value so diagnostics remain specific.
+
+## Translation Boundaries
+
+- **Provider layer:** raise provider-scoped exceptions with provider context; do not emit UI text.
+- **Service layer:** map raw exceptions into internal categories and preserve causal chain.
+- **UI/API layer:** convert internal categories to canonical categories using the centralized mapping.
+
+## Decision Context
+
+### Why taxonomy is category-based (not exception-class-based)
+
+- Categories encode operator-facing recovery semantics (fix input, retry later, investigate internal failure) independent of low-level exception type.
+- This keeps retry and messaging behavior consistent even when provider/client libraries change.
+
+### Why page-level failure is isolated
+
+- Multi-page archival documents often contain a mix of readable and degraded pages.
+- Isolating failures to page scope preserves successful results and avoids all-or-nothing loss when one page fails.
+- Aggregate job status then communicates overall outcome (`transcribed`, `partial_success`, `failed`) without hiding page detail.
+
+### Why retries append evidence instead of mutating rows
+
+- Retry operations are new observations, not corrections of history.
+- Appending attempts preserves forensic traceability, timing history, and provider variability analysis.
+- Projection updates remain explicit user/workflow decisions, separate from immutable evidence.
+
+## Job and Page Failure Semantics
+
+### Page-Level (`JobSource`)
+
+- `pending` -> `transcribed` when attempt succeeds.
+- `pending` -> `failed` when attempt fails terminally.
+- `pending` -> `cancelled` on job cancellation before processing.
+
+### Job-Level (`Job`)
+
+- `transcribed` when all pages transcribe successfully.
+- `partial_success` when mixed success/failure outcomes exist.
+- `failed` when no page transcribes successfully.
+
+## Retry and Retranscription Rules
+
+1. Failed/cancelled pages may be re-queued through retranscription workflows.
+2. Retry attempts must append new `ExecutionAttempt` rows; prior evidence remains immutable.
+3. Selecting a better candidate must update projection pointers, not mutate historical attempt rows.
+
+## Logging and Diagnostics Rules
+
+1. Persist sufficient attempt error metadata (`error_category`, `error_message`, transport evidence) for post-hoc analysis.
+2. Avoid leaking stack traces or local paths into user-facing message envelopes.
+3. Preserve causal exception chains for internal diagnostics.
+
+## Operator Recovery Guidance
+
+- **validation/conflict:** correct input or state and retry manually.
+- **external/timeout:** allow bounded retries and keep prior attempt evidence visible.
+- **internal:** stop automatic retries, surface a safe message, and inspect diagnostics with correlation context.
+
+## UI Messaging Contract
+
+- User-visible errors must be actionable, bounded, and category-consistent.
+- Multi-page jobs must show partial outcomes instead of collapsing into a single opaque failure.
+- Recovery actions (`retry`, `retranscribe`, `edit input`) must be offered where available.
+
+## Cross-Reference
+
+- [Error Handling invariant](../invariant/error_handling.md)
+- [System Requirements](requirements.md)
+- [Data Model](schema.md)

docs/error_handling_v2.md

@@ -1,88 +0,0 @@
-# Error Handling Policy (Version 2)
-
-This document defines the canonical error-handling policy for the V2 document transcription system.
-
-## Error Handling Objectives
-
-* Make failures visible in clear, actionable language at both the document and individual page levels.
-* Support **isolated failure handling** in multi-image batches so single page errors do not crash an entire batch job.
-* Preserve diagnostic detail (Pydantic validation errors, raw provider responses) in PostgreSQL `JSONB` for fast troubleshooting.
-* Ensure consistent error envelope structure across API, UI, and async worker boundaries.
-
-## Scope And Authority
-
-Governs error behavior across NiceGUI pages, FastAPI routes, service orchestration, `asyncio` background tasks, PostgreSQL interactions, and AI provider adapters.
-
-## Error Taxonomy
-
-| Category | Definition | Retriable |
-| --- | --- | --- |
-| `validation_error` | Pydantic payload or parameter schema validation failure | no |
-| `user_input_error` | Unacceptable user file (unsupported image type, corrupt file) | no |
-| `not_found_error` | Requested resource (`Document`, `Source`, `Person`, `Job`) missing | no |
-| `conflict_error` | Operation violates state constraints (e.g., duplicate `document_person` role) | no |
-| `external_provider_error` | AI Provider API failure (rate limit, vision execution error) | yes |
-| `infrastructure_transient_error` | Temporary DB connection reset or HTTP timeout | yes |
-| `infrastructure_persistent_error` | Database down, missing API credentials, misconfiguration | no |
-| `internal_unexpected_error` | Uncaught Python exception or logic defect | no |
-
-## Async Batch & Page-Level Error Behavior
-
-In multi-image `asyncio` batch processing:
-
-1. **Page Isolation:** Exceptions caught during individual page calls are caught within the `asyncio` task wrapper.
-2. **Page Record Logging:** Page failure detail is written directly to `job_source.error_detail` and `job_source.status = 'failed'`.
-3. **Batch Aggregate State:**
-* If **all** page tasks succeed -> `job.status = 'completed'`.
-* If **some** page tasks fail -> `job.status = 'partial_success'`.
-* If **all** page tasks fail -> `job.status = 'failed'`.
-
-
-4. **Retry Strategy:** The UI exposes a "Retry Failed Pages" option for `partial_success` jobs, which spawns a new targeted `Job` containing *only* the `Source` IDs marked as `failed`.
-
-## API Error Response Contract
-
-API error responses return a structured JSON envelope:
-```json
-{
-"error_id": "err_uuid_12345",
-"category": "validation_error",
-"message": "The uploaded payload failed schema validation.",
-"suggestion": "Check file format and metadata fields, then try again.",
-"details": {
-"pydantic_errors": [...]
-},
-"timestamp": "2026-07-31T07:55:00Z"
-}
-```
-
-HTTP Status Mappings:
-
-* `validation_error`, `user_input_error` -> `400`
-* `not_found_error` -> `404`
-* `conflict_error` -> `409`
-* `external_provider_error` -> `502` / `503`
-* `infrastructure_transient_error` -> `503`
-* `infrastructure_persistent_error`, `internal_unexpected_error` -> `500`
-
----
-
-## Technology References
-
-- [FastAPI documentation](https://fastapi.tiangolo.com/)
-- [NiceGUI documentation](https://nicegui.io/documentation)
-- [PostgreSQL documentation](https://www.postgresql.org/docs/)
-- [Python asyncio](https://docs.python.org/3/library/asyncio.html#module-asyncio)
-- [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)
-- [Pydantic AI](https://pydantic.dev/docs/ai/overview/)
-
-## Related Local References
-
-- [System Overview](index_v2.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v2.md)
-- [System Requirements](requirements_v2.md)
-- [Data model](schema_v2.md)
-- Error Handling Policy (this document)
-- [Implementation Plan](implementation_plan_v2.md)

docs/implementation_plan_v2.md

@@ -1,248 +0,0 @@
-# Implementation Plan (Version 2)
-
-This plan defines the path from the V1 baseline to **Version 2 complete**, aligned to the updated multi-image and multi-person relational domain model:
-
-* `Document` acts as a logical parent container for physical artifacts, supporting multi-author and multi-recipient relationships via `DocumentPerson`.
-* `Source` represents an individual image page within a document, maintaining sequential order (`page_number`), cached active machine output (`raw_transcription`), and inline single user revisions (`revised_text`).
-* `Job` acts as an overarching batch orchestrator for multi-page async processing tasks.
-* `JobSource` records individual point-in-time API executions per image page, storing Pydantic-validated `ai_metadata` and raw REST envelopes (`raw_api_response`).
-* **Pydantic V2** acts as the single source of truth for runtime validation, API payload parsing, and PostgreSQL JSONB serialization.
-
-The objective is to complete the V2 scope with production readiness while keeping non-V2 enhancements out of active delivery.
-
----
-
-## V2 Completion Definition
-
-V2 is complete when all of the following are true:
-
-1. **Functional complete**
-* Multi-image and whole-folder uploads assign sequential page numbers to `Source` records under a single `Document`.
-* Batch jobs process pages concurrently using an `asyncio` worker pool with semaphore rate limiting.
-* Partial job failures resolve cleanly to `partial_success`, allowing single-page retries without re-running successful pages.
-* Multi-author and multi-recipient tagging is supported on `Document`.
-
-
-2. **Data-model complete**
-* SQLite is fully replaced with PostgreSQL (using `asyncpg` or `psycopg3`).
-* Pydantic V2 models validate all API payloads, database row mappings, and `JSONB` structures.
-
-
-3. **Operational complete**
-* Concurrency controls, worker pool metrics, and database connections operate safely under batch load.
-
-
-4. **Documentation complete**
-* `schema_v2.md`, `DDL_v2.sql`, Pydantic model contracts are updated and consistent.
-
-
-
----
-
-## Phase 1 — Data Contract Stabilization & Pydantic Baseline
-
-**Goal:** Lock the PostgreSQL schema, DDL, and Pydantic V2 models before refactoring service logic.
-
-### Tasks
-
-1. Finalize DDL for PostgreSQL native types (`UUID`, `TIMESTAMPTZ`, `JSONB`) and junction tables (`document_person`, `job_source`).
-2. Build core Pydantic V2 schemas (`Person`, `Document`, `Source`, `Job`, `JobSource`, `PageAIMetadata`).
-3. Confirm and document data invariants:
-* `source.raw_transcription` and `job_source.raw_transcription` are immutable machine outputs.
-* `source.revised_text` holds user edits. UI renders `COALESCE(revised_text, raw_transcription)`.
-* Page sequence is strictly ordered by `source.page_number ASC`.
-
-
-4. Freeze V2 job status values (`queued`, `processing`, `completed`, `partial_success`, `failed`) and page execution status values (`pending`, `transcribed`, `failed`).
-
-### Deliverables
-
-* Canonical `docs/schema_v2.md` and `docs/DDL_v2.sql`.
-* Centralized Pydantic validation suite in `models/schemas_v2.py`.
-
-### Exit Criteria
-
-* All database tables, relationships, and JSONB structures have corresponding Pydantic V2 models passing unit validation tests.
-
----
-
-## Phase 2 — Persistence Layer Transition (SQLite to PostgreSQL)
-
-**Goal:** Replace the SQLite storage layer with an asynchronous PostgreSQL driver (`asyncpg` or `psycopg3`).
-
-### Tasks
-
-1. Configure PostgreSQL database connection pooling and environment configuration.
-2. Refactor `services/store.py` / repository layers to execute parameterized async SQL queries (`$1`, `$2`).
-3. Implement JSONB serialization and deserialization helpers using Pydantic's `.model_dump_json()` and `.model_validate()`.
-4. Implement database bootstrap routines for PostgreSQL table creation and index initialization.
-
-### Deliverables
-
-* PostgreSQL-native database connection and query service modules.
-* Integration test suite confirming connection pooling and JSONB CRUD operations.
-
-### Exit Criteria
-
-* All database reads/writes run asynchronously against PostgreSQL with zero remaining SQLite driver dependencies.
-
----
-
-## Phase 3 — Service Layer & `asyncio` Engine Refactor
-
-**Goal:** Implement batch orchestration and parallel single-image API execution.
-
-### Tasks
-
-1. Refactor upload service to process folder/multi-image input:
-* Group files into a single `Document`.
-* Create ordered `Source` rows (`page_number = 1..N`).
-
-
-2. Refactor `services/workflows.py` with `asyncio` worker pools:
-* Use `asyncio.Semaphore` to enforce API provider rate limits.
-* Issue parallel single-image requests to Vision APIs (OpenAI/Claude).
-* Parse API responses directly into Pydantic models (`PageAIMetadata`).
-
-
-3. Update execution tracking:
-* Create a `JobSource` row per page call to record `raw_transcription`, `ai_metadata`, and `raw_api_response`.
-* Update active `source.raw_transcription` upon task completion.
-* Calculate aggregate batch status (`completed`, `partial_success`, `failed`) on the parent `Job`.
-
-
-4. Refactor `services/person.py` and `services/documents.py` to handle multi-person roles via `document_person`.
-
-### Deliverables
-
-* Asynchronous batch execution engine in `services/workflows.py`.
-* Service routines for multi-person tagging and page-level retries.
-
-### Exit Criteria
-
-* Executing a folder upload of 10+ images processes concurrently, populates page-level `JobSource` entries, and handles partial worker errors without crashing the batch.
-
----
-
-## Phase 4 — UI & API Contract Alignment
-
-**Goal:** Update API endpoints and frontend/UI views to render multi-page documents and person roles.
-
-### Tasks
-
-1. Update document and job API endpoints to accept batch file arrays and multi-person ID payloads.
-2. Update UI document views:
-* Render multi-page document transcriptions sequentially by `page_number`.
-* Display author and recipient chips/cards linked from `document_person`.
-
-
-3. Update job detail UI to show page-level execution statuses (`transcribed` vs. `failed`) and provide a "Retry Failed Pages" action for `partial_success` jobs.
-4. Align inline page editing controls to update `source.revised_text` and `source.date_revised`.
-
-### Deliverables
-
-* Refactored API routes and UI components supporting multi-page rendering and person management.
-
-### Exit Criteria
-
-* UI successfully displays multi-page document text, allows per-page human revisions, and shows author/recipient metadata.
-
----
-
-## Phase 5 — Test Suite Realignment & Concurrency Testing
-
-**Goal:** Ensure end-to-end system stability under concurrent async execution and load.
-
-### Tasks
-
-1. Write unit tests for Pydantic models, custom validators, and JSONB conversions.
-2. Write integration tests for async database operations:
-* CRUD for `Document`, `Person`, `DocumentPerson`, `Source`, `Job`, and `JobSource`.
-
-
-3. Write mock-backed async workflow tests:
-* Verify `asyncio.Semaphore` bounds concurrent tasks properly.
-* Validate state transition logic for `completed`, `partial_success`, and `failed` jobs.
-* Confirm retry routines process only targeted `JobSource` records marked as `failed`.
-
-
-4. Re-enable CI quality gates (linting, type checking with Pyright/mypy, pytest).
-
-### Deliverables
-
-* Passing asynchronous test suite covering core workflows, edge cases, and failure recoveries.
-
-### Exit Criteria
-
-* CI pipeline is green with comprehensive coverage across database operations, Pydantic models, and worker queues.
-
----
-
-## Phase 6 — Reliability, Operations, and Release Readiness
-
-**Goal:** Prepare V2 for production deployment and operator management.
-
-### Tasks
-
-1. Verify structured logging includes `job_id`, `document_id`, `source_id`, and `person_id`.
-2. Tune PostgreSQL connection pool limits and `asyncio` concurrency thresholds for production infrastructure.
-3. Update operational documentation:
-* Review and update `docs/schema_v2.md` as needed.
-* Create `docs/runbook_v2.md` detailing PostgreSQL maintenance, JSONB index management, and worker queue monitoring.
-* Create `docs/release_checklist_v2.md` for launch sign-off.
-
-
-
-### Deliverables
-
-* Updated project documentation and operational runbooks.
-* V2 release sign-off checklist.
-
-### Exit Criteria
-
-* All documentation reflects V2 architecture; launch checklist is fully verified.
-
----
-
-## Requirement Traceability Focus
-
-Maintain evidence against these V2 requirement groups:
-
-* **Batch & Multi-Image Pipeline:** Folder ingestion, page ordering, async worker execution.
-* **Database & Persistence:** PostgreSQL, native UUIDs, JSONB execution storage, `asyncpg` pooling.
-* **Validation & Schemas:** Pydantic V2 models for DB rows, API requests, and AI vision responses.
-* **Attribution & Metadata:** Multi-author and multi-recipient tagging, biographical entity management.
-* **Error Recovery:** Partial success states, page-level status flags, isolated retry execution.
-
----
-
-## Scope Discipline Rule (V2 Focus)
-
-* Only tasks required for V2 scope (PostgreSQL, Pydantic V2, folder/async processing, multi-person roles) enter this plan.
-* V3 candidate features (such as side-by-side multi-provider model output comparison) remain strictly in the future backlog.
-* Any schema adjustments during implementation require immediate updates to `DDL_v2.sql`, Pydantic models, and `schema_v2.md`.
-
----
-
-## Technology References
-
-- [FastAPI documentation](https://fastapi.tiangolo.com/)
-- [NiceGUI documentation](https://nicegui.io/documentation)
-- [PostgreSQL documentation](https://www.postgresql.org/docs/)
-- [Python asyncio](https://docs.python.org/3/library/asyncio.html#module-asyncio)
-- [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)
-- [Pydantic AI](https://pydantic.dev/docs/ai/overview/)
-
-## Related Local References
-
-- [System Overview](index_v2.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v2.md)
-- [System Requirements](requirements_v2.md)
-- [Data model](schema_v2.md)
-- [Error Handling Policy](error_handling_v2.md)
-- Implementation Plan (this document)
- 
-
-

docs/index.md

@@ -0,0 +1,23 @@
+# Document Transcription System Overview (Version 4)
+
+This directory is the single source of truth for current Version 4 behavior and architecture.
+
+## Canonical Reading Order
+
+1. [System Architecture](architecture.md) for runtime topology, boundaries, and lifecycle ownership.
+2. [System Requirements](requirements.md) for verifiable current-state requirements.
+3. [Data Model](schema.md) for entities, constraints, and evidence persistence rules.
+4. [Error Handling Policy](error_handling.md) for category, translation, and retry behavior.
+
+## Cross-Version Invariants
+
+- [Historical Document Transcription Design Intent](../invariant/intent.md)
+- [Transcription Methodology](../invariant/transcription_methodology.md)
+- [Error Handling](../invariant/error_handling.md)
+- [Digital Evidence and AI Processing Provenance](../invariant/ai_evidence_and_provenance.md)
+- [UI Style Guide](../invariant/ui_style_guide.md)
+
+## Baseline Statement
+
+The current Version 4 baseline includes behavior delivered through the architectural cleanup phases.
+Use this `docs/*` canonical set for active design and implementation decisions.

docs/index_v2.md

@@ -1,47 +0,0 @@
-# Document Transcription System Overview (Version 2)
-
-This project is a personal-scale application for transcribing, indexing, and preserving historical family documents, letters, postcards, and journals.
-
-## Start Here
-
-Read [architecture_v2.md](architecture_v2.md) first for technical overview and system design.
-
-## Core V2 Capabilities
-
-* **Folder & Multi-Image Ingestion:** Upload whole folders or image batches that map sequentially (`page_number`) under a single `Document`.
-* **Parallel Async AI Vision Engine:** Concurrently process single-page image transcriptions using Python `asyncio` bounded by rate limiters.
-* **Robust PostgreSQL Storage:** Relational storage for entities with native `UUID`, `TIMESTAMPTZ`, and `JSONB` for deep AI spatial metadata and raw envelopes.
-* **Pydantic V2 Validation:** End-to-end type safety, DB row mapping, and JSONB payload validation.
-* **Historical Person Management:** Track authors and recipients across documents with rich biographical entities (`Person`).
-* **Page-Level Execution Auditing & Revisions:** Store immutable point-in-time machine output per run while enabling inline human corrections (`revised_text`).
-* **Partial Failure Recovery:** Bounded batch execution that isolates single-page API errors (`partial_success`) for simple retries.
-
-## Technical Stack
-
-* **Application Web Framework:** FastAPI + NiceGUI
-* **Persistence Engine:** PostgreSQL 13+
-* **Data Validation & Schemas:** Pydantic V2
-* **Concurrency & Workers:** Python `asyncio` worker pool with `asyncio.Semaphore`
-* **Vision Providers:** OpenAI (GPT-4o) and Anthropic (Claude 3.5 Sonnet) via native SDKs
-
----
-
-## Technology References
-
-- [FastAPI documentation](https://fastapi.tiangolo.com/)
-- [NiceGUI documentation](https://nicegui.io/documentation)
-- [PostgreSQL documentation](https://www.postgresql.org/docs/)
-- [Python asyncio](https://docs.python.org/3/library/asyncio.html#module-asyncio)
-- [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)
-- [Pydantic AI](https://pydantic.dev/docs/ai/overview/)
- 
-## Documentation Index
-
-- System Overview (this document)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v2.md)
-- [System Requirements](requirements_v2.md)
-- [Data model](schema_v2.md)
-- [Error Handling Policy](error_handling_v2.md)
-- [Implementation Plan](implementation_plan_v2.md)

docs/invariant/ai_evidence_and_provenance.md

@@ -0,0 +1,141 @@
+# Digital Evidence and AI Processing Provenance (Invariant)
+
+## 1. Purpose
+
+This document defines non-negotiable evidence and provenance rules for the transcription application.
+
+The application exists to preserve historical source material and produce useful transcriptions without losing the ability to inspect, reinterpret, or reprocess the evidence later. Provider integrations, model names, schemas, and user interfaces may change; the principles below must remain true.
+
+## 2. Evidence Model
+
+The application distinguishes five kinds of information:
+
+1. **Source evidence**: the canonical stored media used for processing and the facts needed to identify and verify it.
+2. **Execution specification**: the frozen instructions, parameters, source identity, and software context for one processing attempt.
+3. **Transport evidence**: the response received at the application/provider boundary, including safe protocol metadata.
+4. **Normalized data**: selected fields extracted for search, display, accounting, and workflow behavior.
+5. **Derived artifacts**: outputs produced from source evidence, such as transcription text, OCR geometry, confidence data, layout analysis, or entity extraction.
+
+Normalized data and derived artifacts never replace source or transport evidence.
+
+## 3. Core Invariants
+
+### 3.1 Canonical Source Preservation
+
+1. Each source must have one canonical stored byte stream used for processing and provenance.
+2. Canonical storage may apply deterministic ingest normalization before persistence.
+3. Canonical stored bytes must have a cryptographic content digest, byte size, and stable identity.
+4. Post-ingest processing derivatives must not overwrite canonical stored bytes.
+5. Moving or renaming a stored file must not change its evidence identity.
+
+### 3.2 Append-Only Processing History
+
+1. Every processing attempt must have a distinct execution record, whether it succeeds, partially succeeds, times out, or fails.
+2. A later attempt must not overwrite the evidence from an earlier attempt.
+3. A convenient “latest transcription” value may be maintained as a cache or projection, but it is not the authoritative execution history.
+4. Human revisions must remain distinguishable from all machine-generated outputs.
+5. Reprocessing a source must create new evidence rather than rewriting historical evidence.
+
+### 3.3 Frozen Execution Specification
+
+Each execution must preserve enough information to understand what the application asked the processor to do:
+
+1. Requested provider, model, and provider-routing constraints.
+2. Full effective system and user instructions.
+3. Prompt asset name and content digest when a prompt asset is used.
+4. Every explicitly supplied generation or processing parameter.
+5. Whether an optional parameter was explicitly set or omitted.
+6. Canonical source digest (and derivative digests when used), media type, dimensions or page geometry when known, and page identity.
+7. A secret-safe representation of the request structure.
+8. Application, provider-adapter, and client-library versions sufficient to interpret the execution.
+
+The execution specification must not contain credentials, authorization headers, secret query values, or unnecessary duplicate source binaries.
+
+### 3.4 Evidence-Layer Terminology
+
+The following terms are not interchangeable:
+
+- **Transport response**: the status, safe headers, and exact response body received by the application at its HTTP boundary.
+- **Router-normalized response**: a response transformed by an intermediary into its common schema.
+- **SDK-parsed response**: an object created when a client library validates or filters a response.
+- **Normalized metadata**: application-selected fields derived from a response.
+- **Native provider response**: the upstream provider's own response before any intermediary transformation.
+
+The application and its documentation must identify which layer is stored. A response must not be described as “raw,” “complete,” or “native” without naming the boundary at which that claim is true.
+
+### 3.5 Transport Evidence
+
+1. Preserve the exact successful response body received at the application's transport boundary before SDK model parsing can discard unknown fields.
+2. Preserve the response status and an allowlisted set of non-secret headers needed for correlation, content interpretation, rate-limit diagnosis, or audit.
+3. Preserve provider/router request and generation identifiers when available.
+4. Preserve safe response evidence for unsuccessful calls when a response was received.
+5. Record explicitly when no response was received, such as a local timeout or connection failure.
+6. Retain parsed and normalized forms only as additional representations of the preserved response.
+
+Wire-level packet capture, TLS session data, credentials, and unrestricted headers are neither required nor permitted.
+
+These requirements apply to executions performed after transport capture is implemented. For earlier executions, the absence of transport evidence must be represented explicitly. An SDK snapshot or normalized record must never be relabeled or backfilled as transport evidence.
+
+### 3.6 Derived Artifact Provenance
+
+1. Every derived artifact must identify its source evidence and producing execution.
+2. Each artifact must declare its semantic type, media/serialization format, schema name and version, producer, producer version, and creation time.
+3. Artifact content must be stored directly or referenced by a stable path or object identifier and protected by a cryptographic digest.
+4. Coordinates must declare their coordinate system, units, origin, page/image dimensions, and transformation history.
+5. Confidence values must identify the producer and scope to which they apply; values from different producers must not be treated as directly comparable without validation.
+6. Provider-specific payloads may be retained, but durable application behavior must not depend on undocumented provider fields.
+
+This model must accommodate future OCR text, word or line polygons, layout regions, confidence data, alternate transcriptions, and structured extraction without adding a dedicated column for every possible feature.
+
+### 3.7 Integrity and Auditability
+
+1. Stored evidence must be exportable with enough identifiers and metadata to verify relationships and digests outside the application.
+2. Evidence mutation, deletion, and retention behavior must be explicit and testable.
+3. Schema upgrades must preserve existing evidence and its original meaning.
+4. Backfills must be identified as backfills; they must not imply that previously uncaptured evidence existed.
+5. Integrity verification must distinguish a missing file, digest mismatch, unavailable external artifact, and malformed metadata.
+
+### 3.8 Security and Privacy
+
+1. API keys, authorization headers, cookies, and credentials must never be persisted as provenance.
+2. Persist only headers and metadata fields that appear on an explicit allowlist of known-safe fields. Discard all other fields before storage; never persist an unrestricted capture and attempt to redact it afterward.
+3. Request manifests should reference source content by identity instead of duplicating base64 source data.
+4. Diagnostic displays and exports must avoid exposing secrets or machine-local details that are not necessary for evidence interpretation.
+
+## 4. Reproducibility Limits
+
+Provenance supports explanation, comparison, and best-effort reproduction; it does not guarantee identical output.
+
+Identical requests may produce different results because of model updates, provider routing, nondeterministic computation, undocumented defaults, safety systems, or retired endpoints. The application must preserve whether a parameter was omitted rather than pretending to know the provider default used at that time.
+
+Likewise, preserving a general vision-model response does not create OCR coordinates that were never returned. Future coordinate extraction remains possible because canonical source evidence is preserved and can be processed again by a suitable system.
+
+## 5. Model Evaluation Policy
+
+Model selection must be based on a representative sample of the actual archive rather than vendor claims alone.
+
+Evaluation should:
+
+1. Use manually reviewed reference transcriptions following the project's [Transcription Methodology](transcription_methodology.md).
+2. Represent printed, typed, handwritten, degraded, tabular, multilingual, and spatially complex material present in the archive.
+3. Measure character and word error rates where appropriate.
+4. Separately record silent corrections, invented text, omitted text, uncertainty handling, layout fidelity, cost, and latency.
+5. Preserve the exact model, endpoint or route, parameters, prompt, source digest, and scoring method for every comparison.
+6. Treat model rankings as corpus- and version-specific, not permanent declarations of a universal “best” model.
+
+Benchmark material containing family records remains private application data unless explicitly approved for publication.
+
+## 6. Ownership and Change Policy
+
+1. Canonical V4 architecture, schema, requirements, and error-policy documents define how current behavior satisfies this invariant.
+2. Provider adapters own the capture of provider-boundary evidence.
+3. Services own validation, persistence, retention, and export behavior.
+4. UI pages may inspect evidence through service contracts but do not define evidence semantics.
+5. If implementation conflicts with this invariant, either correct the implementation or explicitly revise this document before accepting the behavior.
+6. Revisions to this document require deliberate review because they change the long-term preservation contract.
+
+## 7. Related Invariants
+
+- [Historical Document Transcription Design Intent](intent.md)
+- [Transcription Methodology & Style Guide](transcription_methodology.md)
+- [UI Style Guide](ui_style_guide.md)

docs/invariant/error_handling.md

@@ -0,0 +1,101 @@
+# Error Handling (Invariant)
+
+## 1. Purpose
+
+This document defines the non-negotiable failure-handling principles for the transcription application.
+
+Error categories, API envelopes, status codes, framework integrations, and persistence fields may change between versions. Failures must nevertheless remain visible, safe, diagnosable, and consistent across every application boundary.
+
+## 2. Core Invariants
+
+### 2.1 Failures Are Visible
+
+1. An operation must not report success when all or part of the requested work failed.
+2. Invalid input, unavailable dependencies, persistence failures, provider failures, and unexpected defects must be surfaced through the application's established error path.
+3. Code must not silently discard an exception, provider response, invalid value, or failed state transition.
+4. When work can partially succeed, the successful and failed portions must be identified separately.
+
+### 2.2 Messages Are Actionable
+
+1. Operator-facing errors must explain what failed in concise language.
+2. When a safe corrective action is known, the error must state it.
+3. Expected validation or conflict failures must not be presented as unexplained internal defects.
+4. Internal diagnostics must not replace a usable operator-facing message.
+
+### 2.3 Errors Have Stable Identity and Classification
+
+1. Every surfaced failure must have a stable correlation identifier or equivalent trace identity.
+2. Failures must be classified into a documented, machine-readable category.
+3. Boundary-specific representations must preserve the original category and correlation identity.
+4. Unknown exceptions must be converted at an explicit boundary, retain their causal chain for diagnostics, and be classified as unexpected rather than disguised as an expected failure.
+
+### 2.4 Boundary Translation Is Consistent
+
+1. UI, API, service, worker, persistence, and provider boundaries must use one shared error model or deterministic translations between documented models.
+2. A boundary may simplify presentation, but it must not change the meaning, retryability, or identity of a failure.
+3. Domain and service code must not depend on UI notifications or HTTP response types.
+4. UI and API layers must not infer error categories by parsing message text.
+
+### 2.5 State Changes Are Safe
+
+1. A failed atomic operation must leave persisted state unchanged.
+2. Batch operations may preserve successful independent items only when partial success is an explicit part of the workflow contract.
+3. A failed item must retain enough state to identify what was attempted and whether retry is safe.
+4. Error handling must not overwrite earlier successful results or historical execution evidence.
+
+### 2.6 Retry Is Explicit and Bounded
+
+1. Validation, authorization, policy, conflict, and other deterministic failures must not be retried automatically without a relevant input or state change.
+2. Automatic retry is permitted only for failures classified as transient and only when the operation is idempotent or otherwise protected from duplicate effects.
+3. Retry count, delay, and terminal behavior must be bounded and observable.
+4. Exhausted retries must end in a visible terminal failure rather than an indefinitely pending state.
+
+### 2.7 Diagnostics Are Preserved Safely
+
+1. Logs and persisted diagnostic evidence must retain enough context to correlate the failure with the affected operation and record.
+2. Provider and infrastructure failures must preserve safe diagnostic evidence at the boundary where it is available.
+3. Credentials, authorization headers, cookies, secret values, and unnecessary personal data must not appear in errors, logs, notifications, or exports.
+4. Diagnostic metadata capture must use explicit safe-field allowlists where unrestricted content could contain secrets.
+5. User-facing messages must not expose stack traces, local filesystem details, database credentials, or raw internal exceptions.
+
+AI execution failures also follow the evidence rules in [Digital Evidence and AI Processing Provenance](ai_evidence_and_provenance.md).
+
+### 2.8 Cancellation and Timeout Are Distinct Outcomes
+
+1. User cancellation, application shutdown, local timeout, remote timeout, and provider rejection must remain distinguishable.
+2. Cancellation must not be converted into success or a generic unexpected error.
+3. Timeout handling must identify whether a provider response was received when that fact is known.
+4. Cleanup after cancellation or timeout must preserve consistency and must not conceal a completed side effect.
+
+### 2.9 Logging Must Support Audit Without Becoming the Record
+
+1. Structured logs must include correlation identity, operation, category, and relevant non-secret record identifiers.
+2. Expected operator errors may be logged less severely than unexpected defects, but they must remain observable.
+3. Logs are operational diagnostics and do not replace required database state or archival evidence.
+4. Duplicate logging of the same failure at every layer should be avoided; ownership of the authoritative log event must be clear.
+
+## 3. Verification Policy
+
+Each version must verify:
+
+1. Every documented error category reaches the intended UI and API representation.
+2. Failed atomic writes roll back completely.
+3. Partial-success workflows preserve successful independent results and identify failed items.
+4. Retry behavior is bounded and restricted to eligible failures.
+5. Unexpected exceptions retain correlation and causal information without exposing sensitive details.
+6. Logs, persisted evidence, UI messages, and exports contain no credentials.
+7. Cancellation, timeout, provider response failure, and no-response failure remain distinguishable.
+
+## 4. Versioned Ownership
+
+1. Version-specific error taxonomies, envelopes, HTTP mappings, model fields, and framework behavior belong in the applicable version documentation.
+2. Each versioned error-handling document must state how it satisfies this invariant.
+3. A version may add stricter safeguards but must not weaken these principles without first revising this invariant deliberately.
+4. Implementation and tests must be updated together when a versioned error contract changes.
+
+## 5. Related Invariants
+
+- [Historical Document Transcription Design Intent](intent.md)
+- [Digital Evidence and AI Processing Provenance](ai_evidence_and_provenance.md)
+- [UI Style Guide](ui_style_guide.md)
+

docs/invariant/intent.md

@@ -0,0 +1,25 @@
+# Historical Document Transcription Design Intent
+I have several thousand pages of family history told through letters, postcards, books, and other documents that I want to transcribe to text. 
+
+---
+
+## Goals
+1. Preserve our family history
+2. Unburden my family (and descendants) from having to store and care for the physical media. Once the documents have been transcribed and organized, they can be donated (or kept by a family member that wants to retain and preserve them).
+3. Make the document text easily available and easily searchable.
+4. Ability create timelines for individuals and/or families through document dates or the data contained in them. Perhaps even use AI to generate biographies or family histories.
+
+---
+
+## Source material
+1. **letters, cards, diaries** - handwritten; mostly stored in boxes and tubs with little organization
+2. **books** - typed or typeset; mostly self-published books 50-100 pages in length. This may be expanded to include selected pages from other publications.
+3. **photos** - notes written on the backs of photos and the pages of photo albums
+4. **other ephemera** - newspaper clippings, event programs, invitations, military records, immigration records, etc
+
+---
+
+## Methodology
+
+1. Follow current best practices per **A Guide to Documentary Editing** by Mary-Jo Kline. (See [Transcription Methodology](transcription_methodology.md))
+

docs/invariant/transcription_methodology.md

@@ -45,6 +45,24 @@ The following rules map directly to editorial conventions for handling common ma
 | **Non-Textual Artifacts** | Record non-textual elements (seals, stamps, sketches, physical damage) using brief descriptive text inside square brackets. | [description] | [wax notary seal attached here] or [sketch of a fort layout] |
 | **Marginalia & Addenda** | Explicitly indicate spatial transitions before transcribing content located in margins or non-standard orientations. | [location:] | [written in left margin:] Do not share this with anyone. |
 
+### 3.4 Document-Body Medium
+
+Every transcript must identify the predominant document-body medium exactly once at the beginning:
+
+| Medium | Use | Standard Markup |
+| --- | --- | --- |
+| **Handwritten** | The main body was written by hand. | `[document body handwritten]` |
+| **Typewritten** | The main body was produced with a typewriter. Uneven impressions, monospaced characters, and mechanical defects remain typewritten rather than handwritten. | `[document body typewritten]` |
+| **Typeset** | The main body was composed for printing or produced as printed text rather than with a typewriter. | `[document body typeset]` |
+| **Mixed** | No single medium predominates, or handwritten and printed/typewritten content are structurally interleaved. | `[document body mixed]` |
+
+- Use exactly one document-body marker.
+- Do not wrap each line in `[handwritten: ...]` after declaring the body handwritten.
+- In typewritten or typeset documents, use localized handwriting markers only for genuinely handwritten annotations, insertions, or signatures.
+- In mixed documents, identify handwritten portions locally while preserving their reading context.
+- Preserve tables of contents, tables, forms, columns, captions, marginalia, page numbers, dotted leaders, and associated references in their logical reading order.
+- Produce plain text characters rather than HTML entities for ordinary transcription content.
+
 ## 4. Prompt Asset Integration
 
 When executing programmatic transcriptions via LLM APIs or local models, processing instructions must be packaged into single-purpose system prompts aligned with these rules.

docs/invariant/ui_style_guide.md

@@ -0,0 +1,110 @@
+# UI Style Guide (Invariant)
+
+## 1. Purpose
+This guide defines non-negotiable UI styling rules for the transcription application.
+
+The design system is token-first and class-driven:
+1. Theme tokens are defined in [src/transcription/ui/static/theme.css](src/transcription/ui/static/theme.css).
+2. Python UI code composes semantic classes instead of inline color values.
+3. Pages and components should share a single visual language across Documents, Jobs, People, and Sources flows.
+
+## 2. Source of Truth
+Use these files as the style authority:
+1. [src/transcription/ui/static/theme.css](src/transcription/ui/static/theme.css) for color tokens, semantic utility classes, table styles, and viewer surfaces.
+2. [src/transcription/ui/theme.py](src/transcription/ui/theme.py) for runtime NiceGUI theme bridge and shared UI helpers.
+
+If this document conflicts with implementation, update this document to match the code immediately after intentional style changes.
+
+## 3. Core Design Invariants
+1. Flat, high-density surfaces over decorative depth.
+2. Strong content hierarchy with subdued backgrounds and border-based separation.
+3. Viewer area remains the highest contrast region in image/transcription workflows.
+4. Primary actions are consistent and visually recognizable.
+5. Accessible focus rings are always visible for keyboard users.
+
+## 4. Token System
+
+### 4.1 Palette Tokens
+Base palette variables live under :root in [src/transcription/ui/static/theme.css](src/transcription/ui/static/theme.css):
+1. --palette-carbon-black: #1c2321
+2. --palette-cool-steel: #7d98a1
+3. --palette-blue-slate: #5e6572
+4. --palette-powder-blue: #a9b4c2
+5. --palette-platinum: #eef1ef
+
+### 4.2 Semantic Theme Tokens
+Do not style components directly with palette tokens when a semantic token exists.
+
+Semantic tokens currently include:
+1. --theme-text and --theme-text-muted
+2. --theme-page, --theme-surface, --theme-surface-raised, --theme-surface-muted
+3. --theme-border
+4. --theme-primary and --theme-primary-hover
+5. --theme-secondary and --theme-focus
+6. --theme-inverse-text
+7. --theme-viewer, --theme-viewer-border, --theme-viewer-muted
+
+## 5. Approved Semantic Classes
+
+### 5.1 Text and Background
+1. ui-text-primary
+2. ui-text-muted
+3. ui-text-inverse
+4. ui-bg-page
+5. ui-bg-surface
+6. ui-bg-surface-raised
+7. ui-bg-surface-muted
+8. ui-bg-viewer
+9. ui-bg-viewer-overlay
+10. ui-bg-viewer-overlay-soft
+
+### 5.2 Borders and Surfaces
+1. ui-border-subtle
+2. ui-border-viewer
+3. ui-header-divider
+4. ui-card-surface
+5. ui-row-surface
+6. ui-note-box
+7. ui-card-error
+
+### 5.3 Interactive Elements
+1. ui-btn-primary
+2. ui-btn-secondary
+3. ui-link-primary
+4. ui-text-accent
+5. ui-chip-primary
+6. ui-badge-secondary
+7. ui-status and ui-status--<status>
+
+### 5.4 Table Patterns
+1. ui-table
+2. ui-table-header
+3. ui-table-body
+
+Use existing class combinations from [src/transcription/ui/components](src/transcription/ui/components) and [src/transcription/ui/pages](src/transcription/ui/pages) as reference implementations.
+
+## 6. Legacy Class Policy
+Legacy `vibe-` presentation classes are prohibited. Use `ui-` semantic classes from `theme.css`.
+
+## 7. Prohibited Patterns
+1. Inline hex colors in Python UI class strings or style blocks, except in isolated bridge code explicitly marked for migration.
+2. Ad-hoc one-off class names that duplicate existing semantic class intent.
+3. Page-specific palette forks that bypass theme tokens.
+4. Hidden or low-contrast focus states on interactive controls.
+5. Embedded `<style>` blocks or NiceGUI `.style(...)` calls in Python UI code.
+6. Additional page- or component-specific stylesheets; `theme.css` is the single CSS source.
+
+## 8. Implementation Rules For Contributors
+1. Prefer composing existing semantic classes before creating new ones.
+2. If a new class is required, add it to [src/transcription/ui/static/theme.css](src/transcription/ui/static/theme.css) with a semantic name, then reuse it.
+3. Keep behavior ownership in Python and appearance ownership in CSS.
+4. Update UI tests that assert exact text or labels when intentional copy changes are made.
+5. Avoid introducing class churn unrelated to the feature being changed.
+
+## 9. Verification Checklist
+Before merging UI changes, verify:
+1. No new inline hex colors were introduced in UI pages/components.
+2. New styles are token-backed and added to [src/transcription/ui/static/theme.css](src/transcription/ui/static/theme.css).
+3. Primary buttons, links, cards, and tables still render with consistent semantics.
+4. Keyboard focus ring visibility is preserved.
+5. Relevant UI and integration tests pass.

docs/production-runbook.md

@@ -0,0 +1,84 @@
+# Production Runbook
+
+This runbook is the operational checklist for releasing and monitoring the transcription system.
+
+## 1. Pre-release gate checklist
+
+1. Run the full suite: `uv run pytest`
+2. Confirm contract guardrails are green:
+   - `uv run pytest tests/test_meta_contract_guards.py`
+3. Confirm health endpoint includes worker liveness payload (`/healthz` returns `worker.state`).
+4. Confirm required runtime settings are present in deployment environment:
+   - `OPENROUTER_API_KEY`
+   - `DATABASE__*`
+   - filesystem paths for data/logs/backups.
+5. Confirm schema contract alignment is current:
+   - `src/transcription/db/models.py`
+   - `docs/schema.md`
+
+## 2. Release execution steps
+
+1. Deploy artifact/config to target environment.
+2. Validate service startup:
+   - `/healthz` responds `200`
+   - `worker.state` is `running`
+3. Execute one smoke workflow:
+   - create a document/job with at least one source
+   - verify terminal job outcome updates
+   - verify execution evidence row appended
+4. Verify log flow:
+   - stdout aggregation receives events
+   - file logs are written under `./data/logs`
+
+## 3. Rollback triggers and actions
+
+### Trigger conditions
+
+1. `/healthz` reports `worker.state=failed`
+2. Repeated provider timeout/error spikes beyond normal baseline
+3. Evidence write failures or DB persistence failures
+
+### Actions
+
+1. Roll back app artifact and config to previous release.
+2. Restart service and re-check `/healthz`.
+3. Re-run smoke workflow and confirm worker returns to `running`.
+4. Preserve incident evidence:
+   - `./data/logs`
+   - relevant DB rows (`job`, `job_source`, `execution_attempt`)
+
+## 4. Post-release monitoring checklist
+
+## First 24 hours
+
+1. Monitor `/healthz` periodically for `worker.state`.
+2. Track job terminal distribution (`transcribed`, `partial_success`, `failed`).
+3. Sample timeout/error categories for abnormal increase.
+4. Spot-check new `execution_attempt` records for append-only growth and timing metadata.
+
+## First 72 hours
+
+1. Re-check error/timeout trend versus 24h baseline.
+2. Verify no recurring worker-failed states.
+3. Verify storage growth and rotation behavior under `./data/logs`.
+4. Confirm incident response notes are captured for any production anomalies.
+
+## 5. Operator playbook for common incidents
+
+### Worker failed
+
+1. Check `/healthz` payload (`error_id`, `error_category`).
+2. Locate matching error in logs.
+3. If non-transient defect persists, roll back.
+
+### Provider timeout spike
+
+1. Confirm provider reachability and rate limits.
+2. Review timeout frequency and impacted job volume.
+3. If sustained, execute rollback criteria and notify stakeholders.
+
+### Partial-success increase
+
+1. Inspect affected `job_source` and `execution_attempt` records.
+2. Confirm failures are category-aligned (`external`/`timeout`/`internal`).
+3. Triage whether issue is source quality, provider, or runtime regression.

docs/requirements.md

@@ -0,0 +1,85 @@
+# System Requirements (Version 4)
+
+These requirements define the active Version 4 contract and align to current implementation.
+
+## Functional Requirements
+
+### Domain and Record Management
+
+- **REQ-4-001 Document Registry:** The system must create and update `Document` records with title, type, language, comments, date metadata, and optional location.
+- **REQ-4-002 Source Registry:** The system must create and update `Source` records linked to exactly one `Document`.
+- **REQ-4-003 People Registry:** The system must create and update `Person` records and support many-to-many links to `Document` with role and confidence.
+- **REQ-4-004 Registry Semantics:** Document types and person roles must support optional immutable semantic keys and hard-delete only when unreferenced.
+
+### Job and Workflow Behavior
+
+- **REQ-4-010 Job Creation:** The system must create `Job` records from uploaded sources and from retranscription of existing sources.
+- **REQ-4-011 Prompt Snapshotting:** Job creation must persist effective prompt and runtime settings as immutable per-job snapshots.
+- **REQ-4-012 Queue Membership:** Each `(job, source)` pair must be represented by one `JobSource` row.
+- **REQ-4-013 Job Status Lifecycle:** `Job.status` must use one of `queued`, `processing`, `transcribed`, `partial_success`, `failed`.
+- **REQ-4-014 JobSource Status Lifecycle:** `JobSource.status` must use one of `pending`, `transcribed`, `failed`, `cancelled`.
+- **REQ-4-015 Terminal Job Resolution:** Job terminal status must derive from page outcomes as `transcribed`, `partial_success`, or `failed`.
+- **REQ-4-016 Cancellation Semantics:** Job cancellation must set remaining `pending` page entries to `cancelled`.
+
+### Transcription and Evidence
+
+- **REQ-4-020 Attempt Evidence:** Each provider call must emit one append-only `ExecutionAttempt` record.
+- **REQ-4-021 Attempt Payload:** `ExecutionAttempt` must retain request manifest/hash, outcome, timing, model/provider fields, and error details when present.
+- **REQ-4-022 Transport Evidence:** Provider response evidence must be attached to the attempt when a response is available.
+- **REQ-4-023 Source Projection Rule:** `Source.raw_transcription` is a projection chosen from attempt outcomes and can be repointed by explicit promotion.
+- **REQ-4-024 Candidate Visibility:** UI must expose candidate attempts with metadata needed for comparative review and selection.
+
+### Media and Access
+
+- **REQ-4-030 Ingest Canonicalization:** Stored source bytes may be normalized at ingest (for example orientation correction); stored bytes are the canonical processing source.
+- **REQ-4-031 Path Safety:** Client-facing media URLs must be generated from controlled application paths only.
+- **REQ-4-032 Print Media Validation:** Print/export source media must be served through record-validated API routes.
+
+### Error and UX Contracts
+
+- **REQ-4-040 Error Envelope:** Service/API errors must map to structured, user-safe error categories and messages.
+- **REQ-4-041 Partial Failure Visibility:** Mixed page outcomes must be visible at job and page level.
+- **REQ-4-042 Retry Support:** Failed and cancelled pages must support targeted retranscription without requiring full document recreation.
+
+## Non-Functional Requirements
+
+- **REQ-4-100 Boundary Integrity:** UI pages/components must not access persistence directly and must call service APIs.
+- **REQ-4-101 Service Ownership:** Aggregate writes must occur in owning service/workflow modules, not in UI handlers.
+- **REQ-4-102 Deterministic Loading:** ORM relationship reads in service/UI code must use explicit eager loading compatible with `lazy="raise"`.
+- **REQ-4-103 Async Safety:** Long-running provider calls must not block UI event handlers directly.
+- **REQ-4-104 Evidence Durability:** Attempt evidence must survive process restart once the transaction commits.
+- **REQ-4-105 Test Guardrails:** Architecture boundary tests must remain in place for services and UI boundaries.
+
+## Requirement Interpretation Notes
+
+### Status and lifecycle semantics
+
+- `REQ-4-013` and `REQ-4-015` intentionally bind success to `transcribed`, not a generic `completed`, so docs, tests, and runtime transitions stay consistent.
+- `REQ-4-016` and `REQ-4-042` distinguish cancellation from failure at page level (`cancelled` vs `failed`) while still allowing targeted retranscription.
+
+### Evidence semantics
+
+- `REQ-4-020` through `REQ-4-024` separate authoritative history (`ExecutionAttempt`) from operational projection (`Source.raw_transcription`).
+- This supports immutable provenance while allowing explicit candidate promotion for operator workflows.
+
+### Boundary and loading semantics
+
+- `REQ-4-100` and `REQ-4-101` codify aggregate/service ownership and keep UI out of persistence concerns.
+- `REQ-4-102` exists to enforce deterministic query shape under `lazy="raise"` and avoid hidden data access in rendering callbacks.
+
+## Verification Anchors
+
+- Service boundary enforcement: `tests/test_service_boundaries.py`
+- UI boundary enforcement: `tests/test_ui_boundaries.py`
+- Job lifecycle reliability and terminal status behavior: `tests/services/test_workflows_reliability.py`
+- Evidence append-only and projection behavior: `tests/services/test_store.py`, `tests/services/test_transcription_service.py`
+
+## Traceability Notes
+
+- Source of truth for status enums:
+  - `src/transcription/db/models.py`
+- Source of truth for workflow transitions:
+  - `src/transcription/services/workflows.py`
+  - `src/transcription/services/jobs.py`
+- Source of truth for attempt evidence writes:
+  - `src/transcription/services/sources.py`

docs/requirements_v2.md

@@ -1,42 +0,0 @@
-# Document Transcription System Requirements (Version 2)
-
-This document captures the **Version 2 baseline requirements** for the production implementation.
-
-## Requirements Model
-
-| ID | Category | Requirement | Verify Method |
-| --- | --- | --- | --- |
-| REQ-0 | System | Provide end-to-end multi-page document transcription with persistent, inspectable async job states. | demonstration |
-| REQ-1 | Functional | Allow users to upload folders or multi-image batches as sequential `Source` pages under a `Document`. | test |
-| REQ-2 | Functional | Process multi-page jobs asynchronously using an `asyncio` worker pool bounded by rate limits. | test |
-| REQ-3 | Functional | Persist page-level execution outputs (`raw_transcription`, `ai_metadata`, `raw_api_response`) on `JobSource`. | test |
-| REQ-4 | Functional | Support job states (`queued`, `processing`, `completed`, `partial_success`, `failed`) and page states (`pending`, `transcribed`, `failed`). | inspection |
-| REQ-5 | Functional | Allow users to manage historical `Person` records and link multiple authors/recipients to a `Document` via `DocumentPerson`. | test |
-| REQ-6 | Functional | Maintain immutable original machine output on `Source.raw_transcription` while permitting inline human edits on `Source.revised_text`. | test |
-| REQ-7 | Data Constraint | Store all persistent domain data in PostgreSQL using native `UUID`, `TIMESTAMPTZ`, and `JSONB` columns. | inspection |
-| REQ-8 | Data Constraint | Validate all API requests, database rows, and JSONB structures using Pydantic V2 schemas. | test |
-| REQ-9 | Interface | Render multi-page transcriptions sequentially by `page_number` in the web UI with author/recipient metadata. | demonstration |
-| REQ-10 | Operations | Allow operators to retry only failed pages for jobs in a `partial_success` state. | test |
-
-## Element Satisfaction Mapping
-
-* **UI (NiceGUI):** Satisfies REQ-1, REQ-5, REQ-6, REQ-9, REQ-10.
-* **API (FastAPI):** Satisfies REQ-1, REQ-4, REQ-5, REQ-8.
-* **WORKER (asyncio):** Satisfies REQ-2, REQ-3, REQ-4, REQ-10.
-* **PERSISTENCE (PostgreSQL):** Satisfies REQ-3, REQ-6, REQ-7.
-* **MODELS (Pydantic V2):** Satisfies REQ-8.
-
----
-
-## Related Local References
-
-- [System Overview](index_v2.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v2.md)
-- System Requirements (this document)
-- [Data model](schema_v2.md)
-- [Error Handling Policy](error_handling_v2.md)
-- [Implementation Plan](implementation_plan_v2.md)
-
-

docs/roadmap_plan.md

@@ -0,0 +1,217 @@
+# Roadmap Review: Recommendations & Version Plan
+
+This is a review/consulting deliverable. It organizes your near-term changes,
+nice-to-haves, and long-range goals into version buckets, building on
+`./docs/ver4.8/feature_backlog_v4_8.md` (already-scoped image work) and
+folding in the additional items from your original list.
+
+Versioning convention (per your direction): **4.8–4.11** are incremental
+feature/fix releases; **5.x and 6.x are reserved for major changes** to the
+app (a shared data-model overhaul, and the hosting migration, respectively).
+
+Final deliverable will be saved to `./docs` in the repo per your instruction.
+
+---
+
+## V4.8 — Bug fixes + Image Experience
+
+**Bug fixes (do first within this release — data correctness, not features):**
+1. **Stale Error Detail after resubmit.** Root cause found in `db/models.py`:
+   `Source.latest_error_detail` sorts all `execution_attempts` on the latest
+   job by `attempt_number` descending and returns the **first attempt with
+   any `error_detail`**, even if that's an older attempt and the latest
+   attempt succeeded. Fix: only report error detail from the latest attempt,
+   don't fall through to earlier ones.
+2. **"View Jobs" ≠ "View Sources" behavior.** `/documents/{id}/sources`
+   already redirects to the filtered Transcription Pipeline Jobs page;
+   `/documents/{id}/jobs` renders its own bare two-field custom page instead.
+   Fix: make `/documents/{id}/jobs` redirect the same way, filtered by document.
+
+**Image Experience (Track A from the V4.8 backlog doc — already scoped, high
+value/low risk, storage layer already exists):**
+3. **Homepage Image Gallery** (recommended first feature in that doc):
+   `list_homepage_images()` already returns every stored image; only the
+   multi-image carousel UI and optional slideshow rotation are missing.
+4. **Pan and Zoom on Source Detail**: recover `document_panzoom.py` from git
+   history (deleted in V4.6 Phase 5, `6a3ee26`), but vendor the Panzoom
+   library locally instead of the unpkg CDN, and reuse `resolve_media_url`
+   from `media_urls.py` instead of recreating its old helper. Scope to Source
+   Detail only — do not add it to the shared `dark_room_viewer`.
+
+**Note:** the backlog doc's model-performance/telemetry item still applies as
+stated there — it must wait for V4.7 Phase 4 to land first, since `duration_ms`
+today mixes provider latency with preprocessing/DB commit time.
+
+---
+
+## V4.9 — Detail Page Parity Pass
+
+Small, low-risk structural-parity fixes across Person/Job/Document detail pages:
+- Hide "Maiden Name" on Person detail when empty.
+- Job Record detail "Document Links" box → mirror the Document detail page's
+  "Sources & Pipeline Jobs" box (clickable name, Sources count, single
+  "View Sources" button, no "+Add Job").
+- Transcription Pipeline Jobs page: add a Document Name column; resolve
+  Created vs. Updated by keeping **Updated** as the primary/visible field
+  (more actionable) and de-emphasizing Created rather than deleting it outright.
+- Person detail "Linked Documents" → 3-column table (Document Name, Role,
+  Number of Pages), with clickable document names, dropping the "Open" button.
+
+---
+
+## V4.10 — Settings Consolidation & Small Enhancements
+
+- **Settings page → tabs**, not sub-pages (per your confirmation). Convert the
+  existing stacked cards (Document Types, Person Roles, Prompts) into
+  `ui.tabs`/`ui.tab_panels`, and add a new **Home Page Text** tab. One route,
+  no navigation overhead, scales cleanly as more settings are added.
+- **Google Maps links** for Person birth/death locations — cheap, no
+  scraping, a formatted place string becomes a Maps search link.
+- **FamilySearch ID lookup** (scope confirmed as narrow): on the Person
+  create/edit form, add a FamilySearch ID field and an "Auto-fill from
+  FamilySearch" button that fetches only birth date/place, death date/place,
+  and marriage date/spouse for that specific ID — a single-record lookup, not
+  a crawler. This keeps FamilySearch.org as a companion reference rather than
+  a data source the app tries to replace. Because it's ID-driven and
+  single-record, it's a much smaller, safer feature than open-ended scraping —
+  worth doing at this scope, revisit if FamilySearch's page structure changes
+  and breaks the parser.
+
+---
+
+## V4.11 — Approved Scope
+
+- **Tags** (supersedes "collections"): many-to-many tagging for Documents with
+  Settings-style management (same pattern as Document Types and Person Roles),
+  autocomplete-capable assignment, and a dedicated **Tags** entry point for
+  browse/filter-by-tag workflows.
+- **Source Detail simplification**: remove the separate **Transcription Text**
+  card; show Source image + Editable Revision + Source/SourceJob metadata in a
+  3-column top layout, then keep Candidate Machine Transcriptions below the
+  image/revision area.
+- **Integrity reconciliation checks in tests**:
+  - document folder count under `UPLOAD_DIR/documents` must equal `document`
+    row count.
+  - source file count under each `UPLOAD_DIR/documents/{document_id}` folder
+    must equal `source` row count for that Document.
+  - failures should include actionable mismatch details (missing row/folder or
+    file/source mapping).
+- **UI table updates**:
+  - Archival Documents: remove **Archive Ref**, add **# Sources**.
+  - Archival Entities: People: remove **Display Name** and **Maiden Name**
+    columns, add **FamilySearch ID**.
+  - Transcription Pipeline Jobs: add **# Sources**.
+- **Create Processing Job page**: Provider and Model must be selectable for new
+  job creation.
+
+Deferred out of this release:
+
+- UI theme selection.
+- Settings-based `.env` editing and runtime controls.
+- Person table structural redesign (removing/splitting name fields).
+
+---
+
+## V5.0 — Unified Photos Table (major data-model change)
+
+V5.0 standardizes homepage images and Person portraits into one shared `photo`
+table and one storage layout.
+
+Finalized shape:
+- `photo`: `id`, nullable `person_id`, `path`, `description`, `is_primary`,
+  timestamps.
+- `person_id IS NULL` = homepage photos; non-null = Person photos.
+- `is_primary` is the featured/first photo for that owner (homepage or Person).
+- No separate context enum; ownership is derived from `person_id`.
+- All image files are stored under `UPLOAD_DIR/photos/{photo_id}{suffix}`.
+- `Person.portrait_path` is removed.
+- `HOMEPAGE_DIR` is retired; homepage markdown remains file-backed at
+  `UPLOAD_DIR/homepage.md`.
+
+Migration policy for legacy installs:
+- Export/import rebuild remains the migration mechanism.
+- Legacy `person.portrait_path` values are backfilled into `photo` rows.
+- Legacy homepage images under `UPLOAD_DIR/homepage` are backfilled into
+  homepage `photo` rows.
+- Legacy homepage markdown is relocated to `UPLOAD_DIR/homepage.md`.
+
+---
+
+## V6.0 — Server Hosting Migration
+
+Your stated approach (Postgres in Docker, app in Docker, Cloudflare Tunnel) is
+the standard, low-maintenance way to get secure remote access without exposing
+ports or running your own VPN/reverse-proxy TLS setup. Sequential stages
+(each de-risks the next):
+
+**Stage 0 — Prerequisites**
+- Confirm data access already goes through the service layer (it does, per
+  `.github/instructions/services.instructions.md`) — this is what makes the
+  DB swap and later auth additive rather than a rewrite.
+- Confirm upload/file storage path (`settings.upload_dir`) is Docker-volume-friendly.
+- Confirm DB URL and upload dir are both env-overridable (`.env`/`.env.example`
+  already exist).
+
+**Stage 1 — Containerize** against the *existing* SQLite file first (smallest
+possible change) to validate the container boundary (networking, volumes,
+permissions) before also switching databases. `Dockerfile`/`docker-compose.yml`
+already exist in the repo — confirm current intent vs. extend for production.
+
+**Stage 2 — Migrate to PostgreSQL.** Since the app uses SQLModel/SQLAlchemy,
+this should mostly be a connection string/dialect change plus a data migration
+script. Do this as its own isolated step so a regression is attributable to
+the DB swap alone. Resolve the Postgres connection through a small
+factory/indirection point rather than a single global engine constant — this
+costs nothing now and keeps the door open for per-user databases later (see
+Beyond, below).
+
+**Stage 3 — Expose via Cloudflare Tunnel.** Add a `cloudflared` container
+pointed at the app's internal port (no public port exposure needed). Put the
+tunnel hostname behind Cloudflare Access (free tier, email OTP/SSO) as the
+**first layer of remote-access auth**, even before the app has its own login
+system — gets secure remote access working quickly and buys time to do
+Stage 4 and later user-auth properly instead of rushed.
+
+**Stage 4 — Backups & operational hygiene.** Automated Postgres backups
+(scheduled `pg_dump`) and uploaded-file backups *before* exposing this to the
+internet — this is the point where a data-loss incident would be most
+damaging. Basic container health checks/restart policies in compose.
+
+---
+
+## V6.1 — Reporting Features
+
+Independent of hosting — can be built any time, including in parallel with
+V6.0:
+- **Person timelines**: a query/aggregation feature (documents by person,
+  ordered by document date), no hosting dependency.
+- **AI-written biographies/family histories**: likely reuses the existing
+  prompt/provider abstraction (`services/prompts.py`) already built for
+  transcription, rather than needing new infrastructure.
+
+---
+
+## Beyond / Pie-in-the-sky (informational only — no version assigned)
+
+Not being built now, but the plan above is checked against these so nothing
+forecloses them later:
+
+- **Individual user logins + roles** (admin/editor/contributor/view-only):
+  Cloudflare Access (V6.0 Stage 3) is a stopgap, not this. When you get here,
+  add a real `User`/`Role` model and auth middleware. Because the codebase
+  already funnels data access through the service layer, adding a
+  "current user + role check" later is additive — this is the reason to keep
+  service-layer discipline intact through all the versions above, so
+  authorization can be bolted on without touching every page.
+- **Per-user databases**: protected for by the Stage 2 connection-factory
+  recommendation above (schema-per-tenant or database-per-tenant becomes
+  much easier if the DB connection is already resolved through an
+  indirection point rather than hardcoded at startup).
+
+---
+
+## Open items for you
+- V5.0 unified `photos` table: needs the follow-up design discussion you
+  flagged (exact schema, how photos link to homepage vs. person context)
+  before implementation.
+- Confirm this version numbering/grouping matches your intent before work starts.

docs/schema.md

@@ -0,0 +1,280 @@
+# Data Model and Persistence Schema (Version 4)
+
+This document is the field-accurate Version 4 schema contract aligned to `src/transcription/db/models.py`.
+
+## Source of Truth Anchors
+
+- `src/transcription/db/models.py:60-78` (status and purpose enums)
+- `src/transcription/db/models.py:80-120` (`DocumentType`, `PersonRole`)
+- `src/transcription/db/models.py:122-172` (`Tag`, `Document`)
+- `src/transcription/db/models.py:175-281` (`Person`, `Photo`, `DocumentPerson`, `DocumentTag`)
+- `src/transcription/db/models.py:285-347` (`Job`)
+- `src/transcription/db/models.py:350-462` (`Source`, `JobSource`)
+- `src/transcription/db/models.py:465-522` (`ExecutionAttempt`)
+
+## Entity Relationship Overview
+
+```mermaid
+erDiagram
+    DocumentType ||--o{ Document : classifies
+    Document ||--o{ Job : has
+    Document ||--o{ Source : has
+    Document ||--o{ DocumentPerson : links
+    Document ||--o{ DocumentTag : tagged
+    Person ||--o{ DocumentPerson : links
+    Person ||--o{ Photo : owns
+    PersonRole ||--o{ DocumentPerson : labels
+    Tag ||--o{ DocumentTag : labels
+    Job ||--o{ JobSource : includes
+    Source ||--o{ JobSource : participates
+    JobSource ||--o{ ExecutionAttempt : attempts
+```
+
+## Authoritative Enumerations
+
+### JobStatus
+
+- `queued`
+- `processing`
+- `transcribed`
+- `partial_success`
+- `failed`
+
+### JobSourceStatus
+
+- `pending`
+- `transcribed`
+- `failed`
+- `cancelled`
+
+### JobPurpose
+
+- `transcription`
+- `retranscription`
+
+## Field-Accurate Table Contracts
+
+### `DocumentType`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `semantic_key` | `str \| None` | nullable unique, indexed |
+| `label` | `str` | required |
+| `normalized_label` | `str` | unique, indexed |
+| `is_active` | `bool` | default `True` |
+| `created_at` | `datetime` | default now |
+| `updated_at` | `datetime` | default now, onupdate |
+
+### `PersonRole`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `semantic_key` | `str \| None` | nullable unique, indexed |
+| `label` | `str` | required |
+| `normalized_label` | `str` | unique, indexed |
+| `is_active` | `bool` | default `True` |
+| `created_at` | `datetime` | default now |
+| `updated_at` | `datetime` | default now, onupdate |
+
+### `Tag`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `semantic_key` | `str \| None` | nullable unique, indexed |
+| `label` | `str` | required |
+| `normalized_label` | `str` | unique, indexed |
+| `is_active` | `bool` | default `True` |
+| `created_at` | `datetime` | default now |
+| `updated_at` | `datetime` | default now, onupdate |
+
+### `Document`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `name` | `str` | required |
+| `document_type_id` | `UUID \| None` | FK -> `document_type.id`, indexed |
+| `document_date` | `date \| None` | optional |
+| `document_date_raw` | `str \| None` | optional |
+| `location_created` | `str \| None` | optional |
+| `notes` | `str \| None` | optional |
+| `archive_identifier` | `str \| None` | optional |
+| `created_at` | `datetime` | default now |
+| `updated_at` | `datetime` | default now, onupdate |
+
+### `Person`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `full_name` | `str` | required |
+| `display_name` | `str \| None` | optional |
+| `maiden_name` | `str \| None` | optional |
+| `birth_date` | `date \| None` | optional |
+| `birth_date_raw` | `str \| None` | optional |
+| `birth_place` | `str \| None` | optional |
+| `death_date` | `date \| None` | optional |
+| `death_date_raw` | `str \| None` | optional |
+| `death_place` | `str \| None` | optional |
+| `biography` | `str \| None` | optional |
+| `family_search_id` | `str \| None` | nullable unique |
+| `metadata_` | `dict[str, JsonValue] \| None` | stored as DB column `metadata` (`JSONBCompat`) |
+| `created_at` | `datetime` | default now |
+| `updated_at` | `datetime` | default now, onupdate |
+
+### `Photo`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `person_id` | `UUID \| None` | nullable FK -> `person.id`, indexed (`NULL` = homepage photo) |
+| `path` | `str` | required upload-root-relative POSIX path (`photos/...`) |
+| `description` | `str \| None` | optional |
+| `is_primary` | `bool` | default `False`; owner-level "featured/primary" marker |
+| `created_at` | `datetime` | default now |
+| `updated_at` | `datetime` | default now, onupdate |
+
+### `DocumentPerson`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `document_id` | `UUID` | FK -> `document.id`, indexed |
+| `person_id` | `UUID` | FK -> `person.id`, indexed |
+| `role_id` | `UUID` | FK -> `person_role.id`, indexed |
+| `created_at` | `datetime` | default now |
+| `updated_at` | `datetime` | default now, onupdate |
+
+Constraint:
+- `UniqueConstraint(document_id, person_id)` named `uq_document_person`
+
+### `DocumentTag`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `document_id` | `UUID` | FK -> `document.id`, indexed |
+| `tag_id` | `UUID` | FK -> `tag.id`, indexed |
+| `created_at` | `datetime` | default now |
+| `updated_at` | `datetime` | default now, onupdate |
+
+Constraint:
+- `UniqueConstraint(document_id, tag_id)` named `uq_document_tag`
+
+### `Job`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `document_id` | `UUID` | FK -> `document.id`, indexed |
+| `status` | `JobStatus` | non-null enum (stored as enum values) |
+| `retry_count` | `int` | default `0`, `ge=0` |
+| `purpose` | `JobPurpose` | non-null enum, default `transcription` |
+| `date_created` | `datetime` | default now |
+| `date_updated` | `datetime` | default now, onupdate |
+| `provider` | `str \| None` | optional |
+| `model` | `str \| None` | optional |
+| `prompt_name` | `str \| None` | optional |
+| `prompt_hash` | `str \| None` | optional |
+| `system_prompt` | `str \| None` | optional |
+| `user_prompt` | `str \| None` | optional |
+| `temperature` | `float \| None` | optional |
+| `top_p` | `float \| None` | optional |
+
+Index:
+- `Index("ix_job_status_date_created", "status", "date_created")`
+
+### `Source`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `document_id` | `UUID` | FK -> `document.id`, indexed |
+| `page_number` | `int` | default `1`, `ge=1` |
+| `upload_name` | `str` | required |
+| `filename` | `str` | required |
+| `file_path` | `str` | required upload-root-relative POSIX path (`documents/...`) |
+| `file_hash` | `str` | required |
+| `file_size_bytes` | `int` | `BigInteger`, non-null |
+| `raw_transcription` | `str \| None` | projection field |
+| `preferred_execution_attempt_id` | `UUID \| None` | nullable FK -> `execution_attempt.id`, indexed (`use_alter`) |
+| `revised_text` | `str \| None` | optional human revision |
+| `date_uploaded` | `datetime` | default now |
+| `date_revised` | `datetime \| None` | optional |
+
+### `JobSource`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `job_id` | `UUID` | FK -> `job.id`, indexed |
+| `source_id` | `UUID` | FK -> `source.id`, indexed |
+| `status` | `JobSourceStatus` | non-null enum, default `pending` |
+
+Constraint:
+- `UniqueConstraint(job_id, source_id)` named `uq_job_source_job_source`
+
+Runtime reconciliation:
+- Startup database operations remove retired V4.6 `job_source` evidence columns (`raw_transcription`, `ai_metadata`, `raw_api_response`, `error_detail`, `executed_at`) when present so persisted schema matches this contract.
+
+### `ExecutionAttempt`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `job_source_id` | `UUID` | FK -> `job_source.id`, indexed |
+| `job_id` | `UUID` | FK -> `job.id`, indexed |
+| `source_id` | `UUID` | FK -> `source.id`, indexed |
+| `attempt_number` | `int` | `ge=1` |
+| `status` | `JobSourceStatus` | non-null enum, value-stable with `JobSource.status` |
+| `provider` | `str` | required |
+| `model` | `str \| None` | optional |
+| `request_manifest` | `dict[str, JsonValue] \| None` | JSONBCompat |
+| `request_manifest_sha256` | `str \| None` | optional |
+| `request_manifest_schema_version` | `str \| None` | optional |
+| `response_received` | `bool` | default `False` |
+| `transport_status_code` | `int \| None` | optional |
+| `transport_body` | `bytes \| None` | LargeBinary |
+| `transport_content_type` | `str \| None` | optional |
+| `transport_content_encoding` | `str \| None` | optional |
+| `transport_safe_headers` | `dict[str, JsonValue] \| None` | JSONBCompat |
+| `router_request_id` | `str \| None` | optional |
+| `router_generation_id` | `str \| None` | optional |
+| `sdk_response_snapshot` | `dict[str, JsonValue] \| None` | JSONBCompat |
+| `normalized_metadata` | `dict[str, JsonValue] \| None` | JSONBCompat; may include app-namespaced `processing_timing` (`provider_call_duration_ms`, `processing_duration_ms`) |
+| `software_context` | `dict[str, JsonValue] \| None` | JSONBCompat |
+| `raw_transcription` | `str \| None` | optional |
+| `error_category` | `str \| None` | optional |
+| `error_detail` | `str \| None` | optional |
+| `failure_phase` | `str \| None` | optional |
+| `started_at` | `datetime` | required |
+| `finished_at` | `datetime` | required |
+| `duration_ms` | `int` | `ge=0` |
+| `created_at` | `datetime` | default now |
+
+Constraint:
+- `UniqueConstraint(job_id, source_id, attempt_number)` named `uq_execution_attempt_number`
+
+## Relationship Loading Contract
+
+- Most ORM relationships are configured with `lazy="raise"`.
+- `JobSource.execution_attempts` is intentionally `lazy="noload"` with ordered attempts.
+- Service/UI read paths must explicitly eager-load required relationships before access.
+
+## Persistence Invariants (Ground Truth)
+
+1. `ExecutionAttempt` is append-only runtime evidence.
+2. `JobSource.status` represents queue/projection execution state and is not a full evidence container.
+3. `Source.raw_transcription` is a mutable projection and not authoritative attempt history.
+4. `Job` terminal status derives from page outcomes (`JobSource` state), not from a separate summary table.
+5. `DocumentType.semantic_key` and `PersonRole.semantic_key` are nullable-unique semantic identifiers.
+
+## Cross-Reference
+
+- [System Architecture](architecture.md)
+- [System Requirements](requirements.md)
+- [Error Handling Policy](error_handling.md)
+- [AI Evidence and Provenance Invariant](../invariant/ai_evidence_and_provenance.md)

docs/schema_v2.md

@@ -1,136 +0,0 @@
-# Database Schema (Version 2)
-
-This document describes the PostgreSQL relational schema for the transcription platform. It incorporates multi-image batch orchestration via `asyncio`, page-level execution tracking, many-to-many author/recipient attribution, and JSONB document storage for AI vision outputs.
-
-All primary and foreign keys are PostgreSQL native UUIDs (`gen_random_uuid()`).
-
-## Entity Relationship Diagram
-
-```mermaid
-erDiagram
-    PERSON {
-        UUID id PK
-        TEXT full_name
-        TEXT display_name
-        TEXT maiden_name
-        DATE birth_date
-        TEXT birth_date_raw
-        TEXT birth_place
-        DATE death_date
-        TEXT death_date_raw
-        TEXT death_place
-        TEXT biography
-        TEXT portrait_path
-        JSONB metadata
-        TIMESTAMPTZ created_at
-        TIMESTAMPTZ updated_at
-    }
-
-    DOCUMENT {
-        UUID id PK
-        TEXT name
-        TEXT document_type
-        DATE document_date
-        TEXT document_date_raw
-        TEXT location_created
-        TEXT notes
-        TEXT archive_identifier
-        TIMESTAMPTZ created_at
-        TIMESTAMPTZ updated_at
-    }
-
-    DOCUMENT_PERSON {
-        UUID id PK
-        UUID document_id FK
-        UUID person_id FK
-        VARCHAR role "author | recipient"
-        TIMESTAMPTZ created_at
-    }
-
-    JOB {
-        UUID id PK
-        UUID document_id FK
-        VARCHAR status "queued | processing | completed | partial_success | failed"
-        INTEGER retry_count
-        TEXT provider
-        TEXT model
-        TEXT prompt_name
-        TIMESTAMPTZ date_created
-        TIMESTAMPTZ date_updated
-    }
-
-    SOURCE {
-        UUID id PK
-        UUID document_id FK
-        INTEGER page_number
-        TEXT upload_name
-        TEXT filename
-        TEXT file_path
-        TEXT raw_transcription
-        TEXT revised_text
-        TIMESTAMPTZ date_uploaded
-        TIMESTAMPTZ date_revised
-    }
-
-    JOB_SOURCE {
-        UUID id PK
-        UUID job_id FK
-        UUID source_id FK
-        VARCHAR status "pending | transcribed | failed"
-        TEXT raw_transcription
-        JSONB ai_metadata
-        JSONB raw_api_response
-        TEXT error_detail
-        TIMESTAMPTZ executed_at
-    }
-
-    DOCUMENT ||--o{ DOCUMENT_PERSON : "has_people"
-    PERSON ||--o{ DOCUMENT_PERSON : "participates_in"
-    DOCUMENT ||--o{ JOB : "has_jobs"
-    DOCUMENT ||--o{ SOURCE : "contains_pages"
-    JOB ||--o{ JOB_SOURCE : "executes"
-    SOURCE ||--o{ JOB_SOURCE : "processed_in"
-```
-
-## Domain Invariants & Rules
-
-### Page-Level Execution & AI Outputs
-
-* Execution Granularity: Every single image execution by an AI model produces a dedicated record in job_source.
-* Point-in-Time Auditability: job_source.raw_api_response stores the unparsed REST response envelope for that specific image page call. job_source.ai_metadata stores spatial bounding boxes, token usage, and layout details for that specific image page call.
-* Active Output Caching: Upon successful completion of an image call, source.raw_transcription is updated with the latest output string from job_source.raw_transcription for fast UI rendering.
-
-### Page Ordering & Revisions
-
-* Sequential Integrity: source.page_number dictates page ordering within a document. Reads assembling full documents must query ORDER BY source.document_id, source.page_number ASC.
-* Inlined Human Corrections: User edits occur at the page level inside source.revised_text. source.raw_transcription remains immutable. If source.revised_text is non-null, application frontends must render source.revised_text.
-
-### Async Job Lifecycle & Failure Isolation
-
-* Batch Orchestrator: A job represents an overarching execution run across one or more source images belonging to a document.
-* Isolated Failures: API requests run concurrently (e.g., using asyncio). A failure on page 3 does not invalidate successful transcriptions on page 1 or 2.
-* Job States:
-    - queued: Created, awaiting worker execution.
-    - processing: Concurrent HTTP tasks actively running.
-    - completed: 100% of linked job_source tasks succeeded (transcribed).
-    - partial_success: At least one job_source succeeded and at least one failed.
-    - failed: All linked job_source tasks failed or a job-level runtime error occurred.
-
-### Attribution & Person Roles
-
-* Multi-Person Roles: Documents support zero, one, or many authors and recipients linked via document_person.
-* Role Uniqueness: (document_id, person_id, role) must be unique to prevent duplicate role tagging.
-
----
-
-## Related Local References
-
-- [System Overview](index_v2.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v2.md)
-- [System Requirements](requirements_v2.md)
-- Data model (this document)
-- [Error Handling Policy](error_handling_v2.md)
-- [Implementation Plan](implementation_plan_v2.md)
-

docs/test_block-beta.md

@@ -1,92 +0,0 @@
-```mermaid
-block-beta
-  columns 3
-
-  %% UI Component Column
-  block:UI["UI COMPONENTS / WIREFRAME"]:1
-    columns 1
-    
-    block:HeaderUI["Header & Nav"]:1
-      columns 1
-      h_title["[Text] Document Name & Type"]
-      h_date["[Text] Date & Origin Location"]
-    end
-
-    block:EditorUI["Page Transcription Editor"]:1
-      columns 1
-      ed_img["[Image Viewer] Source Image"]
-      ed_page["[Badge] Page Number"]
-      ed_raw["[Read-Only] AI Raw Output"]
-      ed_rev["[Textarea] Human Revised Text"]
-    end
-
-    block:PeopleUI["Attribution Sidebar"]:1
-      columns 1
-      p_author["[List] Authors (Full Name)"]
-      p_recip["[List] Recipients (Full Name)"]
-      p_bio["[Card] Person Biography & Dates"]
-    end
-
-    block:JobUI["AI Processing Drawer"]:1
-      columns 1
-      j_status["[Badge] Job Status"]
-      j_model["[Text] Provider & Model"]
-      j_tokens["[JSON View] AI Token Usage"]
-    end
-  end
-
-  %% Directional Mapping / Connectors
-  block:FLOW["MAPPING / FLOW"]:1
-    columns 1
-    f1["Reads / Updates -->"]
-    f2["Renders Active Page -->"]
-    f3["Joins via Role -->"]
-    f4["Executes & Logs -->"]
-  end
-
-  %% Postgres Schema Column
-  block:DB["POSTGRES SQL SCHEMA"]:1
-    columns 1
-
-    block:DocTbl["Table: document"]:1
-      columns 1
-      d_id["id : UUID (PK)"]
-      d_name["name : TEXT"]
-      d_type["document_type : TEXT"]
-      d_date["document_date : DATE"]
-    end
-
-    block:SrcTbl["Table: source"]:1
-      columns 1
-      s_id["id : UUID (PK)"]
-      s_page["page_number : INT"]
-      s_path["file_path : TEXT"]
-      s_raw["raw_transcription : TEXT"]
-      s_rev["revised_text : TEXT"]
-    end
-
-    block:PersonTbl["Table: person & document_person"]:1
-      columns 1
-      p_id["id : UUID (PK)"]
-      p_name["full_name : TEXT"]
-      p_role["role : 'author' | 'recipient'"]
-    end
-
-    block:JobTbl["Table: job & job_source"]:1
-      columns 1
-      j_id["id : UUID (PK)"]
-      j_stat["status : VARCHAR"]
-      j_prov["provider / model : TEXT"]
-      j_meta["ai_metadata : JSONB"]
-    end
-  end
-
-  %% Connections
-  HeaderUI --> DocTbl
-  ed_img --> s_path
-  ed_page --> s_page
-  ed_raw --> s_raw
-  ed_rev --> s_rev
-  PeopleUI --> PersonTbl
-  JobUI --> JobTbl
-```

docs/test_flowchart.md

@@ -1,53 +0,0 @@
-```mermaid
-flowchart LR
-    subgraph UI["UI Components / Wireframe"]
-        direction TB
-        subgraph HeaderUI["Header & Nav"]
-            h_title["[Text] Document Name & Type"]
-            h_date["[Text] Date & Origin Location"]
-        end
-        subgraph EditorUI["Page Transcription Editor"]
-            ed_img["[Image Viewer] Source Image"]
-            ed_page["[Badge] Page Number"]
-            ed_raw["[Read-Only] AI Raw Output"]
-            ed_rev["[Textarea] Human Revised Text"]
-        end
-        subgraph PeopleUI["Attribution Sidebar"]
-            p_author["[List] Authors / Recipients"]
-        end
-        subgraph JobUI["AI Processing Drawer"]
-            j_status["[Badge] Job Status"]
-        end
-    end
-
-    subgraph DB["Postgres SQL Schema"]
-        direction TB
-        subgraph DocTbl["Table: document"]
-            d_name["name : TEXT"]
-            d_type["document_type : TEXT"]
-        end
-        subgraph SrcTbl["Table: source"]
-            s_path["file_path : TEXT"]
-            s_page["page_number : INT"]
-            s_raw["raw_transcription : TEXT"]
-            s_rev["revised_text : TEXT"]
-        end
-        subgraph PersonTbl["Table: person & document_person"]
-            p_name["full_name : TEXT"]
-            p_role["role : author | recipient"]
-        end
-        subgraph JobTbl["Table: job & job_source"]
-            j_stat["status : VARCHAR"]
-            j_meta["ai_metadata : JSONB"]
-        end
-    end
-
-    %% Mappings
-    HeaderUI --> DocTbl
-    ed_img --> s_path
-    ed_page --> s_page
-    ed_raw --> s_raw
-    ed_rev --> s_rev
-    PeopleUI --> PersonTbl
-    JobUI --> JobTbl
-```

docs/ui/README.md

@@ -0,0 +1,59 @@
+# UI Behavioral Contracts
+
+## Purpose
+
+This directory defines the current user-facing behavior of the NiceGUI application. It records what each page is for, which routes and actions it exposes, what information it presents, and how success, empty, validation, and failure states behave.
+
+These documents are written for maintainers and AI contributors. They are behavioral contracts, not historical implementation notes and not substitutes for the database schema.
+
+## Current Page Contracts
+
+- [Home](pages/home.md)
+- [Documents](pages/documents.md)
+- [People](pages/people.md)
+- [Jobs](pages/jobs.md)
+- [Sources](pages/sources.md)
+
+NiceGUI registers the routes shown in each contract without the `/ui` prefix. The application mounts NiceGUI under `/ui`, so `/documents` in page code is served to a browser as `/ui/documents`.
+
+## Authority Hierarchy
+
+When documents disagree, use this order:
+
+1. User-facing page intent and accepted behavior: the page contracts in this directory.
+2. Visual and interaction styling: [UI Style Guide](../invariant/ui_style_guide.md).
+3. UI dependency and ownership boundaries: [UI contributor instructions](../../.github/instructions/ui.instructions.md).
+4. Durable failure behavior: [Error Handling invariant](../invariant/error_handling.md).
+5. Durable AI evidence behavior: [Digital Evidence and AI Processing Provenance](../invariant/ai_evidence_and_provenance.md).
+6. Data definitions and relationships: current models plus the [schema contract](../schema.md).
+7. Implementation truth: current code and tests.
+
+If code intentionally changes accepted page behavior, update the corresponding page contract in the same change. If code accidentally differs, correct the implementation rather than rewriting intent to match a defect.
+
+## Contract Contents
+
+Each page contract contains:
+
+1. Purpose and user goals.
+2. Registered routes and navigation context.
+3. List, detail, and form behavior.
+4. Editable and system-managed information.
+5. Validation, empty, loading, and failure states.
+6. A concise acceptance checklist.
+7. Current implementation and test anchors.
+8. Known limitations and deferred work.
+
+## Maintenance Rules
+
+- Describe current accepted behavior in present tense.
+- Do not mix an obsolete “first release” design with current behavior.
+- Keep future changes in versioned scope documents and link to them from a Deferred Work section.
+- Do not reproduce the complete database field inventory here; include only fields that affect page behavior.
+- Keep service, file, and test anchors current.
+- Do not create separate current-state, target-state, and traceability copies of the same contract.
+- Keep cross-page visual rules in the UI Style Guide instead of repeating them on each page.
+- Keep database joins such as `DocumentPerson` and `JobSource` in schema/architecture documentation unless they directly affect a page interaction.
+
+## Current Baseline
+
+These contracts describe the current flattened Version 4 baseline.

docs/ui/pages/documents.md

@@ -0,0 +1,133 @@
+# Documents Page Contract
+
+## Purpose
+
+Documents manages the archival record for each historical artifact independently of its source files and transcription jobs. A Document can be created first, linked to people in one or more roles, and used later as the parent for Sources and Jobs.
+
+## Routes
+
+| Route | Purpose |
+| --- | --- |
+| `/documents` | Searchable archival Document list. |
+| `/documents/new` | Create a Document. |
+| `/documents/{document_id}` | View one Document and its related records. |
+| `/documents/{document_id}/edit` | Edit metadata and the complete Linked People set. |
+| `/documents/{document_id}/delete` | Confirm or block deletion. |
+| `/documents/{document_id}/jobs` | Show Jobs belonging to the Document. |
+| `/documents/{document_id}/sources` | Redirect to the Document-filtered Sources list. |
+| `/documents/{document_id}/print` | Preview and browser-print the persisted Document. |
+
+## List Behavior
+
+- The title is **Archival Documents**.
+- **Create new document** opens the create route.
+- The table defaults to Document Title order and supports search and column sorting.
+- Columns are Document Title, Type, Author, Document Date, and # Sources.
+- Document Title is left-aligned; the remaining columns are centered.
+- Author lists all linked people in the `author` role.
+- # Sources reflects the count of linked Source rows for each Document.
+- Date display prefers exact date, then approximate date, then `Unknown`.
+- Selecting a row opens Document Detail.
+- No records displays `No documents found in repository.`
+
+## Create and Edit Behavior
+
+Required:
+
+- Document name.
+- Document type selected from the Document Type registry.
+
+Optional:
+
+- Exact date.
+- Approximate date.
+- Document location.
+- Archive identifier.
+- Notes.
+- Tags.
+- Linked People, with exactly one Person Role per linked Person.
+
+Rules:
+
+- Exact date must parse as `YYYY-MM-DD`; browser presentation may follow locale.
+- Existing people appear with disambiguating labels.
+- Tag assignment supports selecting existing tags and adding new labels inline.
+- **Create new person** opens Person creation.
+- `person_id` may preselect that Person in the author role on Document creation.
+- An invalid requested Person produces a warning rather than a broken form.
+- `return_to=jobs_new` returns a successful create to Job creation with the new Document selected.
+- Edit includes active and inactive Document Types so historical values remain maintainable.
+- One Linked People table contains Select, Person, and Role columns.
+- Add and Edit use an inline Person/Role editor; Save, Cancel, and Delete change staged UI state only.
+- A Person may appear once per Document regardless of role.
+- Existing inactive-role links remain visible; only active roles may be newly assigned.
+- Document fields and the complete staged link set commit atomically on the main save.
+- Save success returns to Document Detail.
+
+## Detail Behavior
+
+- The heading shows name, type, and internal ID.
+- The first Source, when present, appears in the dark-room viewer.
+- Archival Metadata shows authors, Document Type, tags, compact Document date, location, and archive identifier. Notes appear in a separate archival-notes block within the same card.
+- System Logistics shows created and updated timestamps.
+- Related People are grouped by role and link to Person Detail.
+- **Sources & Pipeline Jobs** shows counts and actions for filtered Sources, Document Jobs, and adding a Job.
+- **Edit Document**, **Print**, and **Delete** are available from the header.
+- Invalid IDs and missing Documents produce explicit states without rendering a partial page.
+
+## Print Behavior
+
+- Print opens a dedicated preview for persisted Document data.
+- **Facsimile** places each Source image beside its current transcription and starts every Source on a new printed sheet.
+- **Text only** omits images, joins single line breaks inside paragraphs, and preserves blank-line paragraph boundaries.
+- Non-null revised text takes precedence over raw transcription, including an intentionally empty revision.
+- Archival metadata resolves Author through the hidden built-in semantic identity, not its mutable label.
+- Archival metadata includes the Document Type label.
+- Metadata tables use a narrow non-wrapping label column and wider wrapping data columns rather than stretching across the page.
+- Job metadata uses one oldest-to-newest column per Job and ends with Status.
+- Stored text is escaped and Source media uses record-validated application URLs rather than local file paths.
+- Printing uses the browser print dialog; server-generated PDFs are not provided.
+
+## Document Jobs Behavior
+
+- The page lists the Document's Jobs newest first with status and Job ID.
+- **Open Job** navigates to Job Detail.
+- **Create Job** opens Job creation with the Document selected.
+- No jobs displays an explicit empty state.
+
+## Delete Behavior
+
+- Deletion is blocked while any Source or Job belongs to the Document.
+- The blocked state names the dependency categories and provides navigation back and to Jobs.
+- An unlinked Document requires an explicit permanent-delete action.
+- Success returns to the Documents list.
+
+## Acceptance Checklist
+
+- List columns, alignment, search, sorting, date fallback, and row navigation match this contract.
+- Create/edit enforce name, registered type, and valid exact-date input.
+- Linked People staging enforces one role and one row per Person.
+- Document and Linked People writes never partially commit.
+- Person-first Document creation preselects the requested Person as author.
+- Detail links people, Sources, and Jobs to the correct records.
+- Delete never removes a Document with Source or Job dependencies.
+- Both print formats preserve the frozen content, ordering, text-precedence, and safety contracts.
+- Service failures use the shared error presenter and never report false success.
+
+## Implementation Anchors
+
+- `src/transcription/ui/pages/documents_page.py`
+- `src/transcription/ui/components/table/documents.py`
+- `src/transcription/services/documents.py`
+- `src/transcription/services/people.py`
+- `src/transcription/services/workflows.py`
+- `src/transcription/ui/components/linked_people.py`
+- `src/transcription/ui/pages/print_preview_page.py`
+- `src/transcription/api/print_api.py`
+- `tests/ui/test_documents_page.py`
+- `tests/services/test_document_service.py`
+
+## Known Limitations and Deferred Work
+
+- Source page ordering remains read-only in V4.4.
+- Printing other entities, batch printing, and server-side export formats are deferred.

docs/ui/pages/home.md

@@ -0,0 +1,64 @@
+# Home Page Contract
+
+## Purpose
+
+Home provides a user-maintained landing page for the local archive. It combines a database-backed image gallery with Markdown text and lets the operator edit both without changing application source or prompt assets.
+
+## Routes
+
+| Route | Browser path | Purpose |
+| --- | --- | --- |
+| `/homepage` | `/ui/homepage` | View homepage gallery and Markdown. |
+| `/homepage/edit` | `/ui/homepage/edit` | Upload images, manage image metadata, and edit Markdown. |
+
+The application root and `/ui` redirect to `/ui/homepage`.
+
+## View Behavior
+
+- The visible page heading is **Home**; the browser tab title is **VibeScribe Home**.
+- The featured homepage image (`photo.is_primary`) is shown first; remaining images are shown in random order.
+- The current image appears in the shared dark-room viewer with its description.
+- Saved Markdown is rendered in the **Home Text** card.
+- Missing text displays `No homepage text saved yet.`
+- Missing image displays the viewer's empty state.
+- **Edit Home Page** opens the edit route.
+- The same Home Text content is also editable from **Settings → Home Page Text**.
+
+## Edit Behavior
+
+- The image upload accepts JPEG, PNG, GIF, WebP, BMP, and TIFF files and supports multi-file uploads.
+- A successful upload immediately stores files in the shared `photo` table/media layout and displays a positive notification.
+- The editor supports per-image description edits, setting a featured image, and deleting the current image.
+- The Markdown textarea is initialized from the currently stored homepage text.
+- **Save** writes the textarea content, displays `Homepage saved`, and returns to Home.
+- **Cancel** returns to Home without saving textarea changes. An image already uploaded during the edit session remains stored.
+
+## Storage Contract
+
+- Homepage markdown text is mutable application data at `UPLOAD_DIR/homepage.md`.
+- Homepage images are stored as `photo` rows (`person_id = NULL`) with files under `UPLOAD_DIR/photos/`.
+- Uploaded images are renamed to `{photo_id}{suffix}`.
+- Homepage images are database records; markdown remains file-backed.
+
+## Acceptance Checklist
+
+- `/`, `/ui`, and the application brand reach Home.
+- Home renders with or without stored Markdown and image content.
+- Edit loads existing Markdown.
+- Supported image upload stores one or more images and makes the first image featured when no featured image exists yet.
+- Save persists Markdown and returns to Home.
+- Cancel does not save changed Markdown.
+
+## Implementation Anchors
+
+- `src/transcription/ui/pages/home_page.py`
+- `src/transcription/ui/homepage_store.py`
+- `src/transcription/ui/components/app_shell.py`
+- `tests/ui/test_upload_page.py`
+- `tests/ui/test_navigation_and_mounts.py`
+- `tests/ui/test_pages_registration.py`
+
+## Known Limitations
+
+- Homepage markdown storage location is `UPLOAD_DIR/homepage.md` and must remain writable in the active runtime environment.
+- Uploading an image is immediate and is not rolled back by Cancel.

docs/ui/pages/jobs.md

@@ -0,0 +1,97 @@
+# Jobs Page Contract
+
+## Purpose
+
+Jobs manages transcription processing runs. A Job belongs to one Document, links one or more Source pages, records processing provenance, and exposes lifecycle actions without making lifecycle fields directly editable.
+
+## Routes
+
+| Route | Purpose |
+| --- | --- |
+| `/jobs` | Searchable processing Job list. |
+| `/jobs/new` | Create and queue a Job. |
+| `/jobs/{job_id}` | View status, execution logistics, and related records. |
+| `/jobs/{job_id}/cancel` | Confirm cancellation. |
+| `/jobs/{job_id}/resubmit` | Confirm resubmission of failed Sources. |
+| `/jobs/{job_id}/delete` | Confirm or block deletion. |
+
+## List Behavior
+
+- The title is **Transcription Pipeline Jobs**.
+- **Create job** opens Job creation and **Refresh** reloads the table.
+- Columns are Job ID, Status, Document Name, # Sources, Retries, and Updated.
+- Updated is the primary date/sort field.
+- Search covers Job ID, document name, and status.
+- Status is displayed as a semantic status chip.
+- Selecting a row opens Job Detail.
+- No records displays `No job records found in repository.`
+
+## Create Behavior
+
+- A Target Document and at least one source file are required.
+- `document_id` may preselect a Target Document.
+- If no Documents exist, the page explains the prerequisite and links to Document creation with a return path.
+- Provider and Model are selectable when creating a new Job.
+- Upload accepts JPEG, PNG, TIFF, and PDF files and supports multiple/folder selection.
+- The visible upload queue is sorted alphabetically by original filename.
+- Files can be removed individually or cleared before submission.
+- Helper text explains numeric filename prefixes for page ordering.
+- Submission creates the Job, Source records, and JobSource links, notifies the worker, and opens Job Detail.
+- When opened with `source_id`, creation becomes a retranscription flow: Source and Document are locked, Provider is
+  read-only, Model is restricted to `PROVIDER_MODELS`, no upload is accepted, and one existing Source is linked.
+
+## Detail and Lifecycle Behavior
+
+- The heading shows Job ID and a status badge.
+- Execution Logistics shows provider, model, prompt, retry count, and last update.
+- Document Links show a clickable Document Name, Sources count, and a single **View Sources** action using document filtering.
+- Queued and processing Jobs show an auto-refresh notice and reload every four seconds.
+- Polling stops when the Job becomes terminal or a refresh fails.
+- Queued and processing Jobs expose **Cancel**.
+- Jobs other than `transcribed` expose **Resubmit** under the current UI rule. The service blocks resubmission while processing is active or when no failed Sources exist.
+- All Jobs expose **Delete Job**, subject to explicit evidence-deletion guardrails.
+- Invalid and missing IDs produce explicit states.
+
+## Cancel Behavior
+
+- The confirmation explains that processing stops and remaining pending Sources become cancelled.
+- The service decides whether the current state permits cancellation.
+- Success updates the Job, notifies the worker, and returns to Job Detail.
+
+## Resubmit Behavior
+
+- The page shows current status and failed Source count.
+- The page explains that resubmission queues failed linked Sources while preserving immutable prior attempt evidence.
+- The service blocks submission while processing is active or when no failed Sources exist.
+- `JobSource` remains the latest compatibility projection, while every provider call appends an `ExecutionAttempt`.
+- The selected `Source.raw_transcription` projection remains available while a retry is pending or fails.
+- Success reports the number of resubmitted Sources and returns to Job Detail.
+
+## Delete Behavior
+
+- Deletion is blocked while status is `processing`.
+- Allowed deletion explicitly warns that related `JobSource` projections,
+  immutable execution attempts, captured transport responses, and attempt-owned
+  artifacts are permanently removed.
+- Source records and source files remain available for separate deletion.
+- Success returns to the Jobs list.
+
+## Acceptance Checklist
+
+- Job creation cannot proceed without a valid Document and at least one Source.
+- Upload ordering and removal controls match the displayed queue.
+- Detail shows current status and provenance summary with correct related links.
+- Active Jobs refresh without overlapping permanent polling after terminal state.
+- Cancel, resubmit, and delete honor service guardrails and show actionable failures.
+- Lifecycle fields cannot be edited directly.
+
+## Implementation Anchors
+
+- `src/transcription/ui/pages/jobs_page.py`
+- `src/transcription/ui/components/table/jobs.py`
+- `src/transcription/services/jobs.py`
+- `src/transcription/services/store.py`
+- `src/transcription/services/workflows.py`
+- `tests/ui/test_jobs_page.py`
+- `tests/services/test_job_service.py`
+- `tests/services/test_store.py`

docs/ui/pages/people.md

@@ -0,0 +1,95 @@
+# People Page Contract
+
+## Purpose
+
+People manages reusable historical-person records. A Person may appear in many Documents under different relationship roles and may optionally carry one or more photos plus a FamilySearch identifier.
+
+## Routes
+
+| Route | Purpose |
+| --- | --- |
+| `/people` | Searchable People list. |
+| `/people/new` | Create a Person. |
+| `/people/{person_id}` | View one Person and linked Documents. |
+| `/people/{person_id}/edit` | Edit the Person. |
+| `/people/{person_id}/delete` | Confirm permanent deletion. |
+
+## List Behavior
+
+- The title is **Archival Entities: People**.
+- **Create new person** opens the create route.
+- The table defaults to Full Name order and supports search and column sorting.
+- Columns are Full Name, FamilySearch ID, Birth Date, Death Date, and # Documents.
+- Full Name is left-aligned; FamilySearch ID, date columns, and # Documents are centered.
+- # Documents reflects how many linked Documents each Person is connected to.
+- Birth and death values independently prefer exact date, then approximate date, then `Unknown`.
+- Selecting a row opens Person Detail.
+- No records displays `No person records found in repository.`
+
+## Create and Edit Behavior
+
+Required:
+
+- Full name.
+
+Optional:
+
+- Display name and maiden name.
+- Exact and approximate birth/death dates.
+- Birth/death places.
+- Biography.
+- FamilySearch ID.
+
+Rules:
+
+- Missing Full name blocks save with a warning.
+- Exact date inputs are native browser date inputs.
+- FamilySearch IDs are normalized and validated by `PeopleService`.
+- Photos are managed on Person Detail (not in create/edit form fields).
+- Metadata JSON remains hidden.
+- Save success returns to Person Detail.
+
+## Detail Behavior
+
+- The header provides **New Document**, **Edit Person**, and **Delete**.
+- **New Document** opens Document creation with this Person requested for author preselection.
+- Person Detail includes a photo gallery card with multi-file upload, per-photo description edits, set-primary, and delete.
+- Primary photo is shown first and labeled as the primary portrait.
+- Biographical Record shows names, compact birth/death dates, and places.
+- Birth and death place values are clickable links to Google Maps when present.
+- FamilySearch ID is shown as a metadata value and is clickable to the FamilySearch person details route when present.
+- Maiden Name is only shown in Biographical Record when a value exists.
+- Biography has an explicit empty value.
+- Linked Documents render as a table with **Document Name**, **Role**, and **Number of Pages**; selecting a row opens Document Detail.
+- No links shows both an empty state and guidance to link from a Document workflow.
+- System Logistics shows created and updated timestamps.
+
+## Delete Behavior
+
+- The page warns when linked Document relationships exist.
+- Delete is blocked when related Photos exist.
+- Confirmed deletion removes the Person and its relationship links; it does not delete Documents.
+- Success returns to the People list.
+- Missing or already-deleted records return to a safe list state.
+
+## Acceptance Checklist
+
+- List fields, alignment, date fallback, search, sorting, and navigation match this contract.
+- Full name is enforced on create and edit.
+- FamilySearch ID validation and link generation use the fixed supported identifier format.
+- Photo upload and rendering remain constrained to supported media paths.
+- New Document carries the Person context.
+- Linked Documents show the correct role and target.
+- Delete wording distinguishes removal of relationship links from deletion of Documents.
+
+## Implementation Anchors
+
+- `src/transcription/ui/pages/people_page.py`
+- `src/transcription/ui/components/table/people.py`
+- `src/transcription/services/people.py`
+- `tests/ui/test_people_page.py`
+- `tests/services/test_v2_crud.py`
+
+## Deferred Work
+
+- Structured name fields, merge/deduplication, advanced metadata editing, and Person-side relationship editing are not current behavior.

docs/ui/pages/settings.md

@@ -0,0 +1,36 @@
+# Settings Page Contract
+
+## Purpose
+
+Settings manages installation-local registries and editable text assets from one route.
+
+## Route
+
+| Route | Purpose |
+| --- | --- |
+| `/settings` | Manage Document Types, Person Roles, Tags, Prompts, and Home Page Text. |
+
+## Behavior
+
+- The page title is **Settings**.
+- Configuration surfaces are grouped as tabs:
+  - **Document Types**
+  - **Person Roles**
+  - **Tags**
+  - **Prompts**
+  - **Home Page Text**
+- Document Types, Person Roles, and Tags support Add/Edit/Delete with existing guardrails.
+- Prompts exposes only `transcribe_document.md` for editing and restore-from-backup.
+- Home Page Text edits the same Markdown content rendered on `/homepage`.
+
+## Acceptance Checklist
+
+- `/ui/settings` renders all five tabs.
+- Registry and prompt workflows keep existing validation and error handling.
+- Saving Home Page Text persists content for the homepage view.
+
+## Implementation Anchors
+
+- `src/transcription/ui/pages/settings_page.py`
+- `src/transcription/ui/homepage_store.py`
+- `tests/ui/test_pages_registration.py`

docs/ui/pages/sources.md

@@ -0,0 +1,98 @@
+# Sources Page Contract
+
+## Purpose
+
+Sources manages individual archived page/file records. It provides source-media viewing, current processing context, provider evidence inspection, previous/next page navigation, and human revision without allowing machine output to be edited.
+
+## Routes
+
+| Route | Purpose |
+| --- | --- |
+| `/sources` | Global or filtered Source list. |
+| `/sources/{source_id}` | View media, transcription, revision, metadata, and evidence. |
+| `/sources/{source_id}/delete` | Confirm or block deletion. |
+
+The list accepts optional `document_id` and `job_id` query parameters. Document context takes precedence if both parse successfully.
+
+## List Behavior
+
+- The title is **Source Asset Records**, **Sources for Document**, or **Sources for Job** according to context.
+- Global context provides **Create Job**.
+- Filtered context provides **Back to Document** or **Back to Job**.
+- Rows are ordered by page number and then upload name.
+- Columns are Document Name, Page Number, Upload Title, Status, and Error Detail.
+- Document Name, Upload Title, and Error Detail are left-aligned; Status is centered.
+- Stored Filename is intentionally absent from the list.
+- Selecting a row opens Source Detail.
+- No records displays `No source asset records found in repository.`
+
+## Detail Behavior
+
+- The heading shows page number, upload name, and Source ID.
+- **Back to Sources** returns to the global list.
+- **Retranscribe Source** opens Create Processing Job with this Source and its Document locked.
+- **Delete Source** opens the guarded delete route.
+- Previous and Next navigate only among Sources belonging to the same Document in page order; unavailable boundary actions are disabled.
+- The media viewer resolves the stored Source path through the configured upload root.
+- The top layout is adaptive:
+  - Standard pages use three columns with a wider Editable Revision column than the image column.
+  - Wide+narrow landscape images switch to a stacked left layout (image above Editable Revision) with metadata on the right.
+- Editable Revision is seeded from an existing revision or the preferred machine transcription.
+- Source Metadata shows upload name, stored filename, page number, Document Name, Document ID, and stored path. Source ID appears in the page-header subtitle.
+- SourceJob Metadata shows latest status, Job ID, execution time, provider, model, prompt, and failure detail.
+- Revision Logistics shows revised state, last-revised time, and upload time.
+- Candidate Machine Transcriptions appears below the image/revision area, remains compact until expanded, then compares it with the preferred
+  machine result and requires confirmation before **Use this transcription**.
+- Candidate promotion does not alter a human revision. Empty states distinguish no machine result from no candidates.
+- An orientation-normalized artifact appears in evidence only when recognized metadata required a physical rotation.
+
+## Provider Evidence
+
+- Provider Evidence is associated with the latest JobSource execution.
+- New attempts display separate expandable Request Manifest, Transport Response, OpenRouter SDK Response Snapshot,
+  Normalized Metadata, Software Context, and Derived Artifacts sections.
+- Historical `raw_api_response` values are labeled as OpenRouter SDK response snapshots.
+- Missing evidence has an explicit empty state.
+- Historical executions explicitly state that exact transport evidence was not captured.
+- Quality warning artifacts remain attached to their machine attempt and are not recomputed during page rendering.
+- **Export Evidence** downloads a versioned package containing source identity, attempts, artifacts, relationships,
+  schema versions, and integrity digests without source binaries, credentials, or machine-local source paths.
+
+## Revision Behavior
+
+- Machine transcription is never edited directly.
+- A revision must contain non-whitespace text.
+- Save persists revised text and updates the saved timestamp without leaving the page.
+- Reset restores the in-memory revision from page load or the most recent successful save. When no revision exists, it restores the machine transcription; it does not re-read the database.
+- A failed latest execution displays guidance that a human revision can preserve corrected text.
+
+## Delete Behavior
+
+- Deletion is allowed only when the Source has no JobSource links.
+- A linked Source shows cleanup guidance and navigation to Jobs.
+- An unlinked Source requires explicit permanent deletion.
+- Success returns to the Sources list.
+
+## Acceptance Checklist
+
+- Global, Document-filtered, and Job-filtered lists show the correct context and return action.
+- List columns and alignments match this contract and omit Stored Filename.
+- Previous/next navigation never crosses Document boundaries.
+- Detail keeps machine output read-only and human revision separately editable.
+- Retranscription, candidate comparison, warnings, and explicit promotion preserve every prior attempt.
+- Empty, failed, and missing-evidence states remain explicit.
+- JSON evidence is readable without being mislabeled as native transport evidence.
+- Delete cannot remove a Source with processing-history links.
+
+## Implementation Anchors
+
+- `src/transcription/ui/pages/sources_page.py`
+- `src/transcription/ui/components/table/sources.py`
+- `src/transcription/services/sources.py`
+- `tests/ui/test_sources_page.py`
+- `tests/services/test_transcription_service.py`
+- `tests/services/test_v2_crud.py`
+
+## Planned Changes
+
+- Source page reordering is deferred beyond V4.3 and may be reconsidered if a demonstrated workflow need emerges.

docs/ui/pages/tags.md

@@ -0,0 +1,31 @@
+# Tags Page Contract
+
+## Purpose
+
+Tags provides a dedicated browse/filter entry point for document tagging workflows.
+
+## Route
+
+| Route | Purpose |
+| --- | --- |
+| `/tags` | Browse Documents grouped by Tag and filter to one Tag. |
+
+## Behavior
+
+- The page title is **Tags**.
+- When no tags exist, the page shows `No tags are configured yet.`
+- A Tag filter select allows narrowing to one tag.
+- Each rendered group header includes the tag label and document count.
+- Document names are clickable and open Document Detail.
+
+## Acceptance Checklist
+
+- `/ui/tags` renders successfully from the main navigation.
+- Group counts match the number of linked Documents per Tag.
+- Filtering hides non-matching tag groups.
+
+## Implementation Anchors
+
+- `src/transcription/ui/pages/tags_page.py`
+- `src/transcription/services/documents.py`
+- `tests/ui/test_tags_page.py`

docs/ver1/architecture_v1.md

@@ -1,308 +0,0 @@
- # System Architecture (Version 1)
-
-This document describes the production architecture of the personal historical-document transcription system. The system is intentionally optimized for single-user operation, low operational overhead, and clean internal boundaries that support future growth without rewrites.
-
-## Architecture Objectives
-
-The production architecture is designed to:
-
-- preserve verbatim family-history source material as searchable text
-- keep operational complexity low for a personal deployment
-- support asynchronous transcription without requiring distributed infrastructure
-- maintain clear module boundaries so extensions can be added incrementally
-
-## Production Scope And Scale
-
-The deployed system targets personal use and a corpus of several thousand documents processed over time. The architecture favors simple, composable building blocks over distributed orchestration.
-
-Current scope includes:
-
-- content source upload and metadata capture
-- asynchronous transcription jobs
-- prompt-library driven transcription behavior, with one Markdown file per prompt
-- original transcription review and optional revision review
-- full-text search over accepted transcripts
-- export of transcript data
-
-## Deployment Topology
-
-The production deployment uses [Docker Compose](https://docs.docker.com/compose/) and treats containerized databases as extremely lightweight operational dependencies.
-
-Running [PostgreSQL](https://www.postgresql.org/docs/) in its own container is considered simple by default for this system.
-
-Running [MongoDB](https://www.mongodb.com/docs/) in its own container is also considered simple when document-centric storage is enabled.
-
-Container count is not a hard architectural limit; a three-container deployment (app, PostgreSQL, MongoDB) is an acceptable baseline.
-
-### Baseline Topology (Two Containers)
-
-- one application container
-- one PostgreSQL container
-- embedded background worker execution inside the app process
-
-### Expanded Topology (Three Containers)
-
-- application container
-- PostgreSQL container
-- MongoDB container
-
-No additional queue, scheduler, or search-engine containers are required in the baseline production setup.
-
-## Runtime Architecture
-
-```mermaid
-flowchart LR
-    User[Browser User] --> App[FastAPI + NiceGUI Service]
-    App --> Worker[In-process Background Worker]
-    App --> PG[(PostgreSQL)]
-    App --> MG[(MongoDB Document Store)]
-    Worker --> AI[Transcription Provider]
-    Worker --> PG
-    Worker --> MG
-```
-
-## Runtime Ownership And Startup Policy
-
-The current implementation now uses explicit lifespan-owned runtime resources.
-
-- application lifespan initializes and disposes database runtime resources
-- worker lifecycle is owned by application lifespan startup/shutdown
-- worker receives lifespan-owned database engine dependency explicitly
-- schema bootstrap policy is environment-aware and explicit:
-  - development/test default to bootstrap enabled
-  - production defaults to bootstrap disabled
-  - explicit override is available via configuration
-
-This aligns implementation toward REQ-7 and REQ-10 while preserving personal-scale operational simplicity.
-
-## Layered Module Structure
-
-### Interface Layer
-
-Responsibility:
-
-- HTTP API and UI routes
-- request/response validation
-- status and result presentation
-
-Out of scope:
-
-- business-rule enforcement
-- data-access implementation
-
-### Application Layer
-
-Responsibility:
-
-- upload and job orchestration
-- state transitions and retry policy
-- coordination across domain and infrastructure ports
-
-Out of scope:
-
-- provider-specific protocol details
-- ORM or storage-specific logic
-
-### Domain Layer
-
-Responsibility:
-
-- verbatim transcription policy
-- revision and provenance invariants
-- confidence and annotation semantics
-
-Out of scope:
-
-- web framework concerns
-- database and network I/O
-
-### Infrastructure Layer
-
-Responsibility:
-
-- persistence adapters (PostgreSQL and MongoDB)
-- transcription-provider adapter
-
-Out of scope:
-
-- business policy decisions
-
-## Processing Workflow
-
-Production transcription flow:
-
-1. A user uploads one or more content sources through the UI or API.
-2. The application validates payloads and creates document, source, and job records.
-3. The in-process worker de-queues the job and calls the transcription provider.
-4. The application persists original transcription output on the job, plus confidence metadata and provenance events.
-5. Job status transitions from queued to processing to transcribed or failed.
-6. The UI and API expose status, optional revision to original transcription, and searchable transcription text.
-
-## Data Model Ownership
-
-System-of-record entities:
-
-- documents and content sources
-- transcription jobs, original transcription, and status events
-- transcript revisions
-- provenance metadata
-
-### Original Transcription And Revision Ownership
-
-- each processing job stores the original immutable provider output (`text`)
-- provider metadata (`provider`, `model`, `prompt_name`) and failure detail (`error_detail`) are job-owned processing artifacts
-- revisions are optional user-authored edits linked to a content source
-- a revision can be created from original `job.text`
-- many jobs will have zero revisions; revisions are additive and never overwrite original provider output
-- a document groups one or more content sources (images, PDFs, and future source types)
-
-Storage strategy:
-
-- PostgreSQL for relational system-of-record entities
-- MongoDB for document-oriented payloads and large transcription artifacts
-- versioned prompt artifacts stored as individual Markdown files for human editing and refinement
-- in-memory execution state treated as ephemeral
-
-## Transcription Prompt Asset Policy
-
-The production system treats transcription prompts as maintainable content assets.
-
-- each transcription prompt is stored in its own Markdown file
-- prompt files are designed for direct human editing and iterative refinement
-- prompt updates are independent and do not require bundling unrelated prompt changes
-- prompt file identity and revision history are tracked through normal repository version control
-
-## Simplicity Guardrails
-
-The production system enforces these constraints to prevent accidental over-engineering:
-
-- PostgreSQL in a container is treated as a lightweight default dependency
-- MongoDB in a container is treated as a lightweight optional dependency
-- three containers (app, PostgreSQL, MongoDB) is an acceptable simple deployment
-- no dedicated queue or search cluster is introduced without measured need
-- external infrastructure is added only behind existing ports/adapters
-
-## Extension Path
-
-The architecture supports additive growth without changing domain contracts.
-
-### Stage 1: Foundation (Current)
-
-- upload, transcription, review, search, export
-- in-process worker execution
-- single provider adapter
-- app plus PostgreSQL deployment
-
-### Stage 2: Throughput Hardening
-
-- optional MongoDB document-store enablement
-- optional external worker/queue process
-- stronger retry and dead-letter handling
-
-### Stage 3: Intelligence Features
-
-- entity extraction and cross-document linking
-- timeline and narrative assembly
-- optional multi-provider routing
-
-Each stage preserves existing module boundaries and keeps migration risk low.
-
-## Test Strategy
-
-The test strategy is aligned to personal-scale operation with fast, deterministic feedback.
-
-### Unit Tests
-
-- domain transcription rules and annotation behavior
-- revision-history invariants
-- job state-transition logic
-
-### Integration Tests
-
-- repository behavior and transaction boundaries
-- persistence-adapter and provider adapter contract mapping
-- upload-to-persistence roundtrip
-
-### End-to-End Tests
-
-- happy path: upload, transcribe, review, search, export
-- failure path: provider error, retry, surfaced failed status
-
-### CI Execution Model
-
-- fast suite on each push
-- optional slower provider-sandbox checks on scheduled runs
-
-## Risks And Controls
-
-### Runtime Responsiveness
-
-Risk:
-
-- long jobs can reduce responsiveness in a single-process deployment
-
-Control:
-
-- bounded concurrency and visible job status in the UI
-
-### Database Concurrency Limits
-
-Risk:
-
-- contention can appear under sustained concurrent writes in personal-scale infrastructure
-
-Control:
-
-- tuned connection pooling and phased use of MongoDB for document-heavy workloads
-
-### Provider Output Variance
-
-Risk:
-
-- transcription quality varies by content source type, handwriting legibility, and source quality
-
-Control:
-
-- first-class human review and immutable revision history
-
----
-
-## Technology References
-
-- [FastAPI documentation](https://fastapi.tiangolo.com/)
-- [NiceGUI documentation](https://nicegui.io/documentation)
-- [Docker Compose documentation](https://docs.docker.com/compose/)
-- [PostgreSQL documentation](https://www.postgresql.org/docs/)
-- [MongoDB documentation](https://www.mongodb.com/docs/)
-
-## Related Local References
-
-- [System Overview](index_v1.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- System Architecture (this document)
-- [System Requirements](requirements_v1.md)
-- [Data model](schema_v1.md)
-- [Error Handling Policy](error_handling_v1.md)
-- [Implementation Plan](implementation_plan_v1.md)
-
-## Glossary
-
-- Adapter: A component that translates between internal interfaces and external systems such as databases or AI services.
-- Background job: Work executed outside the request/response path so the UI remains responsive.
-- Boundary: A strict separation between modules with different responsibilities.
-- CI (Continuous Integration): Automated test execution for code changes.
-- Contract test: A test that verifies an adapter follows expected input/output behavior at a boundary.
-- Domain layer: The module that contains core business rules and invariants.
-- End-to-end test: A test that validates a full user flow across the running system.
-- Full-text search: Text indexing and querying optimized for natural-language search.
-- In-process worker: A background executor that runs within the same application process.
-- Integration test: A test that verifies interactions between real modules and infrastructure components.
-- MongoDB: A document-oriented database used for flexible, high-variance data structures.
-- Modular monolith: A single deployable application with strongly separated internal modules.
-- Port/Interface: A stable contract used by application/domain code to call infrastructure implementations.
-- Prompt artifact: A single Markdown file that defines one transcription prompt and can be revised independently.
-- Provenance: Metadata that records where generated data came from and how it was produced.
-- Revision history: Optional versioned record of user-authored transcription edits over time.
-- System of record: The authoritative persistent store for canonical data.
-- Vertical slice: A minimal end-to-end feature path spanning UI/API, application logic, and persistence.

docs/ver1/error_handling_v1.md

@@ -1,288 +0,0 @@
-# Error Handling Policy
-
-This document defines the canonical error-handling policy for the document transcription system. It is the single source of truth for how errors are classified, surfaced to users, logged for diagnosis, and handled across UI, API, service, worker, and provider boundaries.
-
-## Error Handling Objectives
-
-The production error-handling model is designed to:
-
-- make failures visible to the user in clear, actionable language
-- preserve enough diagnostic detail for fast troubleshooting
-- keep module behavior consistent across all boundaries
-- distinguish expected domain failures from unexpected defects
-- support safe retries for transient failures without hiding persistent faults
-
-## Scope And Authority
-
-This page governs error-handling behavior for:
-
-- UI interactions (NiceGUI pages)
-- API endpoints (FastAPI routes)
-- application services and orchestration logic
-- in-process background worker execution
-- external provider adapters and persistence adapters
-
-If implementation behavior conflicts with this document, this document is authoritative and implementation should be updated.
-
-## Core Principles
-
-- **Clarity first:** user-facing messages should explain what failed in plain language.
-- **Actionability required:** each surfaced error should include a suggested next step.
-- **Safety by default:** internal details are logged; sensitive details are not exposed by default in UI/API.
-- **Consistency across boundaries:** category and structure should remain stable from source to surface.
-- **Fail explicitly:** silent failure is prohibited.
-- **Traceability:** every non-trivial error should be traceable with an error reference ID.
-
-## Error Taxonomy
-
-The system uses stable, implementation-independent categories:
-
-| Category | Definition | Typical Source | Retriable |
-| --- | --- | --- | --- |
-| `validation_error` | Payload or parameter shape/content is invalid | UI/API input validation, service guards | no |
-| `user_input_error` | User-provided artifact is unacceptable though structurally valid | unsupported file type, empty file, oversized upload | sometimes |
-| `not_found_error` | Requested resource does not exist | missing job/document/source/revision | no |
-| `conflict_error` | Requested operation violates current state constraints | invalid state transition | no |
-| `external_provider_error` | External AI/provider call fails | upstream HTTP/API/provider failures | sometimes |
-| `infrastructure_transient_error` | Temporary environment issue | network timeout, DB connection reset | yes |
-| `infrastructure_persistent_error` | Non-transient environment issue | missing permissions, misconfiguration | no |
-| `internal_unexpected_error` | Unhandled defect or unknown failure | uncaught exceptions, logic errors | unknown (default no) |
-
-### Classification Rules
-
-- Classification occurs as close as possible to the origin boundary.
-- Provider-specific exceptions must be normalized into taxonomy categories before crossing service boundaries.
-- Unknown exceptions are classified as `internal_unexpected_error` and logged with traceback.
-- Category names are stable contracts and must not be changed casually.
-
-## User-Facing Error Experience Contract
-
-When an error is shown in the GUI, it must include:
-
-1. **Title** (short context, e.g., “Upload failed”)
-2. **Message** (plain-language explanation)
-3. **Suggested action** (explicit next step)
-4. **Error reference ID** (for support/debug traceability)
-5. **Technical details** (optional/collapsible for advanced users)
-
-### UI Message Rules
-
-- Do not expose raw stack traces by default.
-- Do not expose secrets, credentials, connection strings, or filesystem internals unless explicitly in debug tooling.
-- Prefer domain language over implementation language.
-- Use persistent visibility for important failures (dialog/card), not only transient toasts.
-
-### Suggested Action Requirements
-
-Every user-visible error must include a suggested course of action, such as:
-
-- retry the operation
-- check file type/size constraints
-- refresh the jobs page
-- verify environment configuration
-- contact operator with error ID and timestamp
-
-## API Error Response Contract
-
-API errors should return a structured envelope with stable fields:
-
-- `error_id`: short unique reference ID
-- `category`: taxonomy category
-- `message`: safe human-readable summary
-- `suggestion`: recommended next step
-- `details`: optional, only when safe and appropriate
-- `timestamp`: UTC ISO-8601
-
-HTTP status mapping guidance:
-
-- `validation_error`, `user_input_error` -> `400`
-- `not_found_error` -> `404`
-- `conflict_error` -> `409`
-- `external_provider_error` -> `502` or `503` (depending on failure mode)
-- `infrastructure_transient_error` -> `503`
-- `infrastructure_persistent_error` -> `500`
-- `internal_unexpected_error` -> `500`
-
-## Logging And Observability Contract
-
-All logged errors must include, where available:
-
-- `error_id`
-- `category`
-- `operation` (e.g., `upload.submit`, `worker.process_job`, `jobs.refresh`)
-- `exception_type`
-- `job_id`, `document_id`, `source_id` (when relevant)
-- UTC timestamp
-
-Rules:
-
-- Use structured logging fields where practical.
-- Use full traceback for unexpected errors (`internal_unexpected_error`).
-- Log at boundary handoff points to preserve causal trail.
-- Avoid duplicate noisy logging for the same exception at every layer.
-
-## Recovery And Retry Policy
-
-### Retriable Conditions
-
-Retriable failures include:
-
-- transient network/provider timeouts
-- intermittent provider unavailability
-- temporary DB/network interruptions
-
-### Non-Retriable Conditions
-
-Non-retriable failures include:
-
-- invalid file formats
-- missing required data
-- permission/configuration failures
-- deterministic domain conflicts
-
-### Worker Behavior
-
-- The worker must classify and persist failure details consistently.
-- Retries should be bounded by configured limits.
-- Exhausted retries must end in explicit failed status with recorded reason.
-- No infinite retry loops are allowed.
-
-## Boundary-Specific Responsibilities
-
-### UI Layer
-
-Responsibility:
-
-- display user-safe error summaries and suggested actions
-- show persistent error visibility for critical failures
-- include error reference IDs in visible output
-
-Out of scope:
-
-- low-level exception parsing
-- provider-specific protocol interpretation
-
-### API Layer
-
-Responsibility:
-
-- map application exceptions into stable error envelopes and HTTP statuses
-- preserve category and error_id continuity
-
-Out of scope:
-
-- domain-specific remediation logic
-
-### Service Layer
-
-Responsibility:
-
-- classify domain and infrastructure exceptions
-- convert adapter-specific failures into taxonomy categories
-- return deterministic error types to callers
-
-Out of scope:
-
-- presentation formatting for UI
-
-### Worker Layer
-
-Responsibility:
-
-- execute retry policy for retriable failures
-- persist terminal failure details for jobs
-- emit operational logs with category and identifiers
-
-Out of scope:
-
-- direct UI messaging
-
-### Provider Adapter Layer
-
-Responsibility:
-
-- normalize provider SDK/HTTP failures into domain-neutral exceptions
-- preserve raw provider context for logs (safely)
-
-Out of scope:
-
-- choosing user-facing wording
-
-## Error Lifecycle Workflow
-
-Standard lifecycle:
-
-1. Failure occurs at a boundary or operation.
-2. Exception is classified into taxonomy category.
-3. `error_id` is created (or propagated).
-4. Error is logged with required structured fields.
-5. User/API receives safe message + suggested action.
-6. Persistent job/resource state is updated when applicable.
-7. Tests verify contract behavior for the pathway.
-
-## Test Strategy For Error Handling
-
-### Unit Tests
-
-- category classification behavior
-- retry eligibility decisions
-- exception-to-message mapping safety
-
-### Integration Tests
-
-- UI pathways show clear message + suggested action for known failures
-- API returns structured error envelope with expected status/category
-- worker persists failed status and failure detail as required
-
-### Regression Tests
-
-- each previously observed production issue should have a guarding test
-- contract tests must cover adapter error normalization behavior
-
-## Known Failure Patterns And Prescribed Responses
-
-| Pattern | Category | User Message | Suggested Action |
-| --- | --- | --- | --- |
-| Upload payload cannot be parsed by UI handler | `internal_unexpected_error` (until narrowed) | Upload failed due to unexpected processing error | Retry once; if repeated, report error ID and check runtime version compatibility |
-| Unsupported extension | `user_input_error` | File type is not supported | Upload JPG, PNG, TIFF, or PDF |
-| Empty file upload | `validation_error` | Uploaded file is empty | Choose a valid non-empty file and retry |
-| Provider timeout | `external_provider_error` or `infrastructure_transient_error` | Transcription provider timed out | Retry from jobs page; if repeated, check provider status |
-| Job lookup missing | `not_found_error` | Requested job was not found | Refresh jobs list and open a valid job |
-
-## Governance And Update Process
-
-This document is a living policy artifact.
-
-Update this document when:
-
-- new error categories are introduced
-- handling behavior changes at any boundary
-- a production incident reveals missing guidance
-- API/UI error contracts change
-
-Change requirements:
-
-- update this document and associated tests in the same change set
-- preserve taxonomy stability; if changed, document migration impact
-- record noteworthy policy changes in project release notes or changelog
-
----
-
-## Related Local References
-
-- [System Overview](index_v1.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v1.md)
-- [System Requirements](requirements_v1.md)
-- [Data model](schema_v1.md)
-- Error Handling Policy (this document)
-- [Implementation Plan](implementation_plan_v1.md)
-
-## Glossary
-
-- Error category: Stable classification used to drive handling, messaging, and status mapping.
-- Error envelope: Structured API payload describing a failure.
-- Error reference ID: Short identifier used to correlate user-visible failure with logs.
-- Retriable error: Failure likely to succeed on a later attempt without code changes.
-- Terminal failure: Failure state after retries are exhausted or retry is not allowed.

docs/ver1/implementation_plan_v1.md

@@ -1,203 +0,0 @@
-# Version 1 Implementation Plan
-
-This plan defines the path from current implementation to **Version 1 complete**, aligned to the updated domain model:
-
-- `Document` groups one or more content `Source` records
-- `Job` owns original immutable provider output (`text`) and processing metadata
-- `Revision` stores optional user-authored edits linked to a `Source`
-
-The objective is to complete V1 scope with production readiness while keeping non-V1 enhancements out of active delivery.
-
----
-
-## V1 Completion Definition
-
-V1 is complete when all of the following are true:
-
-1. **Functional complete**
-   - Upload, queue, processing, status display, and transcription result inspection work end-to-end.
-   - Optional revision workflow is implemented (create/view/update single revision).
-2. **Data-model complete**
-   - Runtime behavior, persistence, and tests all align to `Document` / `Source` / `Job` / `Revision`.
-3. **Operational complete**
-   - Error handling, logs, and runbooks support reliable operation.
-4. **Documentation complete**
-   - Architecture, requirements, schema, error handling, and index are consistent and current.
-
----
-
-## Phase 1 — Data Contract Stabilization (Schema-First)
-
-**Goal:** Lock a single canonical contract before further feature work.
-
-### Tasks
-1. Confirm and document invariants:
-   - `Job.text` is original immutable transcription output.
-   - `Revision` is optional and user-authored.
-   - Revisions are derived from the original `Job.text`.
-2. Verify relationship cardinality assumptions:
-   - `Document` -> many `Source`
-   - `Document` -> many `Job`
-   - `Source` -> one `Job`
-   - `Source` -> one `Revision`
-3. Ensure field naming consistency (`date_created`, `date_updated`, `date_uploaded`) across code and docs.
-4. Freeze V1 status lifecycle to current implementation (`queued`, `processing`, `transcribed`, `failed`).
-
-### Deliverables
-- Updated `schema_v1.md` and `requirements.md` traceability alignment.
-- Explicit V1 data invariants section in architecture docs.
-
-### Exit Criteria
-- No conflicting definitions of ownership/cardinality/status remain in docs.
-
----
-
-## Phase 2 — Service Layer Refactor To New Model
-
-**Goal:** Remove all obsolete `Transcript` assumptions from service/workflow code.
-
-### Tasks
-1. Refactor `services/transcription.py`:
-   - Replace transcript CRUD assumptions with job-output + revision operations.
-2. Refactor `services/jobs.py`:
-   - Replace old timestamp/relationship accessors with current model fields.
-3. Refactor `services/documents.py` and `services/store.py`:
-   - Ensure upload creates and links `Document`, `Source`, and `Job` correctly.
-4. Refactor `services/workflows.py`:
-   - Persist original provider output to `Job`.
-   - Persist failure detail to `Job.error_detail`.
-   - Use `Revision` only for user-authored edits.
-
-### Deliverables
-- Service layer fully aligned with new schema.
-
-### Exit Criteria
-- No service module imports or persists `Transcript` model artifacts.
-
----
-
-## Phase 3 — UI Contract Alignment
-
-**Goal:** Align pages/components to source/job/revision semantics.
-
-### Tasks
-1. Update job detail and related UI components:
-   - Display original immutable transcription from `Job.text`.
-   - Display optional revision sourced from `Source.revision` (0 or 1).
-2. Align date fields with new schema naming.
-3. Preserve clear user messaging when no revisions exist.
-
-### Deliverables
-- Updated jobs page and detail components.
-
-### Exit Criteria
-- UI behavior and labels match documentation and domain model.
-
----
-
-## Phase 4 — Database Bootstrap, Migration, and Safety
-
-**Goal:** Make schema transition safe in dev/test and repeatable for deployment.
-
-### Tasks
-1. Update bootstrap compatibility logic in `db/operations.py`:
-   - Remove obsolete transcript-table assumptions.
-   - Add forward-compatible patches for current tables only.
-2. Define migration/backfill approach for existing local data.
-3. Document rollback and recovery steps.
-4. Rehearse migration path against representative data.
-
-### Deliverables
-- Migration/upgrade runbook.
-- Validated bootstrap behavior for dev/test.
-
-### Exit Criteria
-- Migration path is documented and tested with no unresolved data-loss risk.
-
----
-
-## Phase 5 — Test Suite Realignment
-
-**Goal:** Restore full confidence after the schema redesign.
-
-### Tasks
-1. Rewrite model tests for:
-   - `Document`, `Source`, `Job`, `Revision` relationships and invariants.
-2. Rewrite service/integration tests:
-   - Worker success/failure paths using `Job.text` / `Job.error_detail`.
-   - Optional single-revision creation/update behavior.
-3. Update UI tests for new job-detail/revision rendering behavior.
-4. Re-enable strict CI quality gates (lint, type, tests).
-
-### Deliverables
-- Updated test matrix and passing CI.
-
-### Exit Criteria
-- Critical user flows and failure paths are covered and green.
-
----
-
-## Phase 6 — Reliability, Operations, and Release Readiness
-
-**Goal:** Ensure V1 is operable and launch-safe.
-
-### Tasks
-1. Verify error taxonomy behavior across UI/API/service/worker.
-2. Confirm structured logging includes relevant identifiers (`job_id`, `document_id`, `source_id` when applicable).
-3. Validate retry behavior and terminal failure handling.
-4. Finalize release checklist, deployment steps, and rollback procedure.
-5. Execute final acceptance run against requirements traceability.
-
-### Deliverables
-- V1 release checklist and acceptance evidence.
-- `runbook_v1.md` for incident response and operator workflows.
-- `release_checklist_v1.md` for release sign-off.
-
-### Exit Criteria
-- Stakeholder sign-off and launch readiness achieved.
-
----
-
-## Requirement Traceability Focus
-
-The plan must keep clear evidence against these requirement groups:
-
-- **Core flow:** REQ-0 to REQ-6
-- **Runtime and operations constraints:** REQ-7 to REQ-12
-- **Revision workflow:** REQ-13
-
-A lightweight traceability table should be maintained with:
-
-- requirement ID
-- implementation status (`not started` / `in progress` / `done`)
-- validation evidence (test name, screenshot, or runbook step)
-
----
-
-## Suggested Execution Rhythm
-
-- **Weekly:** requirement status and risk review
-- **Per PR:** contract checks (model names, field names, lifecycle values)
-- **Milestone checks:** end of Phases 2, 4, and 6
-
----
-
-## Scope Discipline Rule (V1 Focus)
-
-- Only work required to satisfy V1 requirements enters this plan.
-- Nice-to-have enhancements are captured in a separate backlog document.
-- Schema or contract changes after Phase 1 require explicit approval and traceability impact review.
-
----
-
-## Related Local References
-
-- [System Overview](index_v1.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v1.md)
-- [System Requirements](requirements_v1.md)
-- [Data model](schema_v1.md)
-- [Error Handling Policy](error_handling_v1.md)
-- Implementation Plan (this document)
-

docs/ver1/index_v1.md

@@ -1,58 +0,0 @@
-## Document Transcription System Overview
-
-This project is a production application for transcribing and preserving historical family documents. It is intentionally designed for personal-scale use, with a simplicity-first architecture that is easy to operate and easy to extend.
-
-## Start Here
-
-Read [architecture_v1.md](architecture_v1.md) first.
-
-The architecture page is the primary technical reference and defines:
-
-- deployed topology and infrastructure limits
-- module boundaries and dependency flow
-- processing life cycle and data ownership
-- test strategy, risk controls, and extension path
-
-## What The Application Does
-
-At a high level, users upload images or PDFs as content sources for handwritten, typed, or typeset documents, run asynchronous transcription jobs, review optional revisions, and search across accepted text.
-
-### Core capabilities:
-
-- document grouping with one or more content sources and metadata capture
-- asynchronous transcription with visible job status
-- immutable original transcription persisted with each job (plus provider/model/prompt metadata)
-- transcription prompt management with one Markdown file per prompt for human refinement over time
-- optional revisions for user-authored edits of original immutable transcription text
-- full-text search over accepted transcripts
-- export of transcript data
-
-## Production Operating Model
-
-The system runs with minimal operational overhead:
-
-- PostgreSQL in a dedicated Docker container is considered extremely lightweight and simple for this system
-- MongoDB in a dedicated Docker container is also considered extremely lightweight and simple for document-centric persistence
-- a three-container deployment (app, PostgreSQL, MongoDB) is a simple and acceptable baseline
-- no required queue or search-engine containers in the baseline setup
-
-This operating model keeps deployment and maintenance simple while preserving clean boundaries for future scale.
-
----
-
-## Documentation Map
-
-- System Overview (this document)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v1.md)
-- [System Requirements](requirements_v1.md)
-- [Data model](schema_v1.md)
-- [Error Handling Policy](error_handling_v1.md)
-- [Implementation Plan](implementation_plan_v1.md)
-
-## Glossary
-
-- Document-oriented persistence: Storing data as flexible records instead of fixed relational rows.
-- Prompt artifact: A single Markdown file that defines one transcription prompt and is edited independently.
-- System of record: The authoritative persistent store for canonical data.

docs/ver1/release_checklist_v1.md

@@ -1,45 +0,0 @@
-# V1 Release Readiness Checklist
-
-Use this checklist before declaring V1 operationally complete.
-
-## A) Functional Readiness
-
-- [ ] Upload flow works for supported file types.
-- [ ] Worker transitions jobs through `queued -> processing -> transcribed|failed`.
-- [ ] Job detail displays immutable original transcription from `Job.text`.
-- [ ] Revision workflow supports create/update/view/delete for optional single revision.
-
-## B) Reliability and Error Handling
-
-- [ ] Error categories surface with actionable messages in UI/API pathways.
-- [ ] Failed jobs persist `error_detail` and terminal state.
-- [ ] Stale processing recovery verified on restart.
-- [ ] Retry/timeout behavior validated against configured limits.
-
-## C) Operational Readiness
-
-- [ ] `runbook_v1.md` reviewed and current.
-- [ ] `migration_v1.md` reviewed and current.
-- [ ] Backup and rollback procedures tested at least once.
-- [ ] Incident escalation packet template is known to operators.
-
-## D) Quality Gates
-
-- [ ] Lint/type checks pass.
-- [ ] `pytest -m "not external" -q` passes.
-- [ ] Targeted external/provider checks executed (if credentials available).
-- [ ] Release evidence recorded in `release_evidence_v1.md`.
-
-## E) Traceability and Documentation
-
-- [ ] `requirements_v1.md` aligns with implemented V1 behavior.
-- [ ] `architecture_v1.md`, `schema_v1.md`, and `error_handling_v1.md` are consistent.
-- [ ] `traceability_v1.md` is updated with current implementation and test evidence.
-- [ ] `implementation_plan_v1.md` phase status updated with evidence references.
-- [ ] REQ traceability evidence links recorded (tests/runbook/checks).
-
-## Release Sign-Off
-
-- [ ] Technical sign-off complete.
-- [ ] Operational sign-off complete.
-- [ ] V1 completion date recorded.

docs/ver1/release_evidence_v1.md

@@ -1,39 +0,0 @@
-# V1 Release Evidence Log
-
-## Step 5 Quality Gates (2026-07-29)
-
-### Lint
-
-- Command: `python -m ruff check .`
-- Result: ✅ pass
-- Notes: initial findings were auto-fixed (`ruff --fix`) plus small manual line-wrap/annotation adjustments.
-
-### Tests (primary gate)
-
-- Command: `python -m pytest -m "not external" -q`
-- Result: ✅ pass (`[100%]`)
-
-### Tests (external smoke)
-
-- Command: `python -m pytest -m external -q`
-- Result: ✅ pass (`[100%]`)
-
-### Type Check
-
-- Command: `python -m ty check src tests`
-- Result: ⚠️ not passing
-- Summary: existing SQLModel/SQLAlchemy typing incompatibilities and test double typing mismatches remain.
-
-Key current blocker families:
-
-1. SQLModel relationship/query attribute typing (`selectinload`, `order_by`, `.any()`)
-2. SQLAlchemy join clause typing in `services/transcription.py`
-3. Test fake client type mismatch for `OpenRouterTranscriptionProvider(client=...)`
-4. `Settings(**defaults)` typed-dict strictness in `tests/test_config.py`
-
-## Current Gate Status
-
-- Lint: pass
-- Non-external tests: pass
-- External smoke tests: pass
-- Type check: **blocked** (requires dedicated typing cleanup pass)

docs/ver1/requirements_v1.md

@@ -1,98 +0,0 @@
-## Document Transcription System Requirements
-
-This page captures a SysML v1.6-style requirements baseline for the production system described in [index_v1.md](index_v1.md). The model is represented as concise tables and traceability lists that preserve SysML-style IDs and relationship semantics.
-
-## Scope
-
-- System of interest: the single Python application service (NiceGUI + FastAPI) with PostgreSQL as the relational system of record and optional MongoDB for document-oriented persistence.
-- Operational context: local-first execution with Docker Compose and an intentionally lightweight production trajectory.
-- Primary concern: end-to-end transcription job lifecycle from upload through completion or failure.
-
-## Requirements Model (Concise Text Form)
-
-### Requirements
-
-| ID | Category | Requirement | Risk | Verify Method |
-| --- | --- | --- | --- | --- |
-| REQ-0 | System | Provide end-to-end document transcription with persistent, inspectable lifecycle state. | medium | demonstration |
-| REQ-1 | Functional | Allow users to upload one or more images or PDFs as sources from the web UI. | low | test |
-| REQ-2 | Functional | Run each upload through asynchronous processing that returns an original transcription or explicit failure. | high | test |
-| REQ-3 | Functional | Persist and expose job states: queued, processing, transcribed, failed. | high | inspection |
-| REQ-4 | Functional | Persist transcription output, processing history, and failure details. | medium | test |
-| REQ-5 | Interface | Expose API and UI views for status inspection and completed transcription reading. | medium | demonstration |
-| REQ-6 | Performance | Trigger background processing on upload to preserve UI responsiveness. | medium | analysis |
-| REQ-7 | Design Constraint | Keep lifespan-owned runtime resources: SQLAlchemy engine, async session factory, worker resources, provider clients. | medium | inspection |
-| REQ-8 | Design Constraint | Initialize configuration and logging once at startup through centralized mechanisms. | low | inspection |
-| REQ-9 | Design Constraint | Use Docker Compose baseline of app plus PostgreSQL; allow optional MongoDB container when enabled. | medium | demonstration |
-| REQ-10 | Design Constraint | Keep schema bootstrap explicit and opt-in; normal startup does not mutate production schema. | high | inspection |
-| REQ-11 | Design Constraint | Use service-backed persistence for core document and job data. | medium | inspection |
-| REQ-12 | Design Constraint | Store transcription prompts as individual Markdown artifacts for iterative refinement. | medium | inspection |
-| REQ-13 | Functional | Allow users to create one optional revision of transcription text derived from the original job transcription. | low | test |
-
-### Requirement Relationships
-
-- Contains: REQ-0 contains REQ-1 through REQ-13.
-- Derives: REQ-2 -> REQ-3, REQ-3 -> REQ-4.
-- Traces: REQ-5 -> REQ-3.
-- Refines: REQ-6 -> REQ-2.
-
-### Architecture Elements
-
-| Element | Type | Doc Reference |
-| --- | --- | --- |
-| UI | NiceGUI pages | src/transcription/ui/pages |
-| API | FastAPI routes | src/transcription/api/routes.py |
-| GRAPH | Async processing workflow | src/transcription/services, src/transcription/ai |
-| DBREL | PostgreSQL + SQLModel relational persistence | src/transcription/db |
-| DBDOC | MongoDB document persistence | src/transcription/db, src/transcription/services |
-| OPS | Docker Compose runtime | docker-compose.yml |
-| PROMPTS | Transcription prompt artifact library (Markdown files) | .github/prompts, docs |
-| TESTS | Pytest verification suite | tests |
-
-### Satisfaction Mapping
-
-- UI satisfies REQ-1, REQ-5, REQ-13.
-- API satisfies REQ-5.
-- GRAPH satisfies REQ-2, REQ-6.
-- DBREL satisfies REQ-3, REQ-10, REQ-13.
-- DBDOC satisfies REQ-4, REQ-11.
-- OPS satisfies REQ-9.
-- PROMPTS satisfies REQ-12.
-
-### Verification Mapping
-
-- TESTS verifies REQ-1, REQ-2, REQ-3, REQ-4, REQ-5, REQ-10, REQ-11, REQ-12, REQ-13.
-
-## Requirement Notes
-
-- Requirement IDs (`REQ-*`) are stable references for planning, implementation, and test traceability.
-- The model uses compact tables and traceability lists for renderer compatibility while preserving SysML-style requirement IDs and relationship semantics.
-- Requirement categories (functional, interface, performance, and design constraints) are preserved as explicit REQ entries and relationship labels to keep change impact visible.
-- PostgreSQL containerization and optional MongoDB containerization are both treated as extremely lightweight and simple operational choices in this architecture.
-
-## Verification Intent
-
-- Demonstration: validate end-to-end behavior via running system flows and operator-visible outcomes.
-- Inspection: verify architecture and startup/runtime policies in code and configuration.
-- Analysis: evaluate asynchronous execution behavior and design sufficiency.
-- Test: automate behavioral checks through pytest suites and service-level tests.
-
----
-
-## Related Local References
-
-- [System Overview](index_v1.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v1.md)
-- System Requirements (this document)
-- [Data model](schema_v1.md)
-- [Error Handling Policy](error_handling_v1.md)
-- [Implementation Plan](implementation_plan_v1.md)
-
-## Glossary
-
-- Document-oriented persistence: A storage approach that uses flexible document structures for variable data shapes.
-- Prompt artifact: A single Markdown file that defines one transcription prompt and is revised independently.
-- SysML: Systems Modeling Language used to express structured requirements and traceability.
-- System of record: The authoritative persistent store for canonical business data.

docs/ver1/runbook_v1.md

@@ -1,129 +0,0 @@
-# V1 Operations Runbook
-
-This runbook provides day-2 operational procedures for the V1 baseline.
-
-## Scope
-
-Applies to:
-
-- local/hosted V1 runtime
-- SQLite-backed persistence
-- in-process worker lifecycle
-- OpenRouter provider integration
-
-## Preconditions
-
-- `.env` contains `OPENROUTER_API_KEY`
-- app starts successfully
-- `uploads/` and `prompts/` are writable
-- health endpoint responds at `/healthz`
-
-## Standard Startup Procedure
-
-1. Start the app using the project-standard command.
-2. Open `/healthz` and verify `{"status":"ok"}`.
-3. Open `/ui/upload` and submit a small valid file.
-4. Confirm job transitions from `queued` -> `processing` -> `transcribed` (or `failed` with detail).
-
-## Standard Shutdown Procedure
-
-1. Stop the application process.
-2. Ensure no active process still holds the SQLite file.
-3. If maintenance is planned, copy the DB file before edits:
-   - `transcription.db` (or configured `DATABASE_URL` file path)
-
-## Incident: Jobs Stuck In `processing`
-
-### Symptoms
-
-- Jobs remain `processing` for longer than provider timeout
-- New uploads queue but do not complete
-- provider usage increases but no terminal job state is visible
-
-### Checks
-
-1. Confirm app process is still running.
-2. Confirm worker loop is active (startup logs include worker lifespan start).
-3. Inspect recent app logs for:
-   - `worker.process_job`
-   - `error_id`
-   - `category`
-   - `job_id` / `document_id` / `source_id`
-4. Verify provider credentials and provider status.
-
-### Recovery
-
-1. Restart the app to trigger stale-processing recovery.
-2. On startup, app re-queues stale processing jobs based on timeout policy.
-3. Re-check jobs page and confirm terminal state progression.
-4. If persistent, capture logs + error IDs and move to deep investigation.
-
-## Incident: Provider Authentication Failures
-
-### Symptoms
-
-- failures categorized as provider/auth
-- jobs fail quickly with authentication guidance
-
-### Recovery
-
-1. Validate `OPENROUTER_API_KEY` value.
-2. Restart app after updating env.
-3. Re-run a small transcription to confirm recovery.
-
-## Incident: Upload Failures
-
-### Symptoms
-
-- UI reports upload errors
-- unsupported extension or empty payload
-
-### Recovery
-
-1. Validate file extension (`.jpg`, `.jpeg`, `.png`, `.tif`, `.tiff`, `.pdf`).
-2. Validate file is not empty.
-3. Validate upload directory permissions.
-4. Retry upload.
-
-## Incident: Database File/Permission Issues
-
-### Symptoms
-
-- persistence errors during upload/job update
-- startup failures around schema/runtime
-
-### Recovery
-
-1. Confirm the configured DB file path exists and is writable.
-2. Confirm parent directory permissions.
-3. Restore from last known backup copy if corruption is suspected.
-4. Restart app and run smoke test.
-
-## Logging Requirements (Operational)
-
-Operational triage should always capture:
-
-- `error_id`
-- category
-- operation name
-- `job_id`, `document_id`, `source_id` when applicable
-- UTC timestamp
-
-## Escalation Packet (When opening an issue)
-
-Include:
-
-- exact timestamp window
-- one failing `job_id`
-- relevant `error_id` values
-- latest 100 lines of app logs
-- environment summary (`DATABASE_URL` type, app version/commit)
-
-## Post-Incident Validation
-
-After mitigation, verify:
-
-1. Upload works.
-2. One job reaches `transcribed`.
-3. One induced failure reaches `failed` with error detail.
-4. Jobs page and detail page render correctly.

docs/ver1/schema_v1.md

@@ -1,98 +0,0 @@
-## Database Schema (V1 Baseline)
-
-This document describes the current relational schema for the transcription system.
-
-All primary and foreign keys in the domain models are UUID-based in V1.
-
----
-
-## Schema Diagram
-
-```mermaid
-erDiagram
-    DOCUMENT {
-        UUID id PK
-        TEXT name
-    }
-
-    JOB {
-        UUID id PK
-        UUID document_id FK
-        TEXT status
-        INTEGER retry_count
-        DATETIME date_created
-        DATETIME date_updated
-        TEXT provider
-        TEXT model
-        TEXT prompt_name
-        TEXT text
-        TEXT error_detail
-    }
-
-    SOURCE {
-        UUID id PK
-        UUID document_id FK
-        UUID job_id FK
-        TEXT upload_name
-        TEXT filename
-        TEXT file_path
-        DATETIME date_uploaded
-    }
-
-    REVISION {
-        UUID id PK
-        UUID source_id "FK, UK"
-        INTEGER revision
-        TEXT text
-        DATETIME date_created
-    }
-
-    DOCUMENT ||--o{ SOURCE : has_many
-    DOCUMENT ||--o{ JOB : has_many
-    JOB ||--o{ SOURCE : referenced_by
-    SOURCE ||--o| REVISION : has_optional_one
-```
-
----
-
-## Table Relationships and Constraints
-
-- A `Document` can have zero or more `Source` records.
-- A `Document` can have zero or more `Job` records.
-- A `Source` belongs to exactly one `Document` and one `Job`.
-- A `Source` may have one optional `Revision`.
-- Optional `0..1` revision cardinality is enforced by uniqueness on `revision.source_id`.
-
-### Invariants
-
-- `Job.text` stores immutable original provider transcription output.
-- `Revision` rows are optional user-authored edits derived from original transcription.
-- Revisions do not overwrite original `Job.text`.
-- Job status lifecycle values are: `queued`, `processing`, `transcribed`, `failed`.
-
-### Timestamp Fields
-
-- `Job.date_created`
-- `Job.date_updated`
-- `Source.date_uploaded`
-- `Revision.date_created`
-
----
-
-## Related Local References
-
-- [System Overview](index_v1.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v1.md)
-- [System Requirements](requirements_v1.md)
-- Data model (this document)
-- [Error Handling Policy](error_handling_v1.md)
-- [Implementation Plan](implementation_plan_v1.md)
-
-## Glossary
-
-- **Document**: logical grouping for one or more transcribed sources.
-- **Source**: uploaded file content (image/PDF) linked to a job.
-- **Job**: processing record that stores lifecycle status and original output.
-- **Revision**: optional single user-authored edited text linked to a source.

docs/ver1/traceability_v1.md

@@ -1,40 +0,0 @@
-# V1 Traceability Matrix
-
-This matrix provides implementation and validation evidence for V1 requirements (`REQ-0` through `REQ-13`).
-
-Status values:
-
-- `done`: implemented and evidence recorded
-- `in progress`: partially implemented or evidence incomplete
-- `not started`: no implementation/evidence yet
-
-## Requirement Evidence Table
-
-| Requirement | Status | Implementation Evidence | Validation Evidence |
-| --- | --- | --- | --- |
-| REQ-0 | done | End-to-end upload + worker pipeline in `src/transcription/services/store.py`, `src/transcription/worker.py`, `src/transcription/services/workflows.py` | `tests/integration/test_pipeline_flow.py` |
-| REQ-1 | done | Upload UI/page flow in `src/transcription/ui/pages/upload_page.py`, `src/transcription/ui/components/upload.py` | `tests/ui/test_upload_page.py`, `tests/integration/test_pipeline_flow.py` |
-| REQ-2 | done | Async worker execution and provider call orchestration in `src/transcription/worker.py`, `src/transcription/services/workflows.py` | `tests/integration/test_pipeline_flow.py`, `tests/services/test_workflows_reliability.py` |
-| REQ-3 | done | Job lifecycle state model + transitions in `src/transcription/models.py`, `src/transcription/services/jobs.py`, `src/transcription/services/workflows.py` | `tests/services/test_job_service.py`, `tests/ui/test_jobs_page.py` |
-| REQ-4 | done | Persistence of original output and failure detail in `src/transcription/services/transcription.py`, `src/transcription/services/workflows.py` | `tests/integration/test_pipeline_flow.py`, `tests/services/test_workflows_reliability.py` |
-| REQ-5 | done | Status/result inspection via UI pages and API health route in `src/transcription/ui/pages/jobs_page.py`, `src/transcription/api/health.py` | `tests/ui/test_jobs_page.py`, `tests/ui/test_pages_registration.py`, `tests/api/test_health.py` |
-| REQ-6 | done | Background processing trigger/worker notifier and non-blocking workflow in `src/transcription/ui/components/upload.py`, `src/transcription/worker.py` | `tests/test_app.py`, `tests/services/test_workflows_reliability.py` |
-| REQ-7 | done | Lifespan-owned runtime resources in `src/transcription/app.py`, `src/transcription/db/runtime.py` | `tests/test_app.py`, `tests/test_db.py` |
-| REQ-8 | done | Centralized settings/logging initialization in `src/transcription/config.py`, `src/transcription/app.py` | `tests/test_config.py`, `tests/test_app.py` |
-| REQ-9 | done | Containerized runtime baseline in `docker-compose.yml`, `Dockerfile` | `release_checklist_v1.md` (Ops checklist), manual demonstration step |
-| REQ-10 | done | Explicit schema bootstrap policy + runtime controls in `src/transcription/config.py`, `src/transcription/app.py`, `src/transcription/db/operations.py` | `tests/test_db.py`, `tests/test_config.py` |
-| REQ-11 | done | Service/workflow persistence boundaries in `src/transcription/services/*.py`, `src/transcription/services/workflows.py` | `tests/services/test_job_service.py`, `tests/services/test_transcription_service.py` |
-| REQ-12 | done | Prompt artifacts in `prompts/` and loading/validation in `src/transcription/services/transcription.py` | `tests/test_prompts.py` |
-| REQ-13 | done | Optional single revision create/update/view/delete in `src/transcription/services/transcription.py`, `src/transcription/ui/pages/jobs_page.py` | `tests/services/test_transcription_service.py`, `tests/ui/test_jobs_page.py` |
-
-## Operational Evidence (Step 3 Artifacts)
-
-- Runbook: `runbook_v1.md`
-- Migration/backfill/rollback guidance: `migration_v1.md`
-- Release readiness checklist: `release_checklist_v1.md`
-
-## Verification Cadence
-
-- Per change: maintain `tests/test_traceability.py` mappings for touched requirements.
-- Per milestone: update this table status and evidence links.
-- Pre-release: confirm all rows are `done` and non-external suite is green.

docs/ver4.8/feature_backlog_v4_8.md

@@ -0,0 +1,84 @@
+# V4.8 Feature Backlog
+
+**Status: not scoped.** This is a parking document, not a frozen boundary. It records feature work deferred out of V4.6 and V4.7 together with the evidence gathered so far, so that scoping V4.8 does not start from a blank page.
+
+V4.8 is the first release since V4.5 to add **new user-facing behavior**. V4.6 was pure remediation and V4.7 is architectural cleanup; both were held to "no new features." That constraint ends here, which means V4.8 needs a different verification gate: V4.6 and V4.7 could be validated by "the suite still passes unchanged," and V4.8 cannot.
+
+## Dependency on V4.7
+
+**The model-performance rollup below must not begin until V4.7 Phase 4 lands.** `duration_ms` currently measures provider call *plus* image normalization, artifact persistence, and a DB commit, while the timeout governs only the provider call. A rollup built on it would chart preprocessing time mixed with provider latency and look authoritative while quietly misleading. V4.7 Phase 1 removes normalization and artifact persistence from that window, but the commit remains inside it until Phase 4.
+
+## Candidate Features
+
+### 1. Pan and Zoom on Source Detail
+
+**Practicality: high. Effort: S.**
+
+`ui/components/document_panzoom.py` existed and was **deleted in V4.6 Phase 5** (`6a3ee26`) because it was exported but wired to no page. It is 136 lines and recoverable:
+
+```
+git show 6a3ee26^:src/transcription/ui/components/document_panzoom.py
+```
+
+It already handled both images and PDFs (the latter via an iframe).
+
+Two things must change on reintroduction - this is not a straight revert:
+
+- It loaded Panzoom from the **unpkg CDN**. For an archival application the library should be vendored locally, otherwise the viewer breaks offline and depends on a third party staying available.
+- It carried its own `_document_url()` helper. V4.6 Phase 5 extracted exactly that logic into `ui/components/media_urls.py` as `resolve_media_url`. Reintroducing the old helper would recreate the duplication Phase 5 removed.
+
+Scope note: apply it to **Source Detail only**. `dark_room_viewer` (`ui/components/viewers.py`) is shared by four pages - `sources_page.py:268`, `home_page.py:25` and `:88`, `people_page.py:453`, `documents_page.py:524` - so a flag on it would leak pan-zoom into the homepage and document detail, which is not wanted. Add a separate component and use it only at `sources_page.py:268`.
+
+Numbering note: the Phase 5 commit message states pan-zoom would return "in V4.7 alongside the other photo/image work." Moving it to V4.8 preserves that **intent** - it stays grouped with the photo work - and changes only the release number.
+
+### 2. Homepage Image Gallery
+
+**Practicality: high. Effort: S. Recommended first feature.**
+
+The storage layer is already built:
+
+- `ui/homepage_store.py:82` `list_homepage_images()` already returns **every** stored image, sorted by modification time.
+- `store_homepage_image()` already accumulates files rather than overwriting.
+- Today the UI calls only `latest_homepage_image()` and displays one image. `list_homepage_images()` is currently exercised **only by tests**.
+
+So multi-image upload is effectively done; what is missing is presentation. NiceGUI 3.13.0 provides `ui.carousel` for left/right navigation and `ui.timer` for rotation.
+
+Sub-items:
+
+- Multi-image display with left/right navigation - small, mostly wiring.
+- Optional slideshow rotating every ~10 minutes.
+
+**Performance caveat:** `list_homepage_images()` performs a directory scan with a `stat()` per file on every call, and `home_page.py` already performs blocking I/O in the page handler (V4.6 review log [25], which was deliberately left alone). A rotating timer that re-enumerates on every tick would repeat that scan indefinitely. Enumerate once at page load and cache the list.
+
+### 3. Multiple Person Portraits
+
+**Practicality: medium. Effort: M/L. Defer behind item 2.**
+
+`Person.portrait_path` is a **single string column**. Supporting multiple portraits requires a new table, a data migration, and upload UI - a materially larger job than item 2, and a different one.
+
+### 4. Image Descriptions
+
+**Practicality: medium, conditional. Effort: M.**
+
+Homepage images are **filesystem-only with no metadata store**, so a caption has nowhere to live today. This needs either a sidecar JSON file or a real table.
+
+This is cheap **only if** item 3 is being done at the same time, since both need the same metadata layer. Designing that layer twice would be wasteful; design it once or not at all.
+
+## Suggested Grouping
+
+If V4.8 is scoped as one release, the natural split is:
+
+**Track A - image experience:** items 1 and 2. Both are small, both are self-contained UI work, and item 2's storage layer already exists. This is the highest value for the least risk.
+
+**Track B - metadata layer:** items 3 and 4 together, since they share a table. Only worth starting if both are wanted.
+
+**Track C - telemetry:** item 5, gated on V4.7 Phase 4.
+
+Item 6 is not recommended.
+
+## Open Questions for Scoping
+
+- Should Track B happen at all, or is one portrait per person sufficient?
+- Should the slideshow interval be configurable, or fixed?
+- Should vendored Panzoom be committed to the repository, or fetched at build time?
+

prompts/README.md

@@ -5,9 +5,11 @@ This directory stores transcription prompts as individual Markdown artifacts.
 ## Conventions
 - Keep one prompt per file.
 - Use stable, descriptive snake_case file names.
+- Store prompt files directly in this directory; nested paths are rejected.
 - Prefer incremental edits to a single prompt per change for clean history.
 - Keep prompts human-readable and policy-focused.
 - Do not store secrets in prompt files.
+- Runtime jobs snapshot prompt text, SHA-256 provenance, and sampling configuration.
 
 ## Current Prompt
 - `transcribe_document.md`: baseline verbatim transcription policy for historical documents.

prompts/system.txt

@@ -1,10 +0,0 @@
-You are an assistant that may call tools.
-
-Tool safety rules:
-1) Tool arguments MUST be strict JSON matching the schema exactly.
-2) Never place disallowed, sensitive, explicit, or policy-violating text directly into tool arguments.
-3) If user content may be unsafe, first produce a brief neutral summary and pass only that summary.
-4) Prefer IDs, enums, booleans, and short fields over raw free-form text.
-5) Keep all string arguments <= 300 chars unless schema says otherwise.
-6) If you cannot safely provide valid tool args, do not call the tool; respond with "NO_TOOL_CALL" and explain briefly.
-7) Never include markdown/code fences in tool arguments.

prompts/transcribe_document.md

@@ -6,9 +6,15 @@ Do not summarize. Do not paraphrase. Do not modernize style.
 
 ## Output Contract
 - Return only the transcription text.
+- Begin with exactly one applicable body marker:
+  - `[document body handwritten]`
+  - `[document body typewritten]`
+  - `[document body typeset]`
+  - `[document body mixed]`
 - Preserve original wording, punctuation, and meaningful structure.
 - Keep line/section flow readable while preserving intent and document organization.
 - Never invent missing content.
+- Use ordinary plain-text characters rather than HTML entities.
 
 ## Rules for Ambiguous or Damaged Text
 
@@ -46,6 +52,31 @@ Do not summarize. Do not paraphrase. Do not modernize style.
 - Signal location before the note text.
 - Example form: `[written in left margin: ...]`
 
+### Document body medium
+- Use `[document body handwritten]` when the main body is written by hand.
+- Use `[document body typewritten]` for mechanically typewritten pages. Uneven impressions,
+  monospaced characters, worn type, and other typewriter defects are not handwriting.
+- Use `[document body typeset]` for printed pages composed with movable type or comparable
+  typesetting.
+- Use `[document body mixed]` when substantial body content uses more than one medium, such
+  as a completed printed form.
+- Preserve printed and handwritten text together in their original reading context.
+- On mixed documents, leave printed labels and instructions unmarked and wrap only actual
+  handwritten entries in `[handwritten: ...]`.
+- Mark handwritten signatures as `[handwritten signature: ...]`.
+- If the main body is entirely handwritten, use its one body marker rather than wrapping
+  each line in `[handwritten: ...]`.
+- Mark later notes or uncertain additions as `[handwritten annotation: ...]`.
+- When authorship is unclear, use `[handwritten annotation, author uncertain: ...]`.
+- Do not infer authorship, writing date, or whether different handwriting belongs to different people unless explicitly evident.
+
+### Structured layouts
+- Preserve tables of contents as associated title, dotted-leader, and page-reference rows.
+- Preserve tables and forms in reading order while keeping labels associated with their values.
+- Preserve columns in their evident reading order; do not interleave unrelated rows.
+- Preserve captions with the visual element they describe.
+- Preserve marginalia with its location marker and page numbers in their evident position.
+
 ### Line-break hyphenation
 - Rejoin words split across line breaks when they are clearly one word.
 - Remove only line-break hyphens used for wrapping.
@@ -70,3 +101,5 @@ Before finalizing, ensure:
 2. Uncertain/illegible areas are explicitly marked.
 3. Crossed-out and inserted text are preserved with required tags.
 4. Structure/ordering is preserved as faithfully as possible.
+5. Exactly one document-body marker appears, and localized handwriting markers are used only where applicable.
+6. Tables, forms, columns, captions, marginalia, dotted leaders, and page references retain their associations.

pyproject.toml

@@ -17,6 +17,7 @@ dependencies = [
     "fastapi>=0.138.0",
     "nicegui==3.13.0",
     "openrouter>=0.7.0",
+    "pillow>=10.0.0",
     "psycopg2-binary>=2.9.12",
     "pydantic>=2.13.4",
     "pydantic-settings>=2.9.1",
@@ -40,6 +41,7 @@ dev = [
 [tool.pytest.ini_options]
 addopts = "--strict-markers -q"
 asyncio_mode = "strict"
+asyncio_default_fixture_loop_scope = "function"
 filterwarnings = [
     "error:coroutine .* was never awaited:RuntimeWarning",
 ]

src/transcription/__main__.py

@@ -0,0 +1,27 @@
+import uvicorn
+from fastapi import FastAPI
+
+from .app import create_app
+from .config import parse_cli_settings
+
+
+def create_cli_app() -> FastAPI:
+    """Create an app from CLI settings for Uvicorn's reload process."""
+    return create_app(settings=parse_cli_settings())
+
+
+def main() -> None:
+    settings = parse_cli_settings()
+    application = "transcription.__main__:create_cli_app" if settings.reload else create_app(settings=settings)
+    uvicorn.run(
+        application,
+        factory=settings.reload,
+        host=settings.host,
+        port=settings.port,
+        log_level=settings.log_level,
+        reload=settings.reload,
+    )
+
+
+if __name__ == "__main__":
+    main()

src/transcription/api/documents_api.py

@@ -0,0 +1,202 @@
+"""API routes for relationship and classification registries."""
+
+from __future__ import annotations
+
+from typing import Annotated
+from uuid import UUID
+
+from fastapi import APIRouter
+from fastapi import Depends
+from fastapi import Request
+from fastapi import Response
+from pydantic import BaseModel
+from pydantic import ConfigDict
+from pydantic import Field
+
+from transcription.db.models import Document
+from transcription.db.models import DocumentPerson
+from transcription.db.models import DocumentType
+from transcription.db.models import PersonRole
+from transcription.services import DocumentService
+from transcription.services import PeopleService
+
+router = APIRouter(prefix="/api", tags=["documents"])
+
+
+class ApiModel(BaseModel):
+    model_config = ConfigDict(extra="forbid", frozen=True, str_strip_whitespace=True)
+
+
+class DocumentTypeRead(ApiModel):
+    id: UUID
+    label: str
+    is_active: bool
+
+
+class PersonRoleRead(ApiModel):
+    id: UUID
+    label: str
+    is_active: bool
+
+
+class DocumentTypeWriteRequest(ApiModel):
+    document_type_id: UUID
+
+
+class DocumentTypeWriteResponse(ApiModel):
+    document_id: UUID
+    document_type_id: UUID
+
+
+class DocumentPersonWriteRequest(ApiModel):
+    person_id: UUID
+    role_id: UUID
+
+
+class DocumentPersonRoleUpdateRequest(ApiModel):
+    role_id: UUID
+
+
+class DocumentPersonRead(ApiModel):
+    id: UUID
+    document_id: UUID
+    person_id: UUID
+    role_id: UUID
+    role_label: str | None = None
+    person_name: str | None = None
+
+
+class DocumentPeopleResponse(ApiModel):
+    document_id: UUID
+    links: list[DocumentPersonRead] = Field(default_factory=list)
+
+
+def _document_type_to_read(item: DocumentType) -> DocumentTypeRead:
+    item_id, label, is_active = _registry_read_values(item)
+    return DocumentTypeRead(id=item_id, label=label, is_active=is_active)
+
+
+def _person_role_to_read(item: PersonRole) -> PersonRoleRead:
+    item_id, label, is_active = _registry_read_values(item)
+    return PersonRoleRead(id=item_id, label=label, is_active=is_active)
+
+
+def _registry_read_values(item: DocumentType | PersonRole) -> tuple[UUID, str, bool]:
+    return item.id, item.label, item.is_active
+
+
+def _document_person_to_read(item: DocumentPerson) -> DocumentPersonRead:
+    person_name = item.person.full_name if item.person is not None else None
+    return DocumentPersonRead(
+        id=item.id,
+        document_id=item.document_id,
+        person_id=item.person_id,
+        role_id=item.role_id,
+        role_label=item.role_ref.label if item.role_ref is not None else None,
+        person_name=person_name,
+    )
+
+
+def _document_to_type_response(item: Document) -> DocumentTypeWriteResponse:
+    if item.document_type_id is None:
+        raise ValueError("Document Type assignment did not persist")
+    return DocumentTypeWriteResponse(
+        document_id=item.id,
+        document_type_id=item.document_type_id,
+    )
+
+
+def get_document_service(request: Request) -> DocumentService:
+    """Resolve the document service from app lifespan state when available."""
+    services = getattr(request.app.state, "services", None)
+    if services is not None:
+        return services.documents
+    return DocumentService()
+
+
+def get_people_service(request: Request) -> PeopleService:
+    """Resolve the People service from app lifespan state when available."""
+    services = getattr(request.app.state, "services", None)
+    if services is not None:
+        return services.people
+    return PeopleService()
+
+
+DocumentServiceDependency = Annotated[DocumentService, Depends(get_document_service)]
+PeopleServiceDependency = Annotated[PeopleService, Depends(get_people_service)]
+
+
+@router.get("/document-types", response_model=list[DocumentTypeRead])
+async def list_document_types(
+    service: DocumentServiceDependency,
+    active_only: bool = True,
+) -> list[DocumentTypeRead]:
+    items = await service.list_document_types(active_only=active_only)
+    return [_document_type_to_read(item) for item in items]
+
+
+@router.get("/person-roles", response_model=list[PersonRoleRead])
+async def list_person_roles(
+    service: PeopleServiceDependency,
+    active_only: bool = True,
+) -> list[PersonRoleRead]:
+    items = await service.list_person_roles(active_only=active_only)
+    return [_person_role_to_read(item) for item in items]
+
+
+@router.put("/documents/{document_id}/type", response_model=DocumentTypeWriteResponse)
+async def set_document_type(
+    document_id: UUID,
+    payload: DocumentTypeWriteRequest,
+    service: DocumentServiceDependency,
+) -> DocumentTypeWriteResponse:
+    document = await service.set_document_type(
+        document_id=document_id,
+        document_type_id=payload.document_type_id,
+    )
+    return _document_to_type_response(document)
+
+
+@router.get("/documents/{document_id}/people", response_model=DocumentPeopleResponse)
+async def list_document_people(
+    document_id: UUID,
+    service: PeopleServiceDependency,
+) -> DocumentPeopleResponse:
+    links = await service.list_document_people(document_id=document_id)
+    return DocumentPeopleResponse(document_id=document_id, links=[_document_person_to_read(item) for item in links])
+
+
+@router.post("/documents/{document_id}/people", response_model=DocumentPersonRead)
+async def add_document_person_link(
+    document_id: UUID,
+    payload: DocumentPersonWriteRequest,
+    service: PeopleServiceDependency,
+) -> DocumentPersonRead:
+    link = await service.add_document_person_link(
+        document_id=document_id,
+        person_id=payload.person_id,
+        role_id=payload.role_id,
+    )
+    return _document_person_to_read(link)
+
+
+@router.patch("/document-people/{document_person_id}", response_model=DocumentPersonRead)
+async def set_document_person_role(
+    document_person_id: UUID,
+    payload: DocumentPersonRoleUpdateRequest,
+    service: PeopleServiceDependency,
+) -> DocumentPersonRead:
+    link = await service.set_document_person_role(
+        document_person_id=document_person_id,
+        role_id=payload.role_id,
+    )
+    return _document_person_to_read(link)
+
+
+@router.delete("/document-people/{document_person_id}", status_code=204)
+async def delete_document_person_link(
+    document_person_id: UUID,
+    service: PeopleServiceDependency,
+) -> Response:
+    await service.remove_document_person_link(document_person_id=document_person_id)
+    return Response(status_code=204)

src/transcription/api/errors.py

@@ -21,7 +21,9 @@ _STATUS_BY_CATEGORY: dict[ErrorCategory, int] = {
     ErrorCategory.NOT_FOUND: 404,
     ErrorCategory.CONFLICT: 409,
     ErrorCategory.EXTERNAL_PROVIDER: 503,
+    ErrorCategory.EXTERNAL_TIMEOUT: 503,
     ErrorCategory.INFRA_TRANSIENT: 503,
+    ErrorCategory.PROCESSING: 500,
     ErrorCategory.INFRA_PERSISTENT: 500,
     ErrorCategory.INTERNAL_UNEXPECTED: 500,
 }

src/transcription/api/health.py

@@ -1,16 +1,44 @@
 """Health endpoint routes."""
 
+from typing import NotRequired
+from typing import TypedDict
+
 from fastapi import APIRouter
+from fastapi import Request
+
+from transcription.worker import resolve_worker_health
 
 router = APIRouter()
 
 
-def healthz() -> dict[str, str]:
-    """Return a simple health status payload."""
-    return {"status": "ok"}
+class WorkerHealthPayload(TypedDict):
+    state: str
+    error_id: NotRequired[str]
+    error_category: NotRequired[str]
+
+
+class HealthPayload(TypedDict):
+    status: str
+    worker: WorkerHealthPayload
+
+
+def healthz(request: Request) -> HealthPayload:
+    """Return health status with worker-liveness signal."""
+    worker = resolve_worker_health(request.app.state)
+    payload: HealthPayload = {
+        "status": "ok",
+        "worker": {
+            "state": worker.state,
+        },
+    }
+    if worker.error_id is not None:
+        payload["worker"]["error_id"] = worker.error_id
+    if worker.error_category is not None:
+        payload["worker"]["error_category"] = worker.error_category
+    return payload
 
 
 @router.get("/healthz")
-def healthz_route() -> dict[str, str]:
+def healthz_route(request: Request) -> HealthPayload:
     """Route wrapper for health status payload."""
-    return healthz()
+    return healthz(request)

src/transcription/api/print_api.py

@@ -0,0 +1,54 @@
+"""Safe media route for Document print previews."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Annotated
+from uuid import UUID
+
+from fastapi import APIRouter
+from fastapi import Depends
+from fastapi import HTTPException
+from fastapi import Request
+from fastapi.responses import FileResponse
+
+from transcription.services.source_media import SOURCE_MIME_TYPES
+from transcription.services.sources import SourceService
+
+router = APIRouter(prefix="/api", tags=["print"])
+
+
+def get_source_service(request: Request) -> SourceService:
+    services = getattr(request.app.state, "services", None)
+    if services is not None:
+        return services.sources
+    return SourceService()
+
+
+SourceServiceDependency = Annotated[SourceService, Depends(get_source_service)]
+
+
+@router.get("/documents/{document_id}/sources/{source_id}/media", response_class=FileResponse)
+async def read_document_source_media(
+    document_id: UUID,
+    source_id: UUID,
+    service: SourceServiceDependency,
+) -> FileResponse:
+    """Serve one validated Source through record identifiers, never a supplied path."""
+    source = await service.read_source(source_id)
+    if source.document_id != document_id:
+        raise HTTPException(status_code=404, detail="Source not found for Document")
+
+    upload_root = service.settings.upload_dir.resolve()
+    path = (upload_root / Path(source.file_path)).resolve()
+    try:
+        path.relative_to(upload_root)
+    except ValueError as exc:
+        raise HTTPException(status_code=404, detail="Source media is outside managed storage") from exc
+    if not path.is_file():
+        raise HTTPException(status_code=404, detail="Source media is unavailable")
+
+    media_type = SOURCE_MIME_TYPES.get(path.suffix.lower())
+    if media_type is None:
+        raise HTTPException(status_code=415, detail="Unsupported Source media type")
+    return FileResponse(path, media_type=media_type)

src/transcription/app.py

@@ -15,14 +15,18 @@ from fastapi.responses import RedirectResponse
 from fastapi.staticfiles import StaticFiles
 
 from .api.errors import register_error_handlers
+from .api.documents_api import router as documents_router
 from .api.health import router as health_router
+from .api.print_api import router as print_router
+from .config import Settings
 from .config import configure_logging
 from .config import get_settings
 from .db import create_all
 from .db import dispose_database_runtime
 from .db import initialize_database_runtime
+from .db import reconcile_canonical_media_paths
+from .db import reconcile_legacy_job_source_columns
 from .services import ServiceBundle
-from .services.jobs import JobService
 from .ui import register_pages
 from .worker import worker_consumer_lifespan
 
@@ -31,24 +35,28 @@ logger = logging.getLogger(__name__)
 
 @asynccontextmanager
 async def _lifespan(app: FastAPI):
-    configure_logging()
-
     settings = getattr(app.state, "settings", None) or get_settings()
+    configure_logging(settings)
     app.state.settings = settings
-    app.state.services = ServiceBundle()
     app.state.runtime = initialize_database_runtime(settings=settings)
+    session_factory = app.state.runtime.session_factory
+    app.state.services = ServiceBundle.from_session_factory(session_factory, settings=settings)
 
     if settings.should_bootstrap_schema:
         await create_all(engine=app.state.runtime.engine)
+    await reconcile_legacy_job_source_columns(engine=app.state.runtime.engine)
+    await reconcile_canonical_media_paths(engine=app.state.runtime.engine)
 
     settings.upload_dir.mkdir(parents=True, exist_ok=True)
     settings.prompt_dir.mkdir(parents=True, exist_ok=True)
+    settings.log_dir.mkdir(parents=True, exist_ok=True)
+    settings.database_backup_dir.mkdir(parents=True, exist_ok=True)
 
     await _recover_stale_processing_jobs(app)
 
     async with AsyncExitStack() as stack:
         stack.push_async_callback(dispose_database_runtime)
-        stop_event, worker_notifier = await stack.enter_async_context(
+        stop_event, worker_notifier, worker_health = await stack.enter_async_context(
             worker_consumer_lifespan(
                 session_factory=app.state.runtime.session_factory,
                 poll_interval_seconds=1.0,
@@ -56,6 +64,7 @@ async def _lifespan(app: FastAPI):
         )
         app.state.worker_stop_event = stop_event
         app.state.worker_notifier = worker_notifier
+        app.state.worker_health = worker_health
         yield
 
 
@@ -67,32 +76,33 @@ async def _recover_stale_processing_jobs(app: FastAPI) -> None:
     """
     settings = app.state.settings
     stale_before = datetime.now(UTC) - timedelta(seconds=settings.worker_provider_timeout_seconds)
-    job_service = JobService(session_factory=app.state.runtime.session_factory)
-    recovered = await job_service.requeue_stale_processing_jobs(stale_before=stale_before)
+    recovered = await app.state.services.jobs.requeue_stale_processing_jobs(stale_before=stale_before)
     if recovered > 0:
         logger.warning("Recovered %s stale processing job(s) at startup", recovered)
 
 
-def create_app() -> FastAPI:
+def create_app(settings: Settings | None = None) -> FastAPI:
     """Create and configure the FastAPI application."""
     app = FastAPI(title="Transcription", lifespan=_lifespan)
-    settings = get_settings()
-    app.state.settings = settings
+    active_settings = settings or get_settings()
+    app.state.settings = active_settings
     app.mount(
         "/uploads",
-        StaticFiles(directory=settings.upload_dir, check_dir=False),
+        StaticFiles(directory=active_settings.upload_dir, check_dir=False),
         name="uploads",
     )
 
     @app.get("/", include_in_schema=False)
     async def root_redirect() -> RedirectResponse:
-        return RedirectResponse(url="/ui", status_code=status.HTTP_307_TEMPORARY_REDIRECT)
+        return RedirectResponse(url="/ui/homepage", status_code=status.HTTP_307_TEMPORARY_REDIRECT)
 
     @app.get("/ui", include_in_schema=False)
     async def ui_redirect() -> RedirectResponse:
-        return RedirectResponse(url="/ui/upload", status_code=status.HTTP_307_TEMPORARY_REDIRECT)
+        return RedirectResponse(url="/ui/homepage", status_code=status.HTTP_307_TEMPORARY_REDIRECT)
 
     register_error_handlers(app)
-    register_pages(app)
     app.include_router(health_router)
+    app.include_router(documents_router)
+    app.include_router(print_router)
+    register_pages(app)
     return app

src/transcription/app_state.py

@@ -1,39 +0,0 @@
-"""Helpers for accessing lifespan-owned application state resources."""
-
-from __future__ import annotations
-
-from fastapi import FastAPI
-from sqlalchemy.ext.asyncio import async_sessionmaker
-from sqlmodel.ext.asyncio.session import AsyncSession
-
-from transcription.db.runtime import DatabaseRuntime
-from transcription.db.runtime import get_session_factory
-from transcription.worker import WorkerNotifier
-from transcription.worker import resolve_worker_notifier
-
-
-def resolve_database_runtime(state: object) -> DatabaseRuntime | None:
-    """Return database runtime from app-like state objects when available."""
-    runtime = getattr(state, "runtime", None)
-    return runtime if isinstance(runtime, DatabaseRuntime) else None
-
-
-def require_database_runtime(state: object) -> DatabaseRuntime:
-    """Return database runtime or raise when app lifespan has not initialized it."""
-    runtime = resolve_database_runtime(state)
-    if runtime is None:
-        raise RuntimeError("Database runtime is not initialized on application state")
-    return runtime
-
-
-def resolve_session_factory(state: object) -> async_sessionmaker[AsyncSession]:
-    """Return DB session factory from state when available, otherwise shared runtime."""
-    runtime = resolve_database_runtime(state)
-    if runtime is not None:
-        return runtime.session_factory
-    return get_session_factory()
-
-
-def get_worker_notifier(app: FastAPI) -> WorkerNotifier:
-    """Return app worker notifier, or a no-op fallback when unavailable."""
-    return resolve_worker_notifier(app.state)

src/transcription/benchmarking.py

@@ -0,0 +1,102 @@
+"""Private-corpus benchmark contracts and deterministic text scoring."""
+
+from __future__ import annotations
+
+from uuid import UUID
+
+from pydantic import BaseModel
+from pydantic import ConfigDict
+from pydantic import Field
+
+
+class BenchmarkModel(BaseModel):
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+
+class BenchmarkItem(BenchmarkModel):
+    """One private benchmark item referenced by archival identity."""
+
+    source_id: UUID
+    source_digest_sha256: str = Field(pattern=r"^[0-9a-f]{64}$")
+    categories: frozenset[str] = Field(min_length=1)
+    reference_transcription: str = Field(min_length=1)
+
+
+class BenchmarkManifest(BenchmarkModel):
+    """Versioned private benchmark definition without copied source media."""
+
+    schema_name: str = "transcription.private-benchmark"
+    schema_version: str = "1"
+    name: str = Field(min_length=1)
+    items: tuple[BenchmarkItem, ...] = Field(min_length=1)
+
+
+class EditorialAssessment(BenchmarkModel):
+    """Manually reviewed errors not represented adequately by CER or WER."""
+
+    omissions: int = Field(default=0, ge=0)
+    inventions: int = Field(default=0, ge=0)
+    silent_normalizations: int = Field(default=0, ge=0)
+    uncertainty_errors: int = Field(default=0, ge=0)
+    layout_errors: int = Field(default=0, ge=0)
+
+
+class BenchmarkScore(BenchmarkModel):
+    """Measured score for one preserved execution attempt."""
+
+    execution_attempt_id: UUID
+    character_error_rate: float = Field(ge=0)
+    word_error_rate: float = Field(ge=0)
+    character_edits: int = Field(ge=0)
+    word_edits: int = Field(ge=0)
+    reference_characters: int = Field(ge=0)
+    reference_words: int = Field(ge=0)
+    assessment: EditorialAssessment
+    latency_ms: int = Field(ge=0)
+    cost_usd: float | None = Field(default=None, ge=0)
+
+
+def score_transcription(
+    *,
+    execution_attempt_id: UUID,
+    reference: str,
+    candidate: str,
+    assessment: EditorialAssessment,
+    latency_ms: int,
+    cost_usd: float | None = None,
+) -> BenchmarkScore:
+    """Score literal text without case-folding or silent normalization."""
+    reference_words = reference.split()
+    candidate_words = candidate.split()
+    character_edits = _levenshtein(list(reference), list(candidate))
+    word_edits = _levenshtein(reference_words, candidate_words)
+    return BenchmarkScore(
+        execution_attempt_id=execution_attempt_id,
+        character_error_rate=character_edits / max(1, len(reference)),
+        word_error_rate=word_edits / max(1, len(reference_words)),
+        character_edits=character_edits,
+        word_edits=word_edits,
+        reference_characters=len(reference),
+        reference_words=len(reference_words),
+        assessment=assessment,
+        latency_ms=latency_ms,
+        cost_usd=cost_usd,
+    )
+
+
+def _levenshtein(reference: list[str], candidate: list[str]) -> int:
+    if len(reference) < len(candidate):
+        reference, candidate = candidate, reference
+    previous = list(range(len(candidate) + 1))
+    for reference_index, reference_value in enumerate(reference, start=1):
+        current = [reference_index]
+        for candidate_index, candidate_value in enumerate(candidate, start=1):
+            current.append(
+                min(
+                    current[-1] + 1,
+                    previous[candidate_index] + 1,
+                    previous[candidate_index - 1] + (reference_value != candidate_value),
+                )
+            )
+        previous = current
+    return previous[-1]

src/transcription/config.py

@@ -5,13 +5,23 @@ once at startup. Provider-specific defaults (model names, base URLs)
 are resolved by the provider adapters, not here.
 """
 
+import copy
 import logging.config
-from contextvars import ContextVar
+from collections.abc import Sequence
 from enum import StrEnum
+from functools import cache
 from pathlib import Path
+from typing import Annotated
+from typing import Any
 from typing import Literal
 
+from pydantic import BaseModel
+from pydantic import ConfigDict
 from pydantic import Field
+from pydantic import SecretStr
+from pydantic import StringConstraints
+from pydantic import field_validator
+from pydantic import model_validator
 from pydantic_settings import BaseSettings
 from pydantic_settings import SettingsConfigDict
 
@@ -22,60 +32,160 @@ class Provider(StrEnum):
     OPENROUTER = "openrouter"
 
 
+NonEmptyStr = Annotated[str, StringConstraints(strip_whitespace=True, min_length=1)]
+PromptFilename = Annotated[str, StringConstraints(strip_whitespace=True, min_length=1, pattern=r"^[^/\\]+$")]
+Probability = Annotated[float, Field(ge=0.0, le=1.0)]
+Temperature = Annotated[float, Field(ge=0.0, le=2.0)]
+DEFAULT_PROVIDER_MODEL = "google/gemini-2.5-flash"
+
+
+class SqliteSettings(BaseModel):
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+    driver: Literal["sqlite"] = "sqlite"
+    path: NonEmptyStr = "./data/transcription.db"
+
+
+class PostgresSettings(BaseModel):
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+    driver: Literal["postgres"] = "postgres"
+    host: NonEmptyStr
+    port: int = Field(default=5432, ge=1, le=65535)
+    database: NonEmptyStr
+    user: NonEmptyStr
+    password: SecretStr
+
+
+DatabaseSettings = Annotated[
+    SqliteSettings | PostgresSettings,
+    Field(discriminator="driver"),
+]
+
+
 class Settings(BaseSettings):
     model_config = SettingsConfigDict(
         env_file=".env",
         env_file_encoding="utf-8",
         extra="ignore",
+        env_nested_delimiter="__",
+        cli_implicit_flags=True,
+        cli_kebab_case=True,
+        frozen=True,
     )
 
+    # --- NiceGUI Server ---
+    host: str = "0.0.0.0"
+    port: int = 8000
+    log_level: Literal["critical", "error", "warning", "info", "debug", "trace"] = "info"
+    reload: bool = False
+    log_dir: Path = Path("./data/logs")
+    log_file_name: NonEmptyStr = "transcription.log"
+    log_file_max_bytes: int = Field(default=10 * 1024 * 1024, gt=0)
+    log_file_backup_count: int = Field(default=5, ge=1)
+
     # --- AI provider ---
     provider: Provider = Provider.OPENROUTER
-    openrouter_api_key: str
-    provider_model: str | None = None
-    openrouter_http_referer: str | None = None
-    openrouter_app_title: str | None = None
+    openrouter_api_key: SecretStr
+    provider_model: NonEmptyStr | None = DEFAULT_PROVIDER_MODEL
+    provider_models: tuple[NonEmptyStr, ...] = ()
+    openrouter_http_referer: NonEmptyStr | None = None
+    openrouter_app_title: NonEmptyStr | None = None
+    default_prompt_name: PromptFilename = "transcribe_document.md"
+    transcription_temperature: Temperature | None = None
+    transcription_top_p: Probability | None = None
 
     # --- runtime environment ---
     environment: Literal["development", "test", "production"] = "development"
+    transcription_commit: NonEmptyStr | None = None
 
     # --- persistence ---
-    database_url: str = "sqlite:///./transcription.db"
-    bootstrap_schema_on_startup: bool | None = None
+    database: DatabaseSettings = Field(default_factory=SqliteSettings)
+    bootstrap_schema_on_startup: bool = False
     sqlite_check_same_thread: bool = False
 
     # --- filesystem paths ---
-    upload_dir: Path = Path("./uploads")
+    upload_dir: Path = Path("./data")
     prompt_dir: Path = Path("./prompts")
+    database_backup_dir: Path = Path("./data/backups")
 
     # --- worker reliability ---
-    worker_max_retries: int = 0
-    worker_retry_backoff_seconds: float = 0.0
-    worker_provider_timeout_seconds: float = Field(default=20.0, gt=0.0, le=20.0)
+    worker_max_retries: int = Field(default=0, ge=0)
+    # Bounded only from below. Vision transcription of a dense page routinely runs
+    # well past twenty seconds, so an upper cap here would silently fail real work.
+    worker_provider_timeout_seconds: float = Field(default=30.0, gt=0.0)
     worker_min_transcription_chars: int = Field(default=0, ge=0)
     worker_min_transcription_lines: int = Field(default=0, ge=0)
     worker_fail_on_finish_reason_length: bool = False
 
+    @field_validator("provider_models", mode="before")
+    @classmethod
+    def validate_provider_models_input(cls, value: object) -> object:
+        if value is None:
+            return ()
+        if isinstance(value, (list, tuple)) and not value:
+            raise ValueError("PROVIDER_MODELS must contain at least one model")
+        return value
+
+    @model_validator(mode="before")
+    @classmethod
+    def normalize_provider_models(cls, data: object) -> object:
+        """Build the immutable model selector with the configured default first.
+
+        This runs before field validation so the derived value is produced by
+        normal construction rather than by mutating a frozen instance.
+        """
+        if not isinstance(data, dict):
+            return data
+
+        default_model = data.get("provider_model") or DEFAULT_PROVIDER_MODEL
+        if not isinstance(default_model, str):
+            return data
+        default_model = default_model.strip()
+
+        configured = data.get("provider_models")
+        if configured is None:
+            configured = ()
+        elif isinstance(configured, str):
+            # Left as-is so the field validator can report the malformed value.
+            return {**data, "provider_model": default_model}
+        elif not isinstance(configured, (list, tuple)):
+            return {**data, "provider_model": default_model}
+        elif not configured:
+            # Preserved so validate_provider_models_input can reject it.
+            return {**data, "provider_model": default_model}
+
+        deduplicated: list[str] = []
+        for model in (default_model, *configured):
+            if not isinstance(model, str):
+                return {**data, "provider_model": default_model}
+            normalized = model.strip()
+            if normalized not in deduplicated:
+                deduplicated.append(normalized)
+
+        return {**data, "provider_model": default_model, "provider_models": tuple(deduplicated)}
+
     @property
     def should_bootstrap_schema(self) -> bool:
         """Return whether startup should auto-create schema for this environment."""
-        if self.bootstrap_schema_on_startup is not None:
+        if "bootstrap_schema_on_startup" in self.model_fields_set:
             return self.bootstrap_schema_on_startup
         return self.environment in {"development", "test"}
 
 
-_settings: ContextVar[Settings | None] = ContextVar("settings", default=None)
+@cache
+def get_settings(**kwargs: Any) -> Settings:
+    """Load cached settings without reading process CLI arguments."""
+    return Settings(_cli_parse_args=False, **kwargs)
 
 
-def get_settings(**kwargs) -> Settings:
-    settings = _settings.get()
-    if settings is None:
-        settings = Settings(**kwargs)  # pyright: ignore[reportCallIssue]
-        _settings.set(settings)
-    return settings
+def parse_cli_settings(args: Sequence[str] | None = None) -> Settings:
+    """Load settings with CLI arguments at the executable boundary."""
+    cli_args = True if args is None else list(args)
+    return Settings(_cli_parse_args=cli_args)
 
 
-LOGGING_CONFIG: dict[str, object] = {
+LOGGING_CONFIG: dict[str, Any] = {
     "version": 1,
     "disable_existing_loggers": False,
     "formatters": {
@@ -89,23 +199,39 @@ LOGGING_CONFIG: dict[str, object] = {
             "class": "logging.StreamHandler",
             "formatter": "standard",
             "stream": "ext://sys.stdout",
+        },
+        "file": {
+            "class": "logging.handlers.RotatingFileHandler",
+            "formatter": "standard",
+            "filename": str(Path("./data/logs") / "transcription.log"),
+            "maxBytes": 10 * 1024 * 1024,
+            "backupCount": 5,
+            "encoding": "utf-8",
         }
     },
     "root": {
         "level": "INFO",
-        "handlers": ["console"],
+        "handlers": ["console", "file"],
     },
     "loggers": {
         "transcription": {
             "level": "DEBUG",
-            "handlers": ["console"],
+            "handlers": ["console", "file"],
             "propagate": False,
         }
     },
 }
 
 
-def configure_logging() -> None:
+def configure_logging(settings: Settings | None = None) -> None:
     """Configure root logging once at startup."""
-    logging.config.dictConfig(LOGGING_CONFIG)
+    cfg = copy.deepcopy(LOGGING_CONFIG)
+    active_settings = settings or get_settings()
+    active_settings.log_dir.mkdir(parents=True, exist_ok=True)
+    file_handler = cfg["handlers"]["file"]
+    file_handler["filename"] = str(active_settings.log_dir / active_settings.log_file_name)
+    file_handler["maxBytes"] = active_settings.log_file_max_bytes
+    file_handler["backupCount"] = active_settings.log_file_backup_count
+    cfg["loggers"]["transcription"]["level"] = active_settings.log_level.upper()
+    logging.config.dictConfig(cfg)
     logger.debug("Logging configured")

src/transcription/db/__init__.py

@@ -1,6 +1,17 @@
 from .operations import create_all
+from .operations import reconcile_canonical_media_paths
+from .operations import reconcile_legacy_job_source_columns
 from .runtime import dispose_database_runtime
-from .runtime import get_session
 from .runtime import initialize_database_runtime
+from .session import session_scope
+from .session import transaction_scope
 
-__all__ = ["create_all", "dispose_database_runtime", "get_session", "initialize_database_runtime"]
+__all__ = [
+    "create_all",
+    "dispose_database_runtime",
+    "initialize_database_runtime",
+    "reconcile_canonical_media_paths",
+    "reconcile_legacy_job_source_columns",
+    "session_scope",
+    "transaction_scope",
+]

src/transcription/db/engine.py

@@ -0,0 +1,86 @@
+from typing import Any
+
+from sqlalchemy import URL
+from sqlalchemy import StaticPool
+from sqlalchemy.ext.asyncio import AsyncEngine
+from sqlalchemy.ext.asyncio import create_async_engine
+
+from ..config import PostgresSettings
+from ..config import Settings
+from ..config import SqliteSettings
+from ..config import get_settings
+
+
+def get_database_url(settings: Settings) -> str:
+    match settings.database:
+        case SqliteSettings(path=path):
+            url = URL.create(
+                drivername="sqlite+aiosqlite",
+                database=path,
+            )
+        case PostgresSettings() as database:
+            url = URL.create(
+                drivername="postgresql+asyncpg",
+                host=database.host,
+                port=database.port,
+                database=database.database,
+                username=database.user,
+                password=database.password.get_secret_value(),
+            )
+    return url.render_as_string(hide_password=False)
+
+
+def resolve_engine(settings: Settings | None = None) -> AsyncEngine:
+    active_settings = settings or get_settings()
+    return get_engine(
+        get_database_url(active_settings),
+        sqlite_check_same_thread=active_settings.sqlite_check_same_thread,
+    )
+
+
+_ENGINES: dict[str, AsyncEngine] = {}
+
+
+def _create_engine(database_url: str, *, sqlite_check_same_thread: bool) -> AsyncEngine:
+    kwargs: dict[str, Any] = {"echo": False, "pool_pre_ping": True}
+    if database_url.startswith("sqlite"):
+        kwargs["connect_args"] = {"check_same_thread": sqlite_check_same_thread}
+        if ":memory:" in database_url:
+            kwargs["poolclass"] = StaticPool
+
+    return create_async_engine(database_url, **kwargs)
+
+
+def get_engine(database_url: str, *, sqlite_check_same_thread: bool = False) -> AsyncEngine:
+    """Return the process-wide engine for ``database_url``, creating it on first use.
+
+    Engines are registered per URL so that disposing one leaves every other
+    database untouched.
+    """
+    engine = _ENGINES.get(database_url)
+    if engine is None:
+        engine = _create_engine(database_url, sqlite_check_same_thread=sqlite_check_same_thread)
+        _ENGINES[database_url] = engine
+    return engine
+
+
+async def dispose_engine(database_url: str) -> None:
+    """Dispose and unregister the engine for ``database_url`` only.
+
+    Unknown URLs are a no-op rather than provoking the creation of an engine
+    purely so that it can be thrown away.
+    """
+    engine = _ENGINES.pop(database_url, None)
+    if engine is not None:
+        await engine.dispose()
+
+
+async def dispose_all_engines() -> None:
+    while _ENGINES:
+        _, engine = _ENGINES.popitem()
+        await engine.dispose()
+
+
+async def refresh_engine(database_url: str) -> AsyncEngine:
+    await dispose_engine(database_url)
+    return get_engine(database_url)

src/transcription/db/loading.py

@@ -0,0 +1,45 @@
+"""Typed loader-option wrappers for SQLModel relationship attributes.
+
+SQLModel declares relationships with their runtime Python type, so
+``Document.jobs`` is annotated ``list[Job]`` even though at runtime it is an
+``InstrumentedAttribute``. SQLAlchemy's loader options are typed against
+``QueryableAttribute``, so every eager-load call site reads as a type error to a
+static checker even though the code is correct.
+
+These wrappers put that reinterpretation in one documented place instead of
+scattering a suppression comment across every eager-load call. Import
+``selectinload`` and ``defer`` from here rather than from ``sqlalchemy.orm``.
+
+Multi-level eager loads must keep using the chained form --
+``selectinload(A.b).selectinload(orm_attribute(B.c))`` -- and not the varargs
+form ``selectinload(A.b, B.c)``. The two produce the same loader path, but
+varargs applies the selectin strategy only to the last element while the
+intermediate falls back to its default strategy. Every relationship here
+declares ``lazy="raise"``, so the varargs form raises at render time.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+from typing import cast
+
+from sqlalchemy.orm import defer as _defer
+from sqlalchemy.orm import selectinload as _selectinload
+from sqlalchemy.orm.attributes import QueryableAttribute
+from sqlalchemy.orm.strategy_options import _AbstractLoad
+
+
+def orm_attribute(attribute: object) -> QueryableAttribute[Any]:
+    """Reinterpret a SQLModel relationship or field as its ORM descriptor."""
+    return cast("QueryableAttribute[Any]", attribute)
+
+
+def selectinload(*keys: object) -> _AbstractLoad:
+    """``sqlalchemy.orm.selectinload`` accepting SQLModel-annotated attributes."""
+    return _selectinload(*(orm_attribute(key) for key in keys))
+
+
+def defer(*keys: object, raiseload: bool = False) -> _AbstractLoad:
+    """``sqlalchemy.orm.defer`` accepting SQLModel-annotated attributes."""
+    first, *rest = (orm_attribute(key) for key in keys)
+    return _defer(first, *rest, raiseload=raiseload)

src/transcription/db/migration.py

@@ -0,0 +1,358 @@
+from __future__ import annotations
+
+import base64
+import json
+import shutil
+from dataclasses import dataclass
+from datetime import date
+from datetime import datetime
+from pathlib import Path
+from typing import Any
+from uuid import UUID
+from uuid import uuid4
+
+from sqlalchemy import URL
+from sqlalchemy import MetaData
+from sqlalchemy import Table
+from sqlalchemy import create_engine
+from sqlalchemy import inspect as sqlalchemy_inspect
+from sqlalchemy import select
+from sqlalchemy.engine import Engine
+from sqlmodel import SQLModel
+
+from transcription.config import Settings
+from transcription.config import get_settings
+
+# Register table metadata.
+from transcription.db import models as _models  # noqa: F401
+from transcription.db.engine import get_database_url
+
+EXPORT_TABLE_ORDER = (
+    "document_type",
+    "person_role",
+    "tag",
+    "document",
+    "person",
+    "photo",
+    "document_person",
+    "document_tag",
+    "job",
+    "source",
+    "job_source",
+    "execution_attempt",
+)
+
+BYTES_FIELDS = {"transport_body"}
+
+
+@dataclass(frozen=True)
+class MigrationPaths:
+    source_db_url: str
+    target_db_url: str
+    source_upload_dir: Path
+    target_upload_dir: Path
+    bundle_dir: Path
+
+
+def export_bundle(*, source_db_url: str, source_upload_dir: Path, bundle_dir: Path) -> None:
+    bundle_dir.mkdir(parents=True, exist_ok=True)
+    export_json = bundle_dir / "database.json"
+    uploads_bundle_dir = bundle_dir / "uploads"
+
+    payload: dict[str, Any] = {
+        "schema_name": "transcription.export-import",
+        "schema_version": "1",
+        "created_at": datetime.now().isoformat(),
+        "tables": {},
+    }
+
+    engine = create_engine(source_db_url)
+    legacy_portrait_rows: list[dict[str, Any]] = []
+    source_has_photo_table = False
+    try:
+        inspector = sqlalchemy_inspect(engine)
+        source_tables = set(inspector.get_table_names())
+        source_has_photo_table = "photo" in source_tables
+        metadata = MetaData()
+        metadata.reflect(bind=engine)
+        current_metadata = SQLModel.metadata
+
+        with engine.connect() as connection:
+            for table_name in EXPORT_TABLE_ORDER:
+                if table_name not in source_tables:
+                    payload["tables"][table_name] = []
+                    continue
+
+                source_table = metadata.tables[table_name]
+                target_table = current_metadata.tables[table_name]
+                export_columns = [
+                    column.name for column in target_table.columns if column.name in source_table.columns
+                ]
+                if table_name == "person" and "portrait_path" in source_table.columns:
+                    legacy_portrait_rows = connection.execute(
+                        select(source_table.c["id"], source_table.c["portrait_path"]).where(
+                            source_table.c["portrait_path"].is_not(None)
+                        )
+                    ).mappings().all()
+                rows = connection.execute(select(*(source_table.c[name] for name in export_columns))).mappings().all()
+                payload["tables"][table_name] = [
+                    _serialize_row(row, table_name=table_name, source_upload_dir=source_upload_dir) for row in rows
+                ]
+    finally:
+        engine.dispose()
+
+    if uploads_bundle_dir.exists():
+        shutil.rmtree(uploads_bundle_dir)
+    if source_upload_dir.exists():
+        shutil.copytree(source_upload_dir, uploads_bundle_dir)
+    else:
+        uploads_bundle_dir.mkdir(parents=True, exist_ok=True)
+
+    _prepare_photo_payload_and_uploads(
+        payload=payload,
+        uploads_bundle_dir=uploads_bundle_dir,
+        source_has_photo_table=source_has_photo_table,
+        legacy_portrait_rows=legacy_portrait_rows,
+    )
+    _relocate_homepage_markdown(uploads_bundle_dir=uploads_bundle_dir)
+
+    export_json.write_text(json.dumps(payload, indent=2), encoding="utf-8")
+
+
+def import_bundle(*, target_db_url: str, target_upload_dir: Path, bundle_dir: Path) -> None:
+    export_json = bundle_dir / "database.json"
+    uploads_bundle_dir = bundle_dir / "uploads"
+    payload = json.loads(export_json.read_text(encoding="utf-8"))
+
+    engine = create_engine(target_db_url)
+    try:
+        SQLModel.metadata.create_all(engine)
+        with engine.begin() as connection:
+            for table_name in reversed(EXPORT_TABLE_ORDER):
+                table = SQLModel.metadata.tables[table_name]
+                connection.execute(table.delete())
+
+            for table_name in EXPORT_TABLE_ORDER:
+                rows = payload.get("tables", {}).get(table_name, [])
+                if not rows:
+                    continue
+                table = SQLModel.metadata.tables[table_name]
+                connection.execute(table.insert(), [_deserialize_row(row, table) for row in rows])
+    finally:
+        engine.dispose()
+
+    if target_upload_dir.exists():
+        shutil.rmtree(target_upload_dir)
+    target_upload_dir.mkdir(parents=True, exist_ok=True)
+    if uploads_bundle_dir.exists():
+        shutil.copytree(uploads_bundle_dir, target_upload_dir, dirs_exist_ok=True)
+
+
+def migrate_via_bundle(paths: MigrationPaths) -> None:
+    export_bundle(
+        source_db_url=paths.source_db_url,
+        source_upload_dir=paths.source_upload_dir,
+        bundle_dir=paths.bundle_dir,
+    )
+    import_bundle(
+        target_db_url=paths.target_db_url,
+        target_upload_dir=paths.target_upload_dir,
+        bundle_dir=paths.bundle_dir,
+    )
+
+
+def sqlite_url_from_path(path: Path) -> str:
+    return URL.create(drivername="sqlite", database=str(path)).render_as_string(hide_password=False)
+
+
+def default_sync_db_url(settings: Settings | None = None) -> str:
+    runtime_settings = settings or get_settings()
+    return get_database_url(runtime_settings).replace("+aiosqlite", "").replace("+asyncpg", "").replace("+psycopg", "")
+
+
+def _serialize_row(row: dict[str, Any], *, table_name: str, source_upload_dir: Path) -> dict[str, Any]:
+    serialized: dict[str, Any] = {}
+    for key, value in row.items():
+        serialized_value = _serialize_value(key, value)
+        if table_name == "source" and key == "file_path" and isinstance(serialized_value, str):
+            serialized[key] = _canonical_media_relative_path(
+                serialized_value,
+                source_upload_dir=source_upload_dir,
+                preferred_prefix="documents/",
+            )
+            continue
+        if table_name == "photo" and key == "path" and isinstance(serialized_value, str):
+            serialized[key] = _canonical_media_relative_path(
+                serialized_value,
+                source_upload_dir=source_upload_dir,
+                preferred_prefix="photos/",
+            )
+            continue
+        serialized[key] = serialized_value
+    return serialized
+
+
+def _serialize_value(key: str, value: Any) -> Any:
+    if isinstance(value, UUID):
+        return str(value)
+    if isinstance(value, (datetime, date)):
+        return value.isoformat()
+    if isinstance(value, bytes):
+        return {"encoding": "base64", "data": base64.b64encode(value).decode("ascii")}
+    if isinstance(value, dict):
+        return {str(k): _serialize_value("", v) for k, v in value.items()}
+    if isinstance(value, list):
+        return [_serialize_value("", item) for item in value]
+    return value
+
+
+def _deserialize_row(row: dict[str, Any], table: Table) -> dict[str, Any]:
+    deserialized: dict[str, Any] = {}
+    for key, value in row.items():
+        if key in BYTES_FIELDS and isinstance(value, dict) and value.get("encoding") == "base64":
+            deserialized[key] = base64.b64decode(value["data"])
+            continue
+        if key in table.columns:
+            try:
+                python_type: type[Any] = table.columns[key].type.python_type
+            except NotImplementedError:
+                deserialized[key] = value
+                continue
+            deserialized[key] = _deserialize_value(python_type, value)
+    return deserialized
+
+
+def _deserialize_value(python_type: type[Any], value: Any) -> Any:
+    if value is None:
+        return None
+    if python_type is UUID and isinstance(value, str):
+        return UUID(value)
+    if python_type is datetime and isinstance(value, str):
+        return datetime.fromisoformat(value)
+    if python_type is date and isinstance(value, str):
+        return date.fromisoformat(value)
+    return value
+
+
+def _canonical_media_relative_path(value: str, *, source_upload_dir: Path, preferred_prefix: str) -> str:
+    normalized = value.strip().replace("\\", "/")
+    lowered = normalized.casefold()
+    upload_root = source_upload_dir.resolve().as_posix().casefold().rstrip("/")
+    if lowered.startswith(upload_root + "/"):
+        normalized = normalized[len(source_upload_dir.resolve().as_posix()) + 1 :]
+        lowered = normalized.casefold()
+
+    if lowered.startswith("/uploads/"):
+        normalized = normalized[len("/uploads/") :]
+        lowered = normalized.casefold()
+    elif lowered.startswith("uploads/"):
+        normalized = normalized[len("uploads/") :]
+        lowered = normalized.casefold()
+    elif lowered.startswith("data/"):
+        normalized = normalized[len("data/") :]
+        lowered = normalized.casefold()
+
+    if preferred_prefix == "persons/" and lowered.startswith("portraits/"):
+        normalized = "persons/" + normalized[len("portraits/") :]
+        lowered = normalized.casefold()
+
+    for prefix in ("documents/", "photos/", "persons/"):
+        marker = f"/{prefix}"
+        index = lowered.find(marker)
+        if index >= 0:
+            normalized = normalized[index + 1 :]
+            lowered = normalized.casefold()
+            break
+
+    if not lowered.startswith(preferred_prefix):
+        return normalized
+    return Path(normalized).as_posix()
+
+
+def _prepare_photo_payload_and_uploads(
+    *,
+    payload: dict[str, Any],
+    uploads_bundle_dir: Path,
+    source_has_photo_table: bool,
+    legacy_portrait_rows: list[dict[str, Any]],
+) -> None:
+    photo_rows = payload.setdefault("tables", {}).setdefault("photo", [])
+    photos_dir = uploads_bundle_dir / "photos"
+    photos_dir.mkdir(parents=True, exist_ok=True)
+
+    if source_has_photo_table:
+        return
+
+    now_iso = datetime.now().isoformat()
+    for row in legacy_portrait_rows:
+        portrait_path = row.get("portrait_path")
+        person_id = row.get("id")
+        if not isinstance(portrait_path, str) or not portrait_path.strip():
+            continue
+        if person_id is None:
+            continue
+        canonical = _canonical_media_relative_path(
+            portrait_path,
+            source_upload_dir=uploads_bundle_dir,
+            preferred_prefix="persons/",
+        )
+        source_file = uploads_bundle_dir / canonical
+        suffix = Path(canonical).suffix.lower() or ".jpg"
+        photo_id = str(uuid4())
+        relative_path = f"photos/{photo_id}{suffix}"
+        if source_file.exists():
+            target_file = uploads_bundle_dir / relative_path
+            target_file.parent.mkdir(parents=True, exist_ok=True)
+            shutil.copy2(source_file, target_file)
+        else:
+            relative_path = canonical
+        photo_rows.append(
+            {
+                "id": photo_id,
+                "person_id": str(person_id),
+                "path": relative_path,
+                "description": None,
+                "is_primary": True,
+                "created_at": now_iso,
+                "updated_at": now_iso,
+            }
+        )
+
+    legacy_homepage_dir = uploads_bundle_dir / "homepage"
+    if not legacy_homepage_dir.exists():
+        return
+
+    homepage_images = sorted(
+        [
+            path
+            for path in legacy_homepage_dir.iterdir()
+            if path.is_file() and path.suffix.lower() in {".jpg", ".jpeg", ".png", ".gif", ".webp", ".bmp", ".tif", ".tiff"}
+        ],
+        key=lambda path: (path.stat().st_mtime, path.name),
+    )
+    for index, image_path in enumerate(homepage_images):
+        photo_id = str(uuid4())
+        relative_path = f"photos/{photo_id}{image_path.suffix.lower()}"
+        target_file = uploads_bundle_dir / relative_path
+        target_file.parent.mkdir(parents=True, exist_ok=True)
+        shutil.copy2(image_path, target_file)
+        photo_rows.append(
+            {
+                "id": photo_id,
+                "person_id": None,
+                "path": relative_path,
+                "description": None,
+                "is_primary": index == 0,
+                "created_at": now_iso,
+                "updated_at": now_iso,
+            }
+        )
+
+
+def _relocate_homepage_markdown(*, uploads_bundle_dir: Path) -> None:
+    legacy_markdown = uploads_bundle_dir / "homepage" / "homepage.md"
+    target_markdown = uploads_bundle_dir / "homepage.md"
+    if not legacy_markdown.exists() or target_markdown.exists():
+        return
+    target_markdown.parent.mkdir(parents=True, exist_ok=True)
+    shutil.copy2(legacy_markdown, target_markdown)

src/transcription/db/models.py

@@ -0,0 +1,522 @@
+"""SQLModel domain models for the V3 transcription system."""
+
+from datetime import UTC
+from datetime import date
+from datetime import datetime
+from enum import StrEnum
+from typing import Any
+from typing import Optional
+from uuid import UUID
+from uuid import uuid4
+
+from pydantic import JsonValue
+from sqlalchemy import JSON
+from sqlalchemy import BigInteger
+from sqlalchemy import Column
+from sqlalchemy import Enum as SAEnum
+from sqlalchemy import ForeignKey
+from sqlalchemy import Index
+from sqlalchemy import LargeBinary
+from sqlalchemy import UniqueConstraint
+from sqlalchemy import Uuid
+from sqlalchemy import inspect as sqlalchemy_inspect
+from sqlalchemy.dialects.postgresql import JSONB
+from sqlalchemy.exc import NoInspectionAvailable
+from sqlalchemy.orm.state import InstanceState
+from sqlalchemy.types import TypeDecorator
+from sqlmodel import Field
+from sqlmodel import Relationship
+from sqlmodel import SQLModel
+
+
+def _loaded_attribute(instance: object, attribute: str) -> Any | None:
+    """Return ``attribute`` only when it is already loaded on ``instance``.
+
+    Relationships in this module declare ``lazy="raise"``, so reading an
+    unloaded attribute is an error rather than a silent query. Callers that
+    render optional detail use this to distinguish "not loaded" from "absent"
+    without catching exceptions indiscriminately.
+    """
+    try:
+        state: InstanceState[Any] = sqlalchemy_inspect(instance, raiseerr=True)
+    except NoInspectionAvailable:
+        return None
+    if attribute in state.unloaded:
+        return None
+    return state.dict.get(attribute)
+
+
+class JSONBCompat(TypeDecorator):
+    """JSONB for PostgreSQL and JSON for SQLite/testing backends."""
+
+    impl = JSON(none_as_null=True)
+
+    def load_dialect_impl(self, dialect):
+        if dialect.name == "postgresql":
+            return dialect.type_descriptor(JSONB(none_as_null=True))
+        return dialect.type_descriptor(JSON(none_as_null=True))
+
+
+class JobStatus(StrEnum):
+    QUEUED = "queued"
+    PROCESSING = "processing"
+    TRANSCRIBED = "transcribed"
+    PARTIAL_SUCCESS = "partial_success"
+    FAILED = "failed"
+
+
+class JobSourceStatus(StrEnum):
+    PENDING = "pending"
+    TRANSCRIBED = "transcribed"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+
+
+class JobPurpose(StrEnum):
+    TRANSCRIPTION = "transcription"
+    RETRANSCRIPTION = "retranscription"
+
+
+class DocumentType(SQLModel, table=True):
+    """Registry of allowed document types."""
+
+    __tablename__ = "document_type"
+
+    id: UUID = Field(default_factory=uuid4, primary_key=True)
+    semantic_key: str | None = Field(default=None, index=True, unique=True)
+    label: str
+    normalized_label: str = Field(index=True, unique=True)
+    is_active: bool = True
+    created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
+    updated_at: datetime = Field(
+        default_factory=lambda: datetime.now(UTC),
+        sa_column_kwargs={"onupdate": lambda: datetime.now(UTC)},
+    )
+
+    documents: list["Document"] = Relationship(
+        back_populates="document_type_ref", sa_relationship_kwargs={"lazy": "raise"}
+    )
+
+
+class PersonRole(SQLModel, table=True):
+    """Registry of allowed document-person relationship roles."""
+
+    __tablename__ = "person_role"
+
+    id: UUID = Field(default_factory=uuid4, primary_key=True)
+    semantic_key: str | None = Field(default=None, index=True, unique=True)
+    label: str
+    normalized_label: str = Field(index=True, unique=True)
+    is_active: bool = True
+    created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
+    updated_at: datetime = Field(
+        default_factory=lambda: datetime.now(UTC),
+        sa_column_kwargs={"onupdate": lambda: datetime.now(UTC)},
+    )
+
+    document_people: list["DocumentPerson"] = Relationship(
+        back_populates="role_ref", sa_relationship_kwargs={"lazy": "raise"}
+    )
+
+
+class Tag(SQLModel, table=True):
+    """Registry of labels that can be attached to Documents."""
+
+    __tablename__ = "tag"
+
+    id: UUID = Field(default_factory=uuid4, primary_key=True)
+    semantic_key: str | None = Field(default=None, index=True, unique=True)
+    label: str
+    normalized_label: str = Field(index=True, unique=True)
+    is_active: bool = True
+    created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
+    updated_at: datetime = Field(
+        default_factory=lambda: datetime.now(UTC),
+        sa_column_kwargs={"onupdate": lambda: datetime.now(UTC)},
+    )
+
+    document_tags: list["DocumentTag"] = Relationship(
+        back_populates="tag_ref",
+        sa_relationship_kwargs={"lazy": "raise"},
+    )
+
+
+class Document(SQLModel, table=True):
+    """An historical document."""
+
+    id: UUID = Field(default_factory=uuid4, primary_key=True)
+    name: str
+    document_type_id: UUID | None = Field(default=None, foreign_key="document_type.id", index=True)
+    document_date: date | None = None
+    document_date_raw: str | None = None
+    location_created: str | None = None
+    notes: str | None = None
+    archive_identifier: str | None = None
+    created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
+    updated_at: datetime = Field(
+        default_factory=lambda: datetime.now(UTC),
+        sa_column_kwargs={"onupdate": lambda: datetime.now(UTC)},
+    )
+
+    jobs: list["Job"] = Relationship(back_populates="document", sa_relationship_kwargs={"lazy": "raise"})
+    sources: list["Source"] = Relationship(back_populates="document", sa_relationship_kwargs={"lazy": "raise"})
+    document_people: list["DocumentPerson"] = Relationship(
+        back_populates="document", sa_relationship_kwargs={"lazy": "raise"}
+    )
+    document_tags: list["DocumentTag"] = Relationship(
+        back_populates="document",
+        sa_relationship_kwargs={"lazy": "raise"},
+    )
+    document_type_ref: Optional["DocumentType"] = Relationship(
+        back_populates="documents", sa_relationship_kwargs={"lazy": "raise"}
+    )
+
+
+class Person(SQLModel, table=True):
+    """A historical person linked to one or more documents."""
+
+    id: UUID = Field(default_factory=uuid4, primary_key=True)
+    full_name: str
+    display_name: str | None = None
+    maiden_name: str | None = None
+    birth_date: date | None = None
+    birth_date_raw: str | None = None
+    birth_place: str | None = None
+    death_date: date | None = None
+    death_date_raw: str | None = None
+    death_place: str | None = None
+    biography: str | None = None
+    family_search_id: str | None = Field(default=None, unique=True)
+    metadata_: dict[str, JsonValue] | None = Field(
+        default=None,
+        sa_column=Column("metadata", JSONBCompat(), nullable=True),
+    )
+    created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
+    updated_at: datetime = Field(
+        default_factory=lambda: datetime.now(UTC),
+        sa_column_kwargs={"onupdate": lambda: datetime.now(UTC)},
+    )
+
+    document_people: list["DocumentPerson"] = Relationship(
+        back_populates="person", sa_relationship_kwargs={"lazy": "raise"}
+    )
+    photos: list["Photo"] = Relationship(
+        back_populates="person",
+        sa_relationship_kwargs={"lazy": "raise"},
+    )
+
+
+class Photo(SQLModel, table=True):
+    """A reusable image record for Person and homepage galleries."""
+
+    __tablename__ = "photo"
+
+    id: UUID = Field(default_factory=uuid4, primary_key=True)
+    person_id: UUID | None = Field(default=None, foreign_key="person.id", index=True)
+    path: str
+    description: str | None = None
+    is_primary: bool = False
+    created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
+    updated_at: datetime = Field(
+        default_factory=lambda: datetime.now(UTC),
+        sa_column_kwargs={"onupdate": lambda: datetime.now(UTC)},
+    )
+
+    person: Optional["Person"] = Relationship(
+        back_populates="photos",
+        sa_relationship_kwargs={"lazy": "raise"},
+    )
+
+
+class DocumentPerson(SQLModel, table=True):
+    """Associates documents with people in a given role."""
+
+    __tablename__ = "document_person"
+
+    id: UUID = Field(default_factory=uuid4, primary_key=True)
+    document_id: UUID = Field(foreign_key="document.id", index=True)
+    person_id: UUID = Field(foreign_key="person.id", index=True)
+    role_id: UUID = Field(foreign_key="person_role.id", index=True)
+    created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
+    updated_at: datetime = Field(
+        default_factory=lambda: datetime.now(UTC),
+        sa_column_kwargs={"onupdate": lambda: datetime.now(UTC)},
+    )
+
+    __table_args__ = (UniqueConstraint("document_id", "person_id", name="uq_document_person"),)
+
+    document: Optional["Document"] = Relationship(
+        back_populates="document_people", sa_relationship_kwargs={"lazy": "raise"}
+    )
+    person: Optional["Person"] = Relationship(
+        back_populates="document_people", sa_relationship_kwargs={"lazy": "raise"}
+    )
+    role_ref: Optional["PersonRole"] = Relationship(
+        back_populates="document_people", sa_relationship_kwargs={"lazy": "raise"}
+    )
+
+
+class DocumentTag(SQLModel, table=True):
+    """Associates Documents with Tags."""
+
+    __tablename__ = "document_tag"
+
+    id: UUID = Field(default_factory=uuid4, primary_key=True)
+    document_id: UUID = Field(foreign_key="document.id", index=True)
+    tag_id: UUID = Field(foreign_key="tag.id", index=True)
+    created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
+    updated_at: datetime = Field(
+        default_factory=lambda: datetime.now(UTC),
+        sa_column_kwargs={"onupdate": lambda: datetime.now(UTC)},
+    )
+
+    __table_args__ = (UniqueConstraint("document_id", "tag_id", name="uq_document_tag"),)
+
+    document: Optional["Document"] = Relationship(
+        back_populates="document_tags",
+        sa_relationship_kwargs={"lazy": "raise"},
+    )
+    tag_ref: Optional["Tag"] = Relationship(
+        back_populates="document_tags",
+        sa_relationship_kwargs={"lazy": "raise"},
+    )
+
+
+class Job(SQLModel, table=True):
+    """A transcription job tied to a single document."""
+
+    __table_args__ = (Index("ix_job_status_date_created", "status", "date_created"),)
+
+    id: UUID = Field(default_factory=uuid4, primary_key=True)
+    document_id: UUID = Field(foreign_key="document.id", index=True)
+    status: JobStatus = Field(
+        default=JobStatus.QUEUED,
+        sa_column=Column(
+            SAEnum(
+                JobStatus,
+                values_callable=lambda enum_cls: [item.value for item in enum_cls],
+                native_enum=False,
+            ),
+            nullable=False,
+        ),
+    )
+    retry_count: int = Field(default=0, ge=0)
+    purpose: JobPurpose = Field(
+        default=JobPurpose.TRANSCRIPTION,
+        sa_column=Column(
+            SAEnum(
+                JobPurpose,
+                values_callable=lambda enum_cls: [item.value for item in enum_cls],
+                native_enum=False,
+            ),
+            nullable=False,
+            default=JobPurpose.TRANSCRIPTION.value,
+        ),
+    )
+    date_created: datetime = Field(default_factory=lambda: datetime.now(UTC))
+    date_updated: datetime = Field(
+        default_factory=lambda: datetime.now(UTC),
+        sa_column_kwargs={"onupdate": lambda: datetime.now(UTC)},
+    )
+    provider: str | None = None
+    model: str | None = None
+    prompt_name: str | None = None
+    prompt_hash: str | None = None
+    system_prompt: str | None = None
+    user_prompt: str | None = None
+    temperature: float | None = None
+    top_p: float | None = None
+
+    document: Optional["Document"] = Relationship(back_populates="jobs", sa_relationship_kwargs={"lazy": "raise"})
+    job_sources: list["JobSource"] = Relationship(back_populates="job", sa_relationship_kwargs={"lazy": "raise"})
+
+    @property
+    def filename(self) -> str:
+        """Return the filename of the first loaded source, when available.
+
+        Relationships on this model use ``lazy="raise"``, so this deliberately
+        inspects load state rather than triggering (or swallowing) a lazy load:
+        a read model that did not eager-load its sources gets "unknown" instead
+        of an unhandled error, and genuine errors are no longer hidden.
+        """
+        for job_source in _loaded_attribute(self, "job_sources") or ():
+            source = _loaded_attribute(job_source, "source")
+            if source is not None:
+                return source.filename
+
+        return "unknown"
+
+
+class Source(SQLModel, table=True):
+    """A document source image or PDF page."""
+
+    id: UUID = Field(default_factory=uuid4, primary_key=True)
+    document_id: UUID = Field(foreign_key="document.id", index=True)
+    page_number: int = Field(default=1, ge=1)
+    upload_name: str
+    filename: str
+    file_path: str
+    file_hash: str
+    file_size_bytes: int = Field(sa_column=Column(BigInteger(), nullable=False))
+    raw_transcription: str | None = None
+    preferred_execution_attempt_id: UUID | None = Field(
+        default=None,
+        sa_column=Column(
+            Uuid(),
+            # use_alter breaks the source / job_source / execution_attempt cycle so
+            # metadata.create_all can order table creation on every dialect.
+            ForeignKey(
+                "execution_attempt.id",
+                use_alter=True,
+                name="fk_source_preferred_execution_attempt_id",
+            ),
+            nullable=True,
+            index=True,
+        ),
+    )
+    revised_text: str | None = None
+    date_uploaded: datetime = Field(default_factory=lambda: datetime.now(UTC))
+    date_revised: datetime | None = None
+
+    document: Optional["Document"] = Relationship(
+        back_populates="sources",
+        sa_relationship_kwargs={"lazy": "raise"},
+    )
+    job_sources: list["JobSource"] = Relationship(
+        back_populates="source",
+        sa_relationship_kwargs={"lazy": "raise"},
+    )
+
+    @property
+    def latest_job_source(self) -> Optional["JobSource"]:
+        """Return the most recent job execution record for this source.
+
+        ``JobSource`` carries no timestamp of its own, so recency is the parent
+        job's creation time. ``(job_id, source_id)`` is unique per source, so
+        this is exactly "the most recent job that included this page".
+        """
+        job_sources = _loaded_attribute(self, "job_sources") or ()
+        dated = [
+            (job, job_source)
+            for job_source in job_sources
+            if (job := _loaded_attribute(job_source, "job")) is not None
+        ]
+        if dated:
+            return max(dated, key=lambda pair: pair[0].date_created)[1]
+        return job_sources[0] if job_sources else None
+
+    @property
+    def latest_status(self) -> JobSourceStatus | None:
+        """Return the execution status of the latest job run."""
+        latest = self.latest_job_source
+        return latest.status if latest else None
+
+    @property
+    def latest_error_detail(self) -> str | None:
+        """Return the error detail of the latest attempt on the latest job run.
+
+        Failure detail lives on ``ExecutionAttempt``; ``JobSource`` records only
+        which page a job is working on and how far it got.
+        """
+        latest = self.latest_job_source
+        if latest is None:
+            return None
+        attempts = _loaded_attribute(latest, "execution_attempts") or ()
+        if not attempts:
+            return None
+        latest_attempt = max(attempts, key=lambda item: item.attempt_number)
+        return latest_attempt.error_detail
+
+    @property
+    def document_name(self) -> str | None:
+        """Return the parent document name if loaded."""
+        return self.document.name if self.document else None
+
+
+class JobSource(SQLModel, table=True):
+    """A single AI execution record for one source page."""
+
+    __tablename__ = "job_source"
+    __table_args__ = (UniqueConstraint("job_id", "source_id", name="uq_job_source_job_source"),)
+
+    id: UUID = Field(default_factory=uuid4, primary_key=True)
+    job_id: UUID = Field(foreign_key="job.id", index=True)
+    source_id: UUID = Field(foreign_key="source.id", index=True)
+    status: JobSourceStatus = Field(
+        default=JobSourceStatus.PENDING,
+        sa_column=Column(
+            SAEnum(
+                JobSourceStatus,
+                values_callable=lambda enum_cls: [item.value for item in enum_cls],
+                native_enum=False,
+            ),
+            nullable=False,
+        ),
+    )
+
+    job: Optional["Job"] = Relationship(back_populates="job_sources", sa_relationship_kwargs={"lazy": "raise"})
+    source: Optional["Source"] = Relationship(back_populates="job_sources", sa_relationship_kwargs={"lazy": "raise"})
+    execution_attempts: list["ExecutionAttempt"] = Relationship(
+        back_populates="job_source",
+        sa_relationship_kwargs={"lazy": "noload", "order_by": "ExecutionAttempt.attempt_number"},
+    )
+
+
+class ExecutionAttempt(SQLModel, table=True):
+    """Immutable evidence for one provider call attempt."""
+
+    __tablename__ = "execution_attempt"
+    __table_args__ = (UniqueConstraint("job_id", "source_id", "attempt_number", name="uq_execution_attempt_number"),)
+
+    id: UUID = Field(default_factory=uuid4, primary_key=True)
+    job_source_id: UUID = Field(foreign_key="job_source.id", index=True)
+    job_id: UUID = Field(foreign_key="job.id", index=True)
+    source_id: UUID = Field(foreign_key="source.id", index=True)
+    attempt_number: int = Field(ge=1)
+    status: JobSourceStatus = Field(
+        sa_column=Column(
+            # Declared identically to job_source.status. Without values_callable
+            # SQLAlchemy persists enum *names*, which is defect [45]: the two
+            # columns spelled the same status differently and never compared equal.
+            SAEnum(
+                JobSourceStatus,
+                values_callable=lambda enum_cls: [item.value for item in enum_cls],
+                native_enum=False,
+            ),
+            nullable=False,
+        )
+    )
+    provider: str
+    model: str | None = None
+    request_manifest: dict[str, JsonValue] | None = Field(default=None, sa_column=Column(JSONBCompat(), nullable=True))
+    request_manifest_sha256: str | None = None
+    request_manifest_schema_version: str | None = None
+    response_received: bool = False
+    transport_status_code: int | None = None
+    transport_body: bytes | None = Field(default=None, sa_column=Column(LargeBinary(), nullable=True))
+    transport_content_type: str | None = None
+    transport_content_encoding: str | None = None
+    transport_safe_headers: dict[str, JsonValue] | None = Field(
+        default=None, sa_column=Column(JSONBCompat(), nullable=True)
+    )
+    router_request_id: str | None = None
+    router_generation_id: str | None = None
+    sdk_response_snapshot: dict[str, JsonValue] | None = Field(
+        default=None, sa_column=Column(JSONBCompat(), nullable=True)
+    )
+    normalized_metadata: dict[str, JsonValue] | None = Field(
+        default=None, sa_column=Column(JSONBCompat(), nullable=True)
+    )
+    software_context: dict[str, JsonValue] | None = Field(default=None, sa_column=Column(JSONBCompat(), nullable=True))
+    raw_transcription: str | None = None
+    error_category: str | None = None
+    error_detail: str | None = None
+    failure_phase: str | None = None
+    started_at: datetime
+    finished_at: datetime
+    duration_ms: int = Field(ge=0)
+    created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
+
+    job_source: Optional["JobSource"] = Relationship(
+        back_populates="execution_attempts", sa_relationship_kwargs={"lazy": "raise"}
+    )

src/transcription/db/operations.py

@@ -1,79 +1,188 @@
 from __future__ import annotations
 
 import logging
+from pathlib import Path
 
-from sqlalchemy import inspect
+from sqlalchemy import inspect as sqlalchemy_inspect
 from sqlalchemy import text
-from sqlalchemy.engine import Connection
 from sqlalchemy.ext.asyncio import AsyncEngine
+from sqlalchemy.ext.asyncio import async_sessionmaker
 from sqlmodel import SQLModel
 from sqlmodel import select
 from sqlmodel.ext.asyncio.session import AsyncSession
 
-from ..models import Job
-from ..models import JobStatus
-from .runtime import get_engine
+from .engine import resolve_engine
+from .models import DocumentType
+from .models import PersonRole
+from .registries import BUILT_IN_DOCUMENT_TYPES
+from .registries import BUILT_IN_PERSON_ROLES
 
 logger = logging.getLogger(__name__)
 
 
-async def get_next_queued_job(*, session: AsyncSession) -> Job | None:
-    """Get the next queued job, if any."""
-    result = await session.exec(
-        select(Job)
-        .where(Job.status == JobStatus.QUEUED)
-        .order_by(Job.date_created)  # pyright: ignore[reportArgumentType]
-        .limit(1)
-    )  # fmt: skip
-    return result.first()
-
-
 async def create_all(*, engine: AsyncEngine | None = None) -> None:
-    """Create all tables on the selected engine."""
+    """Create any missing tables on the selected engine."""
     # Import models so SQLModel metadata is fully registered before bootstrap.
-    from transcription import models as _models  # noqa: F401
+    from transcription.db import models as _models  # noqa: F401
 
-    active_engine = engine or get_engine()
+    active_engine = engine or resolve_engine()
     async with active_engine.begin() as connection:
         await connection.run_sync(SQLModel.metadata.create_all)
-        await connection.run_sync(_ensure_sqlite_compat_columns)
+    await seed_registry_defaults(engine=active_engine)
     logger.debug("Database schema bootstrap complete for database_url=%s", active_engine.url)
 
 
-def _ensure_sqlite_compat_columns(connection: Connection) -> None:
-    """Apply lightweight dev/test SQLite compatibility column patches.
+async def reconcile_legacy_job_source_columns(*, engine: AsyncEngine | None = None) -> int:
+    """Remove stale V4.6 ``job_source`` evidence columns from existing databases.
 
-    This keeps local bootstrap resilient when models evolve but no full
-    migration tooling is in place yet.
+    Runtime models define ``job_source`` as a queue/projection table only. If an
+    older database still carries the retired evidence columns, writes can fail
+    on stale constraints (for example ``executed_at NOT NULL``).
     """
-    if connection.engine.url.get_backend_name() != "sqlite":
-        return
+    active_engine = engine or resolve_engine()
+    if not hasattr(active_engine, "begin"):
+        return 0
 
-    inspector = inspect(connection)
-    table_names = set(inspector.get_table_names())
+    def _reconcile(sync_connection) -> int:
+        inspector = sqlalchemy_inspect(sync_connection)
+        table_names = set(inspector.get_table_names())
+        if "job_source" not in table_names:
+            return 0
+        present_columns = {column["name"] for column in inspector.get_columns("job_source")}
+        dropped = 0
+        for column_name in (
+            "raw_transcription",
+            "ai_metadata",
+            "raw_api_response",
+            "error_detail",
+            "executed_at",
+        ):
+            if column_name not in present_columns:
+                continue
+            sync_connection.execute(text(f'alter table "job_source" drop column "{column_name}"'))
+            dropped += 1
+        return dropped
 
-    if "job" in table_names:
-        job_columns = {column["name"] for column in inspector.get_columns("job")}
-        if "retry_count" not in job_columns:
-            connection.execute(text("ALTER TABLE job ADD COLUMN retry_count INTEGER NOT NULL DEFAULT 0"))
-            logger.warning("Applied SQLite compatibility schema patch table=job column=retry_count default=0")
+    async with active_engine.begin() as connection:
+        dropped_columns = await connection.run_sync(_reconcile)
+    if dropped_columns:
+        logger.warning("Dropped %s legacy job_source column(s) during startup reconciliation", dropped_columns)
+    return dropped_columns
 
-    if "revision" in table_names:
-        revision_columns = {column["name"] for column in inspector.get_columns("revision")}
-        if "source_id" in revision_columns:
-            has_unique_source = False
-            for index in inspector.get_indexes("revision"):
-                if index.get("unique") and index.get("column_names") == ["source_id"]:
-                    has_unique_source = True
-                    break
-            if not has_unique_source:
-                connection.execute(
-                    text(
-                        "CREATE UNIQUE INDEX IF NOT EXISTS "
-                        "ux_revision_source_id ON revision(source_id)"
+
+async def reconcile_canonical_media_paths(*, engine: AsyncEngine | None = None) -> int:
+    """Normalize stored media paths to upload-root-relative POSIX form."""
+    active_engine = engine or resolve_engine()
+    if not hasattr(active_engine, "begin"):
+        return 0
+
+    def _reconcile(sync_connection) -> int:
+        rows_changed = 0
+        inspector = sqlalchemy_inspect(sync_connection)
+        table_names = set(inspector.get_table_names())
+        if "source" in table_names:
+            rows = sync_connection.execute(text('select id, file_path from "source" where file_path is not null')).mappings().all()
+            for row in rows:
+                original = str(row["file_path"])
+                normalized = _canonical_relative_path(original, preferred_prefix="documents/")
+                if normalized is None or normalized == original:
+                    continue
+                sync_connection.execute(
+                    text('update "source" set file_path = :file_path where id = :id'),
+                    {"id": row["id"], "file_path": normalized},
+                )
+                rows_changed += 1
+
+        if "photo" in table_names:
+            rows = sync_connection.execute(text('select id, path from "photo" where path is not null')).mappings().all()
+            for row in rows:
+                original = str(row["path"])
+                normalized = _canonical_relative_path(original, preferred_prefix="photos/")
+                if normalized is None or normalized == original:
+                    continue
+                sync_connection.execute(
+                    text('update "photo" set path = :path where id = :id'),
+                    {"id": row["id"], "path": normalized},
+                )
+                rows_changed += 1
+        return rows_changed
+
+    async with active_engine.begin() as connection:
+        rows_changed = await connection.run_sync(_reconcile)
+    if rows_changed:
+        logger.warning("Normalized %s media-path row(s) to canonical relative format", rows_changed)
+    return rows_changed
+
+
+def _canonical_relative_path(value: str, *, preferred_prefix: str) -> str | None:
+    normalized = value.strip().replace("\\", "/")
+    if not normalized:
+        return None
+
+    lowered = normalized.casefold()
+    if lowered.startswith("http://") or lowered.startswith("https://") or lowered.startswith("data:"):
+        return None
+
+    if lowered.startswith("/uploads/"):
+        normalized = normalized[len("/uploads/") :]
+        lowered = normalized.casefold()
+    elif lowered.startswith("uploads/"):
+        normalized = normalized[len("uploads/") :]
+        lowered = normalized.casefold()
+    elif lowered.startswith("data/"):
+        normalized = normalized[len("data/") :]
+        lowered = normalized.casefold()
+
+    for prefix in ("documents/", "photos/", "persons/", "portraits/"):
+        marker = f"/{prefix}"
+        index = lowered.find(marker)
+        if index >= 0:
+            normalized = normalized[index + 1 :]
+            lowered = normalized.casefold()
+            break
+        if lowered.startswith(prefix):
+            break
+
+    if preferred_prefix == "persons/" and lowered.startswith("portraits/"):
+        normalized = "persons/" + normalized[len("portraits/") :]
+        lowered = normalized.casefold()
+
+    if not lowered.startswith(preferred_prefix):
+        return None
+
+    # Collapse any accidental "." segments while preserving relative semantics.
+    collapsed = Path(normalized).as_posix()
+    if collapsed.startswith("../") or collapsed == "..":
+        return None
+    return collapsed
+
+
+async def seed_registry_defaults(*, engine: AsyncEngine | None = None) -> None:
+    """Seed default registry rows for role and document type taxonomies."""
+    active_engine = engine or resolve_engine()
+    session_factory = async_sessionmaker(active_engine, class_=AsyncSession, expire_on_commit=False)
+
+    async with session_factory() as session:
+        role_keys = set((await session.exec(select(PersonRole.semantic_key))).all())
+        for semantic_key, label in BUILT_IN_PERSON_ROLES:
+            if semantic_key not in role_keys:
+                session.add(
+                    PersonRole(
+                        semantic_key=semantic_key,
+                        label=label,
+                        normalized_label=label.casefold(),
                     )
                 )
-                logger.warning(
-                    "Applied SQLite compatibility schema patch "
-                    "table=revision unique_index=ux_revision_source_id"
+
+        type_keys = set((await session.exec(select(DocumentType.semantic_key))).all())
+        for semantic_key, label in BUILT_IN_DOCUMENT_TYPES:
+            if semantic_key not in type_keys:
+                session.add(
+                    DocumentType(
+                        semantic_key=semantic_key,
+                        label=label,
+                        normalized_label=label.casefold(),
+                    )
                 )
+
+        await session.commit()

src/transcription/db/registries.py

@@ -0,0 +1,20 @@
+"""Application-defined semantic registry entries."""
+
+from __future__ import annotations
+
+BUILT_IN_DOCUMENT_TYPES: tuple[tuple[str, str], ...] = (
+    ("book", "Book"),
+    ("letter", "Letter"),
+    ("postcard", "Postcard"),
+    ("photo", "Photo"),
+    ("journal", "Journal"),
+    ("form", "Form"),
+)
+
+BUILT_IN_PERSON_ROLES: tuple[tuple[str, str], ...] = (
+    ("author", "Author"),
+    ("recipient", "Recipient"),
+    ("mentioned", "Mentioned"),
+)
+
+AUTHOR_ROLE_SEMANTIC_KEY = "author"

src/transcription/db/runtime.py

@@ -1,18 +1,15 @@
 import logging
-from collections.abc import AsyncGenerator
-from contextlib import asynccontextmanager
-from contextvars import ContextVar
 from dataclasses import dataclass
-from functools import partial
 
 from sqlalchemy.ext.asyncio import AsyncEngine
 from sqlalchemy.ext.asyncio import async_sessionmaker
-from sqlalchemy.ext.asyncio import create_async_engine
 from sqlmodel.ext.asyncio.session import AsyncSession
-from sqlmodel.pool import StaticPool
 
 from ..config import Settings
 from ..config import get_settings
+from .engine import get_database_url
+from .engine import get_engine
+from .session import get_session_factory
 
 logger = logging.getLogger(__name__)
 
@@ -25,79 +22,42 @@ class DatabaseRuntime:
     session_factory: async_sessionmaker[AsyncSession]
 
 
-_runtime: ContextVar[DatabaseRuntime | None] = ContextVar("database_runtime", default=None)
+_runtime: DatabaseRuntime | None = None
+
+
+def get_database_runtime() -> DatabaseRuntime | None:
+    """Return the process-owned database runtime."""
+    return _runtime
 
 
 async def dispose_database_runtime() -> None:
     """Dispose lifespan-owned async database resources."""
-    runtime = _runtime.get()
+    global _runtime
+    runtime = _runtime
     if runtime is None:
         return
     await runtime.engine.dispose()
-    _runtime.set(None)
-
-
-def _to_async_database_url(database_url: str) -> str:
-    """Normalize configured database URL to an async SQLAlchemy driver URL."""
-    if database_url.startswith("sqlite://") and not database_url.startswith("sqlite+aiosqlite://"):
-        return database_url.replace("sqlite://", "sqlite+aiosqlite://", 1)
-    if database_url.startswith("postgresql://") and not database_url.startswith("postgresql+asyncpg://"):
-        return database_url.replace("postgresql://", "postgresql+asyncpg://", 1)
-    return database_url
-
-
-def _build_engine(settings: Settings) -> AsyncEngine:
-    database_url = _to_async_database_url(settings.database_url)
-    engine_factory = partial(
-        create_async_engine,
-        url=database_url,
-        echo=False,
-        pool_pre_ping=True,
-    )
-
-    if database_url.startswith("sqlite"):
-        sqlite_connect_settings = {"check_same_thread": settings.sqlite_check_same_thread}
-        engine_factory = partial(engine_factory, connect_args=sqlite_connect_settings)
-        if ":memory:" in database_url:
-            engine_factory = partial(engine_factory, poolclass=StaticPool)
-
-    return engine_factory()
+    _runtime = None
 
 
 def initialize_database_runtime(*, settings: Settings | None = None) -> DatabaseRuntime:
     """Initialize lifespan-owned async DB resources once per process."""
-    runtime = _runtime.get()
+    global _runtime
+    active_settings = settings or get_settings()
+    database_url = get_database_url(active_settings)
+    runtime = _runtime
     if runtime is not None:
+        runtime_url = runtime.engine.url.render_as_string(hide_password=False)
+        if runtime_url != database_url:
+            raise RuntimeError(
+                "Database runtime is already initialized for a different database: "
+                f"{runtime_url!r} != {database_url!r}"
+            )
         return runtime
 
-    active_settings = settings or get_settings()
-    engine = _build_engine(active_settings)
-    session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
+    engine = get_engine(database_url)
+    session_factory = get_session_factory(database_url)
     runtime = DatabaseRuntime(engine=engine, session_factory=session_factory)
-    _runtime.set(runtime)
+    _runtime = runtime
     logger.debug("Initialized async database runtime for database_url=%s", engine.url)
     return runtime
-
-
-def get_engine(settings: Settings | None = None) -> AsyncEngine:
-    """Return the current async SQLAlchemy engine."""
-    runtime = _runtime.get() or initialize_database_runtime(settings=settings)
-    return runtime.engine
-
-
-def get_session_factory(settings: Settings | None = None) -> async_sessionmaker[AsyncSession]:
-    """Return the shared async session factory."""
-    runtime = _runtime.get() or initialize_database_runtime(settings=settings)
-    return runtime.session_factory
-
-
-@asynccontextmanager
-async def get_session(
-    *,
-    settings: Settings | None = None,
-    session_factory: async_sessionmaker[AsyncSession] | None = None,
-) -> AsyncGenerator[AsyncSession]:
-    """Yield a database session and ensure cleanup."""
-    active_session_factory = session_factory or get_session_factory(settings)
-    async with active_session_factory() as session:
-        yield session

src/transcription/db/session.py

@@ -0,0 +1,104 @@
+from collections.abc import AsyncGenerator
+from contextlib import asynccontextmanager
+from typing import Annotated
+
+from fastapi import Depends
+from sqlalchemy.ext.asyncio import async_sessionmaker
+from sqlmodel.ext.asyncio.session import AsyncSession
+
+from ..config import Settings
+from ..config import get_settings
+from .engine import dispose_engine
+from .engine import get_database_url
+from .engine import get_engine
+
+type SessionFactory = async_sessionmaker[AsyncSession]
+
+
+_SESSION_FACTORIES: dict[str, SessionFactory] = {}
+
+
+def get_session_factory(database_url: str) -> SessionFactory:
+    """Return the process-wide session factory for ``database_url``."""
+    factory = _SESSION_FACTORIES.get(database_url)
+    if factory is None:
+        factory = async_sessionmaker(
+            bind=get_engine(database_url),
+            class_=AsyncSession,
+            expire_on_commit=False,
+        )
+        _SESSION_FACTORIES[database_url] = factory
+    return factory
+
+
+def resolve_session_factory(
+    database_url: str | None = None,
+    *,
+    settings: Settings | None = None,
+) -> SessionFactory:
+    if database_url is not None:
+        return get_session_factory(database_url)
+    if settings is None:
+        from .runtime import get_database_runtime
+
+        runtime = get_database_runtime()
+        if runtime is not None:
+            return runtime.session_factory
+    return get_session_factory(get_database_url(settings or get_settings()))
+
+
+type SessionFactoryDep = Annotated[SessionFactory, Depends(resolve_session_factory)]
+
+
+async def dispose_session_factory(database_url: str) -> None:
+    """Drop the session factory and engine for ``database_url`` only."""
+    _SESSION_FACTORIES.pop(database_url, None)
+    await dispose_engine(database_url)
+
+
+@asynccontextmanager
+async def session_scope(
+    *,
+    settings: Settings | None = None,
+    database_url: str | None = None,
+    session_factory: SessionFactory | None = None,
+    session: AsyncSession | None = None,
+) -> AsyncGenerator[AsyncSession]:
+    if session is not None:
+        yield session
+        return
+
+    active_session_factory = session_factory or resolve_session_factory(
+        database_url,
+        settings=settings,
+    )
+    async with active_session_factory() as owned_session:
+        yield owned_session
+
+
+type SessionScopeDep = Annotated[AsyncSession, Depends(session_scope)]
+
+
+@asynccontextmanager
+async def transaction_scope(
+    *,
+    settings: Settings | None = None,
+    database_url: str | None = None,
+    session_factory: SessionFactory | None = None,
+    session: AsyncSession | None = None,
+) -> AsyncGenerator[AsyncSession]:
+    if session is not None:
+        if not session.in_transaction():
+            raise RuntimeError("A supplied session must have an active transaction")
+        yield session
+        return
+
+    active_session_factory = session_factory or resolve_session_factory(
+        database_url,
+        settings=settings,
+    )
+    async with active_session_factory.begin() as owned_session:
+        yield owned_session
+
+
+type TransactionScopeDep = Annotated[AsyncSession, Depends(transaction_scope)]

src/transcription/errors.py

@@ -17,6 +17,7 @@ class ErrorCategory(StrEnum):
     NOT_FOUND = "not_found_error"
     CONFLICT = "conflict_error"
     EXTERNAL_PROVIDER = "external_provider_error"
+    EXTERNAL_TIMEOUT = "external_timeout_error"
     PROCESSING = "processing_error"
     INFRA_TRANSIENT = "infrastructure_transient_error"
     INFRA_PERSISTENT = "infrastructure_persistent_error"
@@ -59,11 +60,28 @@ class ErrorEnvelope:
     timestamp: str
 
 
+def canonical_error_category(error: AppError) -> str:
+    """Map internal categories to canonical API/UI envelope categories."""
+    mapping: dict[ErrorCategory, str] = {
+        ErrorCategory.VALIDATION: "validation",
+        ErrorCategory.USER_INPUT: "validation",
+        ErrorCategory.NOT_FOUND: "not_found",
+        ErrorCategory.CONFLICT: "conflict",
+        ErrorCategory.EXTERNAL_PROVIDER: "external",
+        ErrorCategory.EXTERNAL_TIMEOUT: "timeout",
+        ErrorCategory.INFRA_TRANSIENT: "timeout",
+        ErrorCategory.PROCESSING: "internal",
+        ErrorCategory.INFRA_PERSISTENT: "internal",
+        ErrorCategory.INTERNAL_UNEXPECTED: "internal",
+    }
+    return mapping.get(error.category, "internal")
+
+
 def build_error_envelope(error: AppError) -> ErrorEnvelope:
     """Build an API-safe response envelope from an AppError."""
     return ErrorEnvelope(
         error_id=error.error_id,
-        category=error.category.value,
+        category=canonical_error_category(error),
         message=error.message,
         suggestion=error.suggestion,
         timestamp=datetime.now(UTC).isoformat(),

src/transcription/models.py

@@ -1,110 +0,0 @@
-"""SQLModel domain models for the transcription system.
-
-Core V1 lifecycle:
-    Document  ->  one-to-many  ->  Source
-    Document  ->  one-to-many  ->  Job
-    Source    ->  one-to-one?  ->  Revision (optional)
-"""
-
-from datetime import UTC
-from datetime import datetime
-from enum import StrEnum
-from typing import Optional
-from uuid import UUID
-from uuid import uuid4
-
-from sqlalchemy import UniqueConstraint
-from sqlmodel import Field
-from sqlmodel import Relationship
-from sqlmodel import SQLModel
-
-
-class JobStatus(StrEnum):
-    QUEUED = "queued"
-    PROCESSING = "processing"
-    TRANSCRIBED = "transcribed"
-    FAILED = "failed"
-
-
-class Document(SQLModel, table=True):
-    """An historical document."""
-
-    id: UUID = Field(default_factory=uuid4, primary_key=True)
-    name: str
-
-    # Relationships
-    jobs: list["Job"] = Relationship(back_populates="document")
-    sources: list["Source"] = Relationship(back_populates="document")
-
-
-class Source(SQLModel, table=True):
-    """A document source (image or PDF)."""
-
-    id: UUID = Field(default_factory=uuid4, primary_key=True)
-    document_id: UUID = Field(foreign_key="document.id")
-    job_id: UUID = Field(foreign_key="job.id")
-    upload_name: str
-    """The filename of the source that was uploaded for transcription."""
-    filename: str
-    """The system generated unique source name."""
-    file_path: str
-    """The location where the sources are stored on the local filesystem."""
-    date_uploaded: datetime = Field(default_factory=lambda: datetime.now(UTC))
-
-    # Relationships
-    document: Optional["Document"] = Relationship(back_populates="sources")
-    job: Optional["Job"] = Relationship(back_populates="sources")
-    revision: Optional["Revision"] = Relationship(
-        back_populates="source",
-        sa_relationship_kwargs={"uselist": False},
-    )
-
-
-class Job(SQLModel, table=True):
-    """A transcription job tied to a single document."""
-
-    id: UUID = Field(default_factory=uuid4, primary_key=True)
-    document_id: UUID = Field(foreign_key="document.id")
-    status: JobStatus = Field(default=JobStatus.QUEUED)
-    retry_count: int = Field(default=0, ge=0)
-    date_created: datetime = Field(default_factory=lambda: datetime.now(UTC))
-    date_updated: datetime = Field(default_factory=lambda: datetime.now(UTC))
-    provider: str | None = None
-    """Name of the transcription provider used to generate this transcript."""
-    model: str | None = None
-    """Model identifier used to generate this transcript."""
-    prompt_name: str | None = None
-    """Name of the prompt used to generate this transcript."""
-    text: str | None = None
-    """The transcribed text. This may be None if the job failed or is still in progress."""
-    error_detail: str | None = None
-    """Details of any error that occurred during transcription."""
-
-    # Relationships
-    document: Optional["Document"] = Relationship(back_populates="jobs")
-    sources: list["Source"] = Relationship(back_populates="job")
-
-    @property
-    def filename(self) -> str:
-        """Return the filename of the associated source, when available."""
-        if not self.sources:
-            return "unknown"
-        return self.sources[0].filename
-
-
-class Revision(SQLModel, table=True):
-    """A revision of a transcription text."""
-
-    id: UUID = Field(default_factory=uuid4, primary_key=True)
-    source_id: UUID = Field(foreign_key="source.id")
-    """ID for the associated source."""
-    revision: int = Field(default=1, ge=1)
-    """Revision number of this transcription revision, starting at 1."""
-    text: str
-    """The revised text."""
-    date_created: datetime = Field(default_factory=lambda: datetime.now(UTC))
-
-    __table_args__ = (UniqueConstraint("source_id", name="uq_revision_source_id"),)
-
-    # Relationships
-    source: Optional["Source"] = Relationship(back_populates="revision")

src/transcription/providers/__init__.py

@@ -6,8 +6,12 @@ from transcription.config import get_settings
 from transcription.providers.base import ProviderAuthError
 from transcription.providers.base import ProviderError
 from transcription.providers.base import ProviderResponseError
+from transcription.providers.base import TranscriptionMetadata
 from transcription.providers.base import TranscriptionProvider
 from transcription.providers.base import TranscriptionResult
+from transcription.providers.evidence import RequestManifest
+from transcription.providers.evidence import SourceEvidenceReference
+from transcription.providers.evidence import TransportEvidence
 from transcription.providers.openrouter import OpenRouterTranscriptionProvider
 
 
@@ -25,7 +29,11 @@ __all__ = [
     "ProviderAuthError",
     "ProviderError",
     "ProviderResponseError",
+    "RequestManifest",
+    "SourceEvidenceReference",
+    "TranscriptionMetadata",
     "TranscriptionProvider",
     "TranscriptionResult",
+    "TransportEvidence",
     "get_transcription_provider",
 ]

src/transcription/providers/base.py

@@ -1,12 +1,33 @@
-"""Provider interfaces and shared types for transcription adapters."""
+"""Provider interfaces and validated shared contracts for transcription adapters."""
 
-from dataclasses import dataclass
 from typing import Protocol
 
+from pydantic import BaseModel
+from pydantic import ConfigDict
+from pydantic import Field
+from pydantic import JsonValue
+
+from transcription.providers.evidence import RequestManifest
+from transcription.providers.evidence import SourceEvidenceReference
+from transcription.providers.evidence import TransportEvidence
+
 
 class ProviderError(RuntimeError):
     """Base error for provider failures."""
 
+    def __init__(
+        self,
+        message: str,
+        *,
+        request_manifest: RequestManifest | None = None,
+        transport_evidence: TransportEvidence | None = None,
+        failure_phase: str = "provider_request",
+    ) -> None:
+        super().__init__(message)
+        self.request_manifest = request_manifest
+        self.transport_evidence = transport_evidence
+        self.failure_phase = failure_phase
+
 
 class ProviderAuthError(ProviderError):
     """Raised when provider authentication fails."""
@@ -16,23 +37,100 @@ class ProviderResponseError(ProviderError):
     """Raised when provider responses are malformed or unusable."""
 
 
-@dataclass(frozen=True)
-class TranscriptionResult:
+class ProviderUsage(BaseModel):
+    """Normalized provider token accounting."""
+
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+    input_tokens: int | None = Field(default=None, ge=0)
+    output_tokens: int | None = Field(default=None, ge=0)
+    total_tokens: int | None = Field(default=None, ge=0)
+
+
+class TranscriptionMetadata(BaseModel):
+    """Stable structured metadata persisted for one provider execution."""
+
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+    finish_reason: str | None = Field(default=None, min_length=1)
+    usage: ProviderUsage | None = None
+
+    def as_json_object(self) -> dict[str, JsonValue] | None:
+        payload = self.model_dump(mode="json", exclude_none=True)
+        return payload or None
+
+
+class TranscriptionResult(BaseModel):
     """Normalized output returned by any transcription provider."""
 
-    text: str
-    provider: str
-    prompt_name: str
-    model: str
-    finish_reason: str | None = None
-    usage_input_tokens: int | None = None
-    usage_output_tokens: int | None = None
-    usage_total_tokens: int | None = None
+    model_config = ConfigDict(extra="forbid", frozen=True, str_strip_whitespace=True)
+
+    text: str = Field(min_length=1)
+    provider: str = Field(min_length=1)
+    model: str = Field(min_length=1)
+    prompt_name: str | None = None
+    prompt_hash: str | None = Field(default=None, pattern=r"^[0-9a-f]{64}$")
+    system_prompt: str | None = None
+    user_prompt: str | None = None
+    temperature: float | None = Field(default=None, ge=0.0, le=2.0)
+    top_p: float | None = Field(default=None, ge=0.0, le=1.0)
+    metadata: TranscriptionMetadata = Field(default_factory=TranscriptionMetadata)
+    raw_api_response: dict[str, JsonValue] | None = None
+    request_manifest: RequestManifest | None = None
+    transport_evidence: TransportEvidence | None = None
+
+    @property
+    def finish_reason(self) -> str | None:
+        return self.metadata.finish_reason
+
+    @property
+    def usage_input_tokens(self) -> int | None:
+        return self.metadata.usage.input_tokens if self.metadata.usage else None
+
+    @property
+    def usage_output_tokens(self) -> int | None:
+        return self.metadata.usage.output_tokens if self.metadata.usage else None
+
+    @property
+    def usage_total_tokens(self) -> int | None:
+        return self.metadata.usage.total_tokens if self.metadata.usage else None
+
+    def metadata_payload(self) -> dict[str, JsonValue] | None:
+        return self.metadata.as_json_object()
 
 
 class TranscriptionProvider(Protocol):
     """Contract every transcription provider adapter must satisfy."""
 
-    async def transcribe(self, *, prompt_text: str, image_bytes: bytes, mime_type: str) -> TranscriptionResult:
+    @property
+    def model(self) -> str:
+        """Return the resolved model slug this adapter will call."""
+        ...
+
+    @property
+    def current_request_manifest(self) -> RequestManifest | None:
+        """Return the manifest for the most recent call, for failure evidence."""
+        ...
+
+    @property
+    def current_transport_evidence(self) -> TransportEvidence | None:
+        """Return transport-level evidence for the most recent call."""
+        ...
+
+    async def transcribe(
+        self,
+        *,
+        prompt_text: str,
+        image_bytes: bytes,
+        mime_type: str,
+        temperature: float | None = None,
+        top_p: float | None = None,
+        source_reference: SourceEvidenceReference | None = None,
+        requested_model: str | None = None,
+    ) -> TranscriptionResult:
         """Transcribe the provided image according to the prompt text."""
         ...
+
+    async def aclose(self) -> None:
+        """Release any pooled network resources held by the adapter."""
+        ...

src/transcription/providers/evidence.py

@@ -0,0 +1,161 @@
+"""Versioned, provider-neutral contracts for processing evidence."""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import platform
+from importlib.metadata import PackageNotFoundError
+from importlib.metadata import version
+from typing import Any
+from typing import Literal
+from uuid import UUID
+
+from pydantic import BaseModel
+from pydantic import ConfigDict
+from pydantic import Field
+from pydantic import JsonValue
+
+from transcription.config import Settings
+
+REQUEST_MANIFEST_SCHEMA = "transcription.request-manifest"
+REQUEST_MANIFEST_VERSION = "1"
+SOFTWARE_CONTEXT_SCHEMA = "transcription.software-context"
+SOFTWARE_CONTEXT_VERSION = "1"
+TRANSPORT_EVIDENCE_SCHEMA = "transcription.transport-evidence"
+TRANSPORT_EVIDENCE_VERSION = "1"
+CANONICAL_JSON_ALGORITHM = "transcription-canonical-json-v1"
+
+SAFE_RESPONSE_HEADERS = frozenset(
+    {
+        "content-type",
+        "content-encoding",
+        "date",
+        "retry-after",
+        "x-request-id",
+        "x-openrouter-generation-id",
+        "x-ratelimit-limit",
+        "x-ratelimit-remaining",
+        "x-ratelimit-reset",
+    }
+)
+
+
+class EvidenceModel(BaseModel):
+    """Strict immutable base for persisted evidence contracts."""
+
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+
+class SourceEvidenceReference(EvidenceModel):
+    """Secret-safe identity for source content used by one execution."""
+
+    source_id: UUID
+    digest_sha256: str = Field(pattern=r"^[0-9a-f]{64}$")
+    byte_size: int = Field(ge=0)
+    media_type: str = Field(min_length=1)
+    page_number: int = Field(ge=1)
+    width: int | None = Field(default=None, ge=1)
+    height: int | None = Field(default=None, ge=1)
+    derivative_id: UUID | None = None
+    transformation: str | None = None
+
+
+class SoftwareContext(EvidenceModel):
+    """Versions needed to interpret a provider execution."""
+
+    schema_name: Literal["transcription.software-context"] = SOFTWARE_CONTEXT_SCHEMA
+    schema_version: Literal["1"] = SOFTWARE_CONTEXT_VERSION
+    application_version: str
+    application_commit: str | None = None
+    adapter_name: str
+    adapter_version: str
+    client_library: str
+    client_library_version: str
+    python_version: str
+
+
+class RequestManifest(EvidenceModel):
+    """Frozen, secret-safe representation of one concrete provider request."""
+
+    schema_name: Literal["transcription.request-manifest"] = REQUEST_MANIFEST_SCHEMA
+    schema_version: Literal["1"] = REQUEST_MANIFEST_VERSION
+    provider: str = Field(min_length=1)
+    requested_model: str = Field(min_length=1)
+    request: dict[str, JsonValue]
+    source: SourceEvidenceReference
+    explicitly_supplied_parameters: tuple[str, ...] = ()
+    omitted_optional_parameters: tuple[str, ...] = ()
+    optional_parameter_states: dict[str, Literal["omitted", "null", "value"]]
+    prompt_content: str = Field(min_length=1)
+    prompt_sha256: str = Field(pattern=r"^[0-9a-f]{64}$")
+    timeout_seconds: float = Field(gt=0)
+    retry_policy: str = Field(min_length=1)
+    software: SoftwareContext
+    canonicalization: Literal["transcription-canonical-json-v1"] = CANONICAL_JSON_ALGORITHM
+
+    def canonical_bytes(self) -> bytes:
+        return canonical_json_bytes(self.model_dump(mode="json"))
+
+    def digest(self) -> str:
+        return hashlib.sha256(self.canonical_bytes()).hexdigest()
+
+
+class TransportEvidence(EvidenceModel):
+    """Exact response captured at the application/router HTTP boundary."""
+
+    schema_name: Literal["transcription.transport-evidence"] = TRANSPORT_EVIDENCE_SCHEMA
+    schema_version: Literal["1"] = TRANSPORT_EVIDENCE_VERSION
+    response_received: bool
+    status_code: int | None = Field(default=None, ge=100, le=599)
+    body: bytes | None = None
+    safe_headers: dict[str, str] = Field(default_factory=dict)
+    content_type: str | None = None
+    content_encoding: str | None = None
+    request_id: str | None = None
+    generation_id: str | None = None
+
+
+def canonical_json_bytes(value: Any) -> bytes:
+    """Serialize JSON deterministically for evidence integrity hashes."""
+    return json.dumps(
+        value,
+        ensure_ascii=False,
+        allow_nan=False,
+        sort_keys=True,
+        separators=(",", ":"),
+    ).encode("utf-8")
+
+
+def filter_safe_response_headers(headers: Any) -> dict[str, str]:
+    """Return only explicitly allowlisted response headers."""
+    return {
+        str(name).lower(): str(value) for name, value in headers.items() if str(name).lower() in SAFE_RESPONSE_HEADERS
+    }
+
+
+def package_version(package: str) -> str:
+    """Return an installed package version without failing evidence capture."""
+    try:
+        return version(package)
+    except PackageNotFoundError:
+        return "unknown"
+
+
+def build_software_context(
+    *,
+    adapter_name: str,
+    adapter_version: str,
+    client_library: str,
+    settings: Settings,
+) -> SoftwareContext:
+    """Build the runtime software identity for an execution."""
+    return SoftwareContext(
+        application_version=package_version("transcription"),
+        application_commit=settings.transcription_commit,
+        adapter_name=adapter_name,
+        adapter_version=adapter_version,
+        client_library=client_library,
+        client_library_version=package_version(client_library),
+        python_version=platform.python_version(),
+    )

src/transcription/providers/openrouter.py

@@ -3,172 +3,530 @@
 from __future__ import annotations
 
 import base64
+import hashlib
+import json
 import logging
-from dataclasses import dataclass
+from collections.abc import AsyncIterator
+from collections.abc import Callable
+from typing import Annotated
 from typing import Any
-from typing import cast
+from typing import Literal
 
+import httpx
 from openrouter import OpenRouter
-from openrouter.components.chatmessages import ChatMessagesTypedDict
+from openrouter import errors as openrouter_errors
+from pydantic import BaseModel
+from pydantic import ConfigDict
+from pydantic import Field
+from pydantic import JsonValue
+from pydantic import TypeAdapter
+from pydantic import ValidationError
 
 from transcription.config import Settings
 from transcription.config import get_settings
 from transcription.providers.base import ProviderAuthError
 from transcription.providers.base import ProviderError
 from transcription.providers.base import ProviderResponseError
+from transcription.providers.base import ProviderUsage
+from transcription.providers.base import TranscriptionMetadata
 from transcription.providers.base import TranscriptionResult
+from transcription.providers.evidence import RequestManifest
+from transcription.providers.evidence import SourceEvidenceReference
+from transcription.providers.evidence import TransportEvidence
+from transcription.providers.evidence import build_software_context
+from transcription.providers.evidence import filter_safe_response_headers
 
 logger = logging.getLogger(__name__)
 
 DEFAULT_OPENROUTER_MODEL = "google/gemini-2.5-flash"
+OPENROUTER_ADAPTER_VERSION = "2"
 
 
-@dataclass(frozen=True)
-class OpenRouterRequest:
+class _CapturingAsyncByteStream(httpx.AsyncByteStream):
+    """Copy streamed response bytes without changing what the SDK consumes."""
+
+    def __init__(self, stream: httpx.AsyncByteStream, on_complete: Callable[[bytes], None]):
+        self._stream = stream
+        self._on_complete = on_complete
+
+    async def __aiter__(self) -> AsyncIterator[bytes]:
+        content = bytearray()
+        async for chunk in self._stream:
+            content.extend(chunk)
+            yield chunk
+        self._on_complete(bytes(content))
+
+    async def aclose(self) -> None:
+        await self._stream.aclose()
+
+
+class _CapturingAsyncClient:
+    """Delegate SDK HTTP calls while retaining the response before SDK parsing."""
+
+    def __init__(self, client: httpx.AsyncClient):
+        self._client = client
+        self.last_response: httpx.Response | None = None
+        self.last_body: bytes | None = None
+
+    async def send(self, request: httpx.Request, **kwargs: Any) -> httpx.Response:
+        response = await self._client.send(request, **kwargs)
+        self.last_response = response
+        try:
+            self.last_body = response.content
+        except httpx.ResponseNotRead:
+            stream = response.stream
+            if not isinstance(stream, httpx.AsyncByteStream):
+                raise
+            response.stream = _CapturingAsyncByteStream(stream, self._capture_body)
+        return response
+
+    def build_request(self, *args: Any, **kwargs: Any) -> httpx.Request:
+        return self._client.build_request(*args, **kwargs)
+
+    async def aclose(self) -> None:
+        await self._client.aclose()
+
+    def reset(self) -> None:
+        self.last_response = None
+        self.last_body = None
+
+    def _capture_body(self, body: bytes) -> None:
+        self.last_body = body
+
+
+class _ProviderModel(BaseModel):
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+
+class TextContent(_ProviderModel):
+    type: Literal["text"] = "text"
+    text: str = Field(min_length=1)
+
+
+class ImageUrl(_ProviderModel):
+    url: str = Field(min_length=1)
+
+
+class ImageContent(_ProviderModel):
+    type: Literal["image_url"] = "image_url"
+    image_url: ImageUrl
+
+
+class FileData(_ProviderModel):
+    filename: str = Field(min_length=1)
+    file_data: str = Field(min_length=1)
+
+
+class FileContent(_ProviderModel):
+    type: Literal["file"] = "file"
+    file: FileData
+
+
+MessageContent = Annotated[TextContent | ImageContent | FileContent, Field(discriminator="type")]
+
+
+class UserMessage(_ProviderModel):
+    role: Literal["user"] = "user"
+    content: tuple[MessageContent, ...] = Field(min_length=2)
+
+
+class OpenRouterRequest(_ProviderModel):
     """Normalized request payload fields for OpenRouter calls."""
 
-    model: str
-    messages: list[dict[str, Any]]
+    model: str = Field(min_length=1)
+    messages: tuple[UserMessage, ...] = Field(min_length=1)
     http_referer: str | None
     x_open_router_title: str | None
+    temperature: float | None = Field(ge=0.0, le=2.0)
+    top_p: float | None = Field(ge=0.0, le=1.0)
+
+
+class ResponseContentPart(BaseModel):
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+    text: str | None = None
+
+
+class ResponseMessage(BaseModel):
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+    content: str | tuple[ResponseContentPart, ...] | None = None
+
+
+class ResponseChoice(BaseModel):
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+    message: ResponseMessage
+    finish_reason: str | None = None
+
+
+class ResponseUsage(BaseModel):
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+    prompt_tokens: int | None = Field(default=None, ge=0)
+    completion_tokens: int | None = Field(default=None, ge=0)
+    total_tokens: int | None = Field(default=None, ge=0)
+    input_tokens: int | None = Field(default=None, ge=0)
+    output_tokens: int | None = Field(default=None, ge=0)
+    total: int | None = Field(default=None, ge=0)
+
+
+class OpenRouterResponse(BaseModel):
+    model_config = ConfigDict(extra="allow", frozen=True)
+
+    model: str | None = None
+    choices: tuple[ResponseChoice, ...] = Field(min_length=1)
+    usage: dict[str, JsonValue] | None = None
+
+
+JSON_OBJECT_ADAPTER = TypeAdapter(dict[str, JsonValue])
 
 
 class OpenRouterTranscriptionProvider:
     """Adapter that performs image transcription through OpenRouter."""
 
-    def __init__(self, *, settings: Settings | None = None, client: OpenRouter | None = None):
+    def __init__(
+        self,
+        *,
+        settings: Settings | None = None,
+        client: OpenRouter | None = None,
+        async_client: httpx.AsyncClient | None = None,
+    ):
         self._settings = settings or get_settings()
         self._model = self._settings.provider_model or DEFAULT_OPENROUTER_MODEL
-        self._client = client or OpenRouter(api_key=self._settings.openrouter_api_key)
+        self._capturing_client: _CapturingAsyncClient | None = None
+        self._current_request_manifest: RequestManifest | None = None
+        self._current_transport_evidence: TransportEvidence | None = None
+        if client is None:
+            # httpx defaults every phase to 5s, which silently caps provider calls far
+            # below worker_provider_timeout_seconds. Track the configured budget instead.
+            timeout = httpx.Timeout(
+                self._settings.worker_provider_timeout_seconds,
+                connect=10.0,
+            )
+            self._capturing_client = _CapturingAsyncClient(
+                async_client or httpx.AsyncClient(follow_redirects=True, timeout=timeout)
+            )
+            client = OpenRouter(
+                api_key=self._settings.openrouter_api_key.get_secret_value(),
+                async_client=self._capturing_client,
+            )
+        self._client = client
 
     @property
     def model(self) -> str:
         """Return the resolved OpenRouter model slug."""
         return self._model
 
-    async def transcribe(self, *, prompt_text: str, image_bytes: bytes, mime_type: str) -> TranscriptionResult:
+    @property
+    def current_request_manifest(self) -> RequestManifest | None:
+        return self._current_request_manifest
+
+    @property
+    def current_transport_evidence(self) -> TransportEvidence | None:
+        if self._current_transport_evidence is not None:
+            return self._current_transport_evidence
+        if self._current_request_manifest is None:
+            return None
+        return self._captured_transport_evidence()
+
+    async def aclose(self) -> None:
+        if self._capturing_client is not None:
+            await self._capturing_client.aclose()
+
+    async def transcribe(
+        self,
+        *,
+        prompt_text: str,
+        image_bytes: bytes,
+        mime_type: str,
+        temperature: float | None = None,
+        top_p: float | None = None,
+        source_reference: SourceEvidenceReference | None = None,
+        requested_model: str | None = None,
+    ) -> TranscriptionResult:
         """Send prompt + image to OpenRouter and return normalized text output."""
-        request = self._build_request(prompt_text=prompt_text, image_bytes=image_bytes, mime_type=mime_type)
+        request = self._build_request(
+            prompt_text=prompt_text,
+            image_bytes=image_bytes,
+            mime_type=mime_type,
+            temperature=temperature,
+            top_p=top_p,
+            requested_model=requested_model,
+        )
+        manifest = self._build_request_manifest(
+            request=request,
+            prompt_text=prompt_text,
+            source_reference=source_reference,
+            temperature=temperature,
+            top_p=top_p,
+        )
+        self._current_request_manifest = manifest
+        self._current_transport_evidence = None
+        if self._capturing_client is not None:
+            self._capturing_client.reset()
         try:
             response = await self._client.chat.send_async(
-                messages=cast(list[ChatMessagesTypedDict], request.messages),
-                model=request.model,
-                http_referer=request.http_referer,
-                x_open_router_title=request.x_open_router_title,
+                **request.model_dump(mode="json", exclude_none=True),
+                retries=None,
             )
         except Exception as exc:
-            message = str(exc).lower()
-            if "401" in message or "auth" in message or "api key" in message:
-                raise ProviderAuthError("OpenRouter authentication failed") from exc
-            raise ProviderError("OpenRouter request failed") from exc
+            transport = self._captured_transport_evidence()
+            self._current_transport_evidence = transport
+            if isinstance(exc, openrouter_errors.UnauthorizedResponseError):
+                raise ProviderAuthError(
+                    "OpenRouter authentication failed",
+                    request_manifest=manifest,
+                    transport_evidence=transport,
+                    failure_phase="http_response" if transport.response_received else "connection",
+                ) from exc
+            failure_phase = (
+                "response_validation"
+                if isinstance(exc, openrouter_errors.ResponseValidationError)
+                else "http_response"
+                if transport.response_received
+                else "connection"
+            )
+            raise ProviderError(
+                self._transport_error_message(transport),
+                request_manifest=manifest,
+                transport_evidence=transport,
+                failure_phase=failure_phase,
+            ) from exc
 
-        text = self._extract_text(response)
-        model = self._get_optional_attr(response, "model") or self.model
-        finish_reason = self._extract_finish_reason(response)
-        usage_input_tokens, usage_output_tokens, usage_total_tokens = self._extract_usage(response)
+        transport = self._captured_transport_evidence()
+        self._current_transport_evidence = transport
+        raw_api_response = self._coerce_raw_response(response)
+        try:
+            validated_response = OpenRouterResponse.model_validate(raw_api_response)
+        except ValidationError as exc:
+            raise ProviderResponseError(
+                "OpenRouter response failed schema validation",
+                request_manifest=manifest,
+                transport_evidence=transport,
+                failure_phase="response_validation",
+            ) from exc
+
+        try:
+            text = self._extract_text(validated_response)
+        except ProviderResponseError as exc:
+            raise ProviderResponseError(
+                str(exc),
+                request_manifest=manifest,
+                transport_evidence=transport,
+                failure_phase="response_validation",
+            ) from exc
+        model = validated_response.model or requested_model or self.model
+        metadata = self._build_metadata(validated_response)
         logger.info("OpenRouter transcription completed using model=%s", model)
         return TranscriptionResult(
             text=text,
             provider="openrouter",
-            prompt_name="",
+            prompt_name=None,
+            prompt_hash=None,
+            system_prompt=None,
+            user_prompt=prompt_text,
+            temperature=temperature,
+            top_p=top_p,
             model=model,
-            finish_reason=finish_reason,
-            usage_input_tokens=usage_input_tokens,
-            usage_output_tokens=usage_output_tokens,
-            usage_total_tokens=usage_total_tokens,
+            metadata=metadata,
+            raw_api_response=raw_api_response,
+            request_manifest=manifest,
+            transport_evidence=transport,
         )
 
-    def _build_request(self, *, prompt_text: str, image_bytes: bytes, mime_type: str) -> OpenRouterRequest:
+    def _build_request_manifest(
+        self,
+        *,
+        request: OpenRouterRequest,
+        prompt_text: str,
+        source_reference: SourceEvidenceReference | None,
+        temperature: float | None,
+        top_p: float | None,
+    ) -> RequestManifest | None:
+        if source_reference is None:
+            return None
+        request_payload = request.model_dump(mode="json", exclude_none=True)
+        sanitized_request = self._replace_embedded_media(request_payload, source_reference=source_reference)
+        explicit = tuple(name for name, value in (("temperature", temperature), ("top_p", top_p)) if value is not None)
+        omitted = tuple(name for name in ("temperature", "top_p") if name not in explicit)
+        return RequestManifest(
+            provider="openrouter",
+            requested_model=request.model,
+            request=JSON_OBJECT_ADAPTER.validate_python(sanitized_request),
+            source=source_reference,
+            explicitly_supplied_parameters=explicit,
+            omitted_optional_parameters=omitted,
+            optional_parameter_states={
+                "temperature": "value" if temperature is not None else "omitted",
+                "top_p": "value" if top_p is not None else "omitted",
+            },
+            prompt_content=prompt_text,
+            prompt_sha256=hashlib.sha256(prompt_text.encode("utf-8")).hexdigest(),
+            timeout_seconds=self._settings.worker_provider_timeout_seconds,
+            retry_policy="application-bounded; sdk-retries=0",
+            software=build_software_context(
+                adapter_name="openrouter",
+                adapter_version=OPENROUTER_ADAPTER_VERSION,
+                client_library="openrouter",
+                settings=self._settings,
+            ),
+        )
+
+    def _replace_embedded_media(
+        self,
+        value: Any,
+        *,
+        source_reference: SourceEvidenceReference,
+    ) -> Any:
+        if isinstance(value, str) and value.startswith("data:") and ";base64," in value:
+            return {
+                "source_reference": source_reference.model_dump(mode="json"),
+                "embedded_media_omitted": True,
+            }
+        if isinstance(value, dict):
+            return {
+                str(key): self._replace_embedded_media(item, source_reference=source_reference)
+                for key, item in value.items()
+            }
+        if isinstance(value, list | tuple):
+            return [self._replace_embedded_media(item, source_reference=source_reference) for item in value]
+        return value
+
+    def _captured_transport_evidence(self) -> TransportEvidence:
+        response = self._capturing_client.last_response if self._capturing_client is not None else None
+        if response is None:
+            return TransportEvidence(response_received=False)
+        headers = filter_safe_response_headers(response.headers)
+        body = self._capturing_client.last_body if self._capturing_client is not None else None
+        return TransportEvidence(
+            response_received=True,
+            status_code=response.status_code,
+            body=body,
+            safe_headers=headers,
+            content_type=headers.get("content-type"),
+            content_encoding=headers.get("content-encoding"),
+            request_id=headers.get("x-request-id"),
+            generation_id=headers.get("x-openrouter-generation-id"),
+        )
+
+    @staticmethod
+    def _transport_error_message(transport: TransportEvidence) -> str:
+        message = "OpenRouter request failed"
+        if transport.status_code is not None:
+            message += f" with HTTP {transport.status_code}"
+        if transport.body is None:
+            return message
+        try:
+            payload = json.loads(transport.body)
+        except (UnicodeDecodeError, json.JSONDecodeError):
+            return message
+        if not isinstance(payload, dict):
+            return message
+        error = payload.get("error")
+        detail = error.get("message") if isinstance(error, dict) else None
+        if isinstance(detail, str) and detail.strip():
+            return f"{message}: {detail.strip()[:500]}"
+        return message
+
+    def _build_metadata(self, response: OpenRouterResponse) -> TranscriptionMetadata:
+        choice = response.choices[0]
+        finish_reason = choice.finish_reason.strip() if choice.finish_reason and choice.finish_reason.strip() else None
+        normalized_usage = None
+        if response.usage is not None:
+            try:
+                usage = ResponseUsage.model_validate(response.usage)
+            except ValidationError as exc:
+                logger.warning("Ignoring invalid OpenRouter usage metadata: %s", exc)
+            else:
+                normalized_usage = ProviderUsage(
+                    input_tokens=usage.prompt_tokens if usage.prompt_tokens is not None else usage.input_tokens,
+                    output_tokens=usage.completion_tokens
+                    if usage.completion_tokens is not None
+                    else usage.output_tokens,
+                    total_tokens=usage.total_tokens if usage.total_tokens is not None else usage.total,
+                )
+                if normalized_usage.model_dump(exclude_none=True) == {}:
+                    normalized_usage = None
+        return TranscriptionMetadata(finish_reason=finish_reason, usage=normalized_usage)
+
+    def _coerce_raw_response(self, response: Any) -> dict[str, JsonValue]:
+        payload = self._to_json_compatible(response)
+        try:
+            return JSON_OBJECT_ADAPTER.validate_python(payload)
+        except ValidationError as exc:
+            raise ProviderResponseError("OpenRouter response is not a JSON object") from exc
+
+    def _to_json_compatible(self, value: Any) -> Any:
+        if value is None or isinstance(value, str | int | float | bool):
+            return value
+
+        if isinstance(value, dict):
+            return {str(key): self._to_json_compatible(item) for key, item in value.items()}
+
+        if isinstance(value, list | tuple | set):
+            return [self._to_json_compatible(item) for item in value]
+
+        for method_name in ("model_dump", "to_dict"):
+            serializer = getattr(value, method_name, None)
+            if callable(serializer):
+                try:
+                    serialized = serializer(mode="json") if method_name == "model_dump" else serializer()
+                    return self._to_json_compatible(serialized)
+                except (TypeError, ValueError) as exc:
+                    logger.debug("OpenRouter response serializer %s failed: %s", method_name, exc)
+                    continue
+
+        object_dict = getattr(value, "__dict__", None)
+        if isinstance(object_dict, dict):
+            return {
+                str(key): self._to_json_compatible(item)
+                for key, item in object_dict.items()
+                if not str(key).startswith("_")
+            }
+
+        raise ProviderResponseError(f"OpenRouter response contains unsupported value type: {type(value).__name__}")
+
+    def _build_request(
+        self,
+        *,
+        prompt_text: str,
+        image_bytes: bytes,
+        mime_type: str,
+        temperature: float | None,
+        top_p: float | None,
+        requested_model: str | None = None,
+    ) -> OpenRouterRequest:
         image_b64 = base64.b64encode(image_bytes).decode("ascii")
         data_url = f"data:{mime_type};base64,{image_b64}"
-
-        messages: list[dict[str, Any]] = [
-            {
-                "role": "user",
-                "content": [
-                    {"type": "text", "text": prompt_text},
-                    {"type": "image_url", "image_url": {"url": data_url}},
-                ],
-            }
-        ]
+        media_content: ImageContent | FileContent
+        if mime_type == "application/pdf":
+            media_content = FileContent(file=FileData(filename="source.pdf", file_data=data_url))
+        else:
+            media_content = ImageContent(image_url=ImageUrl(url=data_url))
 
         return OpenRouterRequest(
-            model=self.model,
-            messages=messages,
+            model=requested_model or self.model,
+            messages=(UserMessage(content=(TextContent(text=prompt_text), media_content)),),
             http_referer=self._settings.openrouter_http_referer,
             x_open_router_title=self._settings.openrouter_app_title,
+            temperature=temperature,
+            top_p=top_p,
         )
 
-    def _extract_text(self, response: Any) -> str:
-        choices = self._get_optional_attr(response, "choices")
-        if not choices:
-            raise ProviderResponseError("OpenRouter response missing choices")
-
-        first_choice = choices[0]
-        message = self._get_optional_attr(first_choice, "message")
-        if message is None:
-            raise ProviderResponseError("OpenRouter response missing assistant message")
-
-        content = self._get_optional_attr(message, "content")
+    def _extract_text(self, response: OpenRouterResponse) -> str:
+        content = response.choices[0].message.content
         text = self._normalize_content(content)
         if not text:
             raise ProviderResponseError("OpenRouter response contained no transcription text")
         return text
 
-    def _extract_finish_reason(self, response: Any) -> str | None:
-        choices = self._get_optional_attr(response, "choices")
-        if not choices:
-            return None
-        first_choice = choices[0]
-        finish_reason = self._get_optional_attr(first_choice, "finish_reason")
-        if isinstance(finish_reason, str) and finish_reason.strip():
-            return finish_reason.strip()
-        return None
-
-    def _extract_usage(self, response: Any) -> tuple[int | None, int | None, int | None]:
-        usage = self._get_optional_attr(response, "usage")
-        if usage is None:
-            return None, None, None
-
-        input_tokens = self._as_int(self._get_optional_attr(usage, "prompt_tokens"))
-        output_tokens = self._as_int(self._get_optional_attr(usage, "completion_tokens"))
-        total_tokens = self._as_int(self._get_optional_attr(usage, "total_tokens"))
-
-        if input_tokens is None:
-            input_tokens = self._as_int(self._get_optional_attr(usage, "input_tokens"))
-        if output_tokens is None:
-            output_tokens = self._as_int(self._get_optional_attr(usage, "output_tokens"))
-        if total_tokens is None:
-            total_tokens = self._as_int(self._get_optional_attr(usage, "total"))
-
-        return input_tokens, output_tokens, total_tokens
-
-    def _normalize_content(self, content: Any) -> str:
+    def _normalize_content(self, content: str | tuple[ResponseContentPart, ...] | None) -> str:
         if isinstance(content, str):
             return content.strip()
 
-        if isinstance(content, list):
-            parts: list[str] = []
-            for item in content:
-                text_part = None
-                text_part = item.get("text") if isinstance(item, dict) else self._get_optional_attr(item, "text")
-
-                if isinstance(text_part, str) and text_part.strip():
-                    parts.append(text_part.strip())
+        if isinstance(content, tuple):
+            parts = [item.text.strip() for item in content if item.text and item.text.strip()]
             return "\n".join(parts).strip()
 
         return ""
-
-    @staticmethod
-    def _get_optional_attr(obj: Any, key: str) -> Any:
-        if obj is None:
-            return None
-        if isinstance(obj, dict):
-            return obj.get(key)
-        return getattr(obj, key, None)
-
-    @staticmethod
-    def _as_int(value: Any) -> int | None:
-        if isinstance(value, int):
-            return value
-        return None

src/transcription/services/__init__.py

@@ -2,12 +2,30 @@
 
 from dataclasses import dataclass
 from dataclasses import field
+from typing import Self
 
+from sqlalchemy.ext.asyncio import async_sessionmaker
+from sqlmodel.ext.asyncio.session import AsyncSession
+
+from ..config import Settings
 from .documents import DocumentService
+from .evidence import EvidenceService
 from .jobs import JobService
-from .transcription import TranscriptionService
+from .people import PeopleService
+from .photos import PhotosService
+from .prompts import PromptStore
+from .sources import SourceService
 
-__all__ = ["DocumentService", "JobService", "ServiceBundle", "TranscriptionService"]
+__all__ = [
+    "DocumentService",
+    "EvidenceService",
+    "JobService",
+    "PeopleService",
+    "PhotosService",
+    "PromptStore",
+    "ServiceBundle",
+    "SourceService",
+]
 
 
 @dataclass(frozen=True, slots=True)
@@ -15,5 +33,31 @@ class ServiceBundle:
     """Container for all service instances."""
 
     documents: DocumentService = field(default_factory=DocumentService)
+    sources: SourceService = field(default_factory=SourceService)
     jobs: JobService = field(default_factory=JobService)
-    transcriptions: TranscriptionService = field(default_factory=TranscriptionService)
+    people: PeopleService = field(default_factory=PeopleService)
+    photos: PhotosService = field(default_factory=PhotosService)
+    evidence: EvidenceService = field(default_factory=EvidenceService)
+
+    @classmethod
+    def from_session_factory(
+        cls,
+        session_factory: async_sessionmaker[AsyncSession] | None = None,
+        *,
+        settings: Settings | None = None,
+    ) -> Self:
+        """Build a bundle whose services all share one session factory and settings."""
+        if session_factory is None:
+            return cls()
+        return cls(
+            documents=DocumentService(session_factory=session_factory, settings=settings),
+            sources=SourceService(session_factory=session_factory, settings=settings),
+            jobs=JobService(session_factory=session_factory, settings=settings),
+            people=PeopleService(session_factory=session_factory, settings=settings),
+            photos=PhotosService(session_factory=session_factory, settings=settings),
+            evidence=EvidenceService(session_factory=session_factory, settings=settings),
+        )
+
+    async def aclose(self) -> None:
+        """Release provider resources held by the bundle."""
+        await self.sources.aclose()

src/transcription/services/base.py

@@ -1,14 +1,17 @@
-import asyncio
 from abc import ABC
 from collections.abc import Sequence
 from contextlib import asynccontextmanager
+from typing import Any
 
 from sqlalchemy.ext.asyncio import async_sessionmaker
 from sqlmodel.ext.asyncio.session import AsyncSession
 
 from ..config import Settings
 from ..config import get_settings
-from ..db.runtime import get_session_factory
+from ..db.session import resolve_session_factory
+from ..db.session import session_scope
+from ..errors import AppError
+from ..errors import ErrorCategory
 
 
 class ServiceBase(ABC):
@@ -16,27 +19,23 @@ class ServiceBase(ABC):
 
     settings: Settings
     session_factory: async_sessionmaker[AsyncSession]
-    queue: asyncio.Queue
 
     def __init__(
         self,
         session_factory: async_sessionmaker[AsyncSession] | None = None,
-        queue: asyncio.Queue | None = None,
+        settings: Settings | None = None,
     ):
-        self.settings = get_settings()
-        self.session_factory = session_factory or get_session_factory()
-        self.queue = queue or asyncio.Queue()
+        self.settings = settings or get_settings()
+        self.session_factory = session_factory or resolve_session_factory(settings=self.settings)
 
     @asynccontextmanager
     async def _session_scope(self, session: AsyncSession | None = None):
         """Provide a transactional scope around a series of operations."""
-        if session is not None:
-            # Reuse the provided session if one is passed in
-            yield session
-        else:
-            # Otherwise, create a new session for this scope
-            async with self.session_factory() as new_session:
-                yield new_session
+        async with session_scope(
+            session_factory=self.session_factory,
+            session=session,
+        ) as active_session:
+            yield active_session
 
     async def _finalize(
         self,
@@ -58,3 +57,28 @@ class ServiceBase(ABC):
 
         for obj in refresh:
             await session.refresh(obj)
+
+    async def _get_or_raise[ModelT](
+        self,
+        model: type[ModelT],
+        entity_id: object,
+        *,
+        session: AsyncSession,
+        error: type[AppError],
+        noun: str,
+        suggestion: str,
+        options: Sequence[Any] = (),
+    ) -> ModelT:
+        """Load an entity by primary key or raise a not-found service error.
+
+        ``noun`` and ``suggestion`` are supplied by the caller so each domain
+        keeps its own user-facing wording.
+        """
+        entity = await session.get(model, entity_id, options=list(options) or None)
+        if entity is None:
+            raise error(
+                f"{noun} with id {entity_id} not found",
+                category=ErrorCategory.NOT_FOUND,
+                suggestion=suggestion,
+            )
+        return entity

src/transcription/services/documents.py

@@ -1,18 +1,36 @@
 import logging
+import shutil
 from collections.abc import Sequence
 from dataclasses import dataclass
+from datetime import date
+from datetime import datetime
 from pathlib import Path
+from typing import Any
 from uuid import UUID
 
 from sqlalchemy.exc import IntegrityError
-from sqlalchemy.orm import selectinload
+from sqlalchemy.ext.asyncio import async_sessionmaker
+from sqlmodel import SQLModel
+from sqlmodel import col
 from sqlmodel import select
 from sqlmodel.ext.asyncio.session import AsyncSession
 
+from ..config import Settings
+from ..db.loading import orm_attribute
+from ..db.loading import selectinload
+from ..db.models import Document
+from ..db.models import DocumentPerson
+from ..db.models import DocumentTag
+from ..db.models import DocumentType
+from ..db.models import Tag
+from ..db.registries import AUTHOR_ROLE_SEMANTIC_KEY
 from ..errors import AppError
 from ..errors import ErrorCategory
-from ..models import Document
 from .base import ServiceBase
+from .registry import RegistryService
+from .registry import RegistrySummary
+from .source_media import lookup_source_mime_type
+from .source_media import supported_source_formats
 
 logger = logging.getLogger(__name__)
 
@@ -25,27 +43,141 @@ class MissingSourceError(DocumentError):
     """Raised when a document has no associated sources."""
 
 
-class UploadError(DocumentError):
-    """Raised when uploaded content cannot be persisted safely."""
-
-
 class DocumentAlreadyExistsError(DocumentError):
     """Raised when a document with the same name already exists in the database."""
 
 
-@dataclass(frozen=True)
-class UploadJobResult:
-    """Summary of created upload records."""
+class DocumentDeleteBlockedError(DocumentError):
+    """Raised when a document delete is blocked by dependent records."""
 
-    document_id: UUID
-    job_id: UUID
-    stored_path: Path
-    original_filename: str
+
+class DocumentTypeError(DocumentError):
+    """Raised when Document Type maintenance fails."""
+
+
+class TagError(DocumentError):
+    """Raised when Tag maintenance fails."""
+
+
+class DocumentTypeRegistry(RegistryService[DocumentType]):
+    """Document Type registry maintenance."""
+
+    model = DocumentType
+    error = DocumentTypeError
+    noun = "Document Type"
+    short_noun = "type"
+    referenced_retainer = "historical Documents"
+
+    def reference_model(self) -> type[SQLModel]:
+        return Document
+
+    def reference_id_column(self) -> Any:
+        return col(Document.id)
+
+    def reference_key_column(self) -> Any:
+        return col(Document.document_type_id)
+
+
+type DocumentTypeSummary = RegistrySummary
+
+
+class TagRegistry(RegistryService[Tag]):
+    """Tag registry maintenance."""
+
+    model = Tag
+    error = TagError
+    noun = "Tag"
+    short_noun = "tag"
+    referenced_retainer = "historical Documents"
+
+    def reference_model(self) -> type[SQLModel]:
+        return DocumentTag
+
+    def reference_id_column(self) -> Any:
+        return col(DocumentTag.id)
+
+    def reference_key_column(self) -> Any:
+        return col(DocumentTag.tag_id)
+
+
+type TagSummary = RegistrySummary
+
+
+@dataclass(frozen=True, slots=True)
+class DocumentPrintSource:
+    id: UUID
+    page_number: int
+    media_type: str
+    current_text: str | None
+
+
+@dataclass(frozen=True, slots=True)
+class DocumentPrintJob:
+    id: UUID
+    date_created: datetime
+    provider: str | None
+    model: str | None
+    prompt_name: str | None
+    retry_count: int
+    status: str
+
+
+@dataclass(frozen=True, slots=True)
+class DocumentPrintProjection:
+    id: UUID
+    title: str
+    document_type: str | None
+    authors: tuple[str, ...]
+    document_date: date | None
+    document_date_raw: str | None
+    location_created: str | None
+    archive_identifier: str | None
+    notes: str | None
+    sources: tuple[DocumentPrintSource, ...]
+    jobs: tuple[DocumentPrintJob, ...]
 
 
 class DocumentService(ServiceBase):
     """Thin service class for managing documents in the database."""
 
+    def __init__(
+        self,
+        session_factory: async_sessionmaker[AsyncSession] | None = None,
+        settings: Settings | None = None,
+    ) -> None:
+        super().__init__(session_factory, settings)
+        self._document_types = DocumentTypeRegistry(self.session_factory, self.settings)
+        self._tags = TagRegistry(self.session_factory, self.settings)
+
+    async def _validate_document_type(self, *, session: AsyncSession, document: Document) -> None:
+        """Validate the UUID-backed Document Type reference."""
+        if document.document_type_id is None:
+            return
+        if await session.get(DocumentType, document.document_type_id) is None:
+            raise DocumentError(
+                f"Document type with id {document.document_type_id} not found",
+                category=ErrorCategory.VALIDATION,
+                suggestion="Select a valid document type and retry.",
+            )
+
+    async def _read_document(
+        self,
+        *,
+        session: AsyncSession,
+        document_id: UUID,
+        options: Sequence[Any] = (),
+        suggestion: str = "Verify the document id and retry.",
+    ) -> Document:
+        return await self._get_or_raise(
+            Document,
+            document_id,
+            session=session,
+            error=DocumentError,
+            noun="Document",
+            suggestion=suggestion,
+            options=options,
+        )
+
     #
     # CRUD Operations
     #
@@ -58,6 +190,7 @@ class DocumentService(ServiceBase):
     ) -> Document:
         """Create a new document in the database."""
         async with self._session_scope(session) as _session:
+            await self._validate_document_type(session=_session, document=document)
             _session.add(document)
             try:
                 await self._finalize(session=_session, caller_session=session, refresh=(document,))
@@ -75,21 +208,16 @@ class DocumentService(ServiceBase):
         The selectinload option is used to eagerly load related jobs and sources.
         """
         async with self._session_scope(session) as _session:
-            document = await _session.get(
-                Document,
-                document_id,
+            document = await self._read_document(
+                session=_session,
+                document_id=document_id,
                 options=(
-                    selectinload(Document.jobs),  # pyright: ignore[reportArgumentType]
-                    selectinload(Document.sources),  # pyright: ignore[reportArgumentType]
+                    selectinload(Document.jobs),
+                    selectinload(Document.sources),
                 ),
-            )
-        if document is None:
-            raise DocumentError(
-                f"Document with id {document_id} not found",
-                category=ErrorCategory.NOT_FOUND,
                 suggestion="Re-upload the source document and retry.",
             )
-        elif not document.sources:
+        if not document.sources:
             raise MissingSourceError(
                 f"Document with id {document_id} has no associated source records",
                 category=ErrorCategory.NOT_FOUND,
@@ -100,16 +228,59 @@ class DocumentService(ServiceBase):
     async def update_document(self, document: Document, *, session: AsyncSession | None = None) -> Document:
         """Update an existing document in the database."""
         async with self._session_scope(session) as _session:
+            await self._validate_document_type(session=_session, document=document)
             merged = await _session.merge(document)
             await self._finalize(session=_session, caller_session=session, refresh=(merged,))
             return merged
 
     async def delete_document(self, document: Document, *, session: AsyncSession | None = None) -> None:
         """Delete a document from the database."""
+        document_id = document.id
         async with self._session_scope(session) as _session:
-            await _session.delete(document)
+            existing = await self._read_document(
+                session=_session,
+                document_id=document.id,
+                options=(
+                    selectinload(Document.jobs),
+                    selectinload(Document.sources),
+                    selectinload(Document.document_people),
+                ),
+            )
+
+            has_jobs = bool(existing.jobs)
+            has_sources = bool(existing.sources)
+            if has_jobs or has_sources:
+                blocked_by: list[str] = []
+                if has_sources:
+                    blocked_by.append("Sources")
+                if has_jobs:
+                    blocked_by.append("Jobs")
+                raise DocumentDeleteBlockedError(
+                    f"Document delete blocked by related records: {', '.join(blocked_by)}",
+                    category=ErrorCategory.VALIDATION,
+                    suggestion="Remove related Sources and Jobs first, then retry deletion.",
+                )
+
+            for link in list(existing.document_people):
+                await _session.delete(link)
+
+            await _session.delete(existing)
             await self._finalize(session=_session, caller_session=session)
 
+        self._delete_document_storage_folder(document_id=document_id)
+
+    def _delete_document_storage_folder(self, *, document_id: UUID) -> None:
+        """Best-effort cleanup for document-scoped source storage."""
+        document_dir = self.settings.upload_dir / "documents" / str(document_id)
+        if not document_dir.exists():
+            return
+
+        try:
+            shutil.rmtree(document_dir)
+            logger.info("Deleted document storage folder: %s", document_dir)
+        except OSError:
+            logger.warning("Failed to delete document storage folder: %s", document_dir)
+
     # Query Operations
 
     async def query_documents(
@@ -124,7 +295,336 @@ class DocumentService(ServiceBase):
             return result.all()
 
     async def list_documents(self, *, session: AsyncSession | None = None) -> Sequence[Document]:
-        """List all documents in the database."""
+        """List documents with relations needed by the archival table."""
         async with self._session_scope(session) as _session:
-            result = await _session.exec(select(Document))
+            query = select(Document).options(
+                selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.person)),
+                selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.role_ref)),
+                selectinload(Document.document_type_ref),
+                selectinload(Document.document_tags).selectinload(orm_attribute(DocumentTag.tag_ref)),
+                selectinload(Document.sources),
+            )
+            result = await _session.exec(query)
             return result.all()
+
+    async def read_document_detail(self, document_id: UUID, *, session: AsyncSession | None = None) -> Document:
+        """Read a document with eagerly loaded relations for UI detail rendering."""
+        async with self._session_scope(session) as _session:
+            query = (
+                select(Document)
+                .options(
+                    selectinload(Document.jobs),
+                    selectinload(Document.sources),
+                    selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.person)),
+                    selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.role_ref)),
+                    selectinload(Document.document_type_ref),
+                    selectinload(Document.document_tags).selectinload(orm_attribute(DocumentTag.tag_ref)),
+                )
+                .where(Document.id == document_id)
+                .execution_options(populate_existing=True)
+            )
+            document = (await _session.exec(query)).first()
+        if document is None:
+            raise DocumentError(
+                f"Document with id {document_id} not found",
+                category=ErrorCategory.NOT_FOUND,
+                suggestion="Verify the document id and retry.",
+            )
+        return document
+
+    async def read_document_print_projection(
+        self,
+        document_id: UUID,
+        *,
+        session: AsyncSession | None = None,
+    ) -> DocumentPrintProjection:
+        """Build the safe, deterministic read model used by print previews."""
+        document = await self.read_document_detail(document_id, session=session)
+        authors = sorted(
+            (
+                link.person.full_name
+                for link in document.document_people
+                if link.person is not None
+                and link.role_ref is not None
+                and link.role_ref.semantic_key == AUTHOR_ROLE_SEMANTIC_KEY
+            ),
+            key=str.casefold,
+        )
+        sources = tuple(
+            DocumentPrintSource(
+                id=source.id,
+                page_number=source.page_number,
+                media_type=_print_media_type(source.filename),
+                current_text=_current_print_text(source.revised_text, source.raw_transcription),
+            )
+            for source in sorted(document.sources, key=lambda item: (item.page_number, item.id))
+        )
+        jobs = tuple(
+            DocumentPrintJob(
+                id=job.id,
+                date_created=job.date_created,
+                provider=job.provider,
+                model=job.model,
+                prompt_name=job.prompt_name,
+                retry_count=job.retry_count,
+                status=getattr(job.status, "value", str(job.status)),
+            )
+            for job in sorted(document.jobs, key=lambda item: (item.date_created, item.id))
+        )
+        return DocumentPrintProjection(
+            id=document.id,
+            title=document.name,
+            document_type=document.document_type_ref.label if document.document_type_ref is not None else None,
+            authors=tuple(authors),
+            document_date=document.document_date,
+            document_date_raw=document.document_date_raw,
+            location_created=document.location_created,
+            archive_identifier=document.archive_identifier,
+            notes=document.notes,
+            sources=sources,
+            jobs=jobs,
+        )
+
+    async def list_document_types(
+        self,
+        *,
+        active_only: bool = True,
+        session: AsyncSession | None = None,
+    ) -> Sequence[DocumentType]:
+        """List configured document types."""
+        return await self._document_types.list_entries(active_only=active_only, session=session)
+
+    async def list_tags(
+        self,
+        *,
+        active_only: bool = True,
+        session: AsyncSession | None = None,
+    ) -> Sequence[Tag]:
+        """List configured tags."""
+        return await self._tags.list_entries(active_only=active_only, session=session)
+
+    async def list_document_type_summaries(
+        self,
+        *,
+        session: AsyncSession | None = None,
+    ) -> Sequence[DocumentTypeSummary]:
+        """List Document Types alphabetically with current usage counts."""
+        rows = await self._document_types.list_entries_with_counts(session=session)
+        return [
+            RegistrySummary(
+                id=document_type.id,
+                label=document_type.label,
+                is_active=document_type.is_active,
+                is_built_in=document_type.semantic_key is not None,
+                reference_count=document_count,
+            )
+            for document_type, document_count in rows
+        ]
+
+    async def list_tag_summaries(
+        self,
+        *,
+        session: AsyncSession | None = None,
+    ) -> Sequence[TagSummary]:
+        """List Tags alphabetically with current usage counts."""
+        rows = await self._tags.list_entries_with_counts(session=session)
+        return [
+            RegistrySummary(
+                id=tag.id,
+                label=tag.label,
+                is_active=tag.is_active,
+                is_built_in=tag.semantic_key is not None,
+                reference_count=document_count,
+            )
+            for tag, document_count in rows
+        ]
+
+    async def create_document_type(
+        self,
+        *,
+        label: str,
+        is_active: bool = True,
+        session: AsyncSession | None = None,
+    ) -> DocumentType:
+        """Create a UUID-identified Document Type with a unique label."""
+        return await self._document_types.create_entry(label=label, is_active=is_active, session=session)
+
+    async def create_tag(
+        self,
+        *,
+        label: str,
+        is_active: bool = True,
+        session: AsyncSession | None = None,
+    ) -> Tag:
+        """Create a UUID-identified Tag with a unique label."""
+        return await self._tags.create_entry(label=label, is_active=is_active, session=session)
+
+    async def read_document_type(
+        self,
+        document_type_id: UUID,
+        *,
+        session: AsyncSession | None = None,
+    ) -> DocumentType:
+        """Read a Document Type by id."""
+        return await self._document_types.read_entry(document_type_id, session=session)
+
+    async def read_tag(
+        self,
+        tag_id: UUID,
+        *,
+        session: AsyncSession | None = None,
+    ) -> Tag:
+        """Read a Tag by id."""
+        return await self._tags.read_entry(tag_id, session=session)
+
+    async def update_document_type(
+        self,
+        document_type_id: UUID,
+        *,
+        label: str,
+        is_active: bool,
+        session: AsyncSession | None = None,
+    ) -> DocumentType:
+        """Update a Document Type label and active state."""
+        return await self._document_types.update_entry(
+            document_type_id,
+            label=label,
+            is_active=is_active,
+            session=session,
+        )
+
+    async def update_tag(
+        self,
+        tag_id: UUID,
+        *,
+        label: str,
+        is_active: bool,
+        session: AsyncSession | None = None,
+    ) -> Tag:
+        """Update a Tag label and active state."""
+        return await self._tags.update_entry(
+            tag_id,
+            label=label,
+            is_active=is_active,
+            session=session,
+        )
+
+    async def delete_document_type(
+        self,
+        document_type_id: UUID,
+        *,
+        session: AsyncSession | None = None,
+    ) -> None:
+        """Delete an unreferenced Document Type without cascade behavior."""
+        await self._document_types.delete_entry(document_type_id, session=session)
+
+    async def delete_tag(
+        self,
+        tag_id: UUID,
+        *,
+        session: AsyncSession | None = None,
+    ) -> None:
+        """Delete an unreferenced Tag without cascade behavior."""
+        await self._tags.delete_entry(tag_id, session=session)
+
+    async def is_document_type_referenced(
+        self,
+        document_type_id: UUID,
+        *,
+        session: AsyncSession | None = None,
+    ) -> bool:
+        """Return whether a Document references a Document Type."""
+        return await self._document_types.is_referenced(document_type_id, session=session)
+
+    async def is_tag_referenced(
+        self,
+        tag_id: UUID,
+        *,
+        session: AsyncSession | None = None,
+    ) -> bool:
+        """Return whether a Document references a Tag."""
+        return await self._tags.is_referenced(tag_id, session=session)
+
+    async def set_document_type(
+        self,
+        *,
+        document_id: UUID,
+        document_type_id: UUID,
+        session: AsyncSession | None = None,
+    ) -> Document:
+        """Set a Document Type by UUID."""
+        async with self._session_scope(session) as _session:
+            document = await self._read_document(session=_session, document_id=document_id)
+            document.document_type_id = document_type_id
+            await self._validate_document_type(session=_session, document=document)
+            await self._finalize(session=_session, caller_session=session, refresh=(document,))
+            return document
+
+    async def sync_document_tags_by_labels(
+        self,
+        *,
+        document_id: UUID,
+        labels: Sequence[str],
+        session: AsyncSession | None = None,
+    ) -> None:
+        """Replace a Document's tag set using label-based assignment."""
+        normalized_labels = [self._tags.normalize_label(label) for label in labels]
+        deduplicated_labels = list(dict.fromkeys(normalized_labels))
+        label_keys = [self._tags.label_key(label) for label in deduplicated_labels]
+
+        async with self._session_scope(session) as _session:
+            existing_document = await _session.get(Document, document_id)
+            if existing_document is None:
+                raise DocumentError(
+                    f"Document with id {document_id} not found",
+                    category=ErrorCategory.NOT_FOUND,
+                    suggestion="Refresh and select an existing document.",
+                )
+
+            existing_tags = (
+                (await _session.exec(select(Tag).where(col(Tag.normalized_label).in_(label_keys))))
+                .all()
+                if label_keys
+                else []
+            )
+            tags_by_key = {tag.normalized_label: tag for tag in existing_tags}
+            selected_tag_ids: set[UUID] = set()
+
+            for label in deduplicated_labels:
+                key = self._tags.label_key(label)
+                tag = tags_by_key.get(key)
+                if tag is None:
+                    tag = await self._tags.create_entry(label=label, is_active=True, session=_session)
+                    tags_by_key[key] = tag
+                selected_tag_ids.add(tag.id)
+
+            links = (
+                await _session.exec(select(DocumentTag).where(DocumentTag.document_id == document_id))
+            ).all()
+            existing_ids = {link.tag_id for link in links}
+
+            for link in links:
+                if link.tag_id not in selected_tag_ids:
+                    await _session.delete(link)
+
+            for tag_id in selected_tag_ids - existing_ids:
+                _session.add(DocumentTag(document_id=document_id, tag_id=tag_id))
+
+            await self._finalize(session=_session, caller_session=session)
+
+
+def _print_media_type(filename: str) -> str:
+    """Resolve a stored Source filename to its MIME type for print rendering."""
+    mime_type = lookup_source_mime_type(filename)
+    if mime_type is None:
+        raise DocumentError(
+            f"Unsupported Source format: {Path(filename).suffix.lower() or '<none>'}",
+            category=ErrorCategory.USER_INPUT,
+            suggestion=f"Use one of the supported Source formats: {supported_source_formats()}.",
+        )
+    return mime_type
+
+
+def _current_print_text(revised_text: str | None, raw_transcription: str | None) -> str | None:
+    selected = revised_text if revised_text is not None else raw_transcription
+    return selected if selected is not None and selected.strip() else None

src/transcription/services/errors.py

@@ -0,0 +1,32 @@
+"""Error vocabulary shared across the source, evidence, and prompt services.
+
+These live in a neutral module rather than in the service that raises them
+because more than one service raises them, and ``services.instructions.md``
+forbids a service module from importing a sibling. Orchestration modules and
+the UI import from here, so the exception a caller catches does not change when
+an operation moves between services.
+"""
+
+from __future__ import annotations
+
+from transcription.errors import AppError
+
+
+class PromptLoadError(AppError):
+    """Raised when prompt artifacts cannot be loaded safely."""
+
+
+class TranscriptionError(AppError):
+    """Raised when transcription execution fails."""
+
+
+class TranscriptionNotFoundError(TranscriptionError):
+    """Raised when a transcription-related resource is not found."""
+
+
+class SourceDeleteBlockedError(TranscriptionError):
+    """Raised when source deletion is blocked by dependency policy."""
+
+
+class CandidatePromotionError(TranscriptionError):
+    """Raised when a machine attempt cannot be selected for its Source."""

src/transcription/services/evidence.py

@@ -0,0 +1,210 @@
+"""Read and export the immutable execution evidence trail.
+
+``ExecutionAttempt`` is append-only: one row per provider call, written once by
+the transcription workflow and never updated. Everything here is therefore a
+read, a projection, or an export, with one exception - ``promote_machine_attempt``
+selects which attempt a ``Source`` presents, which is an evidence decision even
+though the write lands on ``Source``.
+"""
+
+from __future__ import annotations
+
+import base64
+import hashlib
+from collections.abc import Sequence
+from dataclasses import dataclass
+from uuid import UUID
+
+from pydantic import JsonValue
+from sqlalchemy import inspect as sqlalchemy_inspect
+from sqlmodel import col
+from sqlmodel import select
+from sqlmodel.ext.asyncio.session import AsyncSession
+
+from transcription.db.models import ExecutionAttempt
+from transcription.db.models import JobSourceStatus
+from transcription.db.models import Source
+from transcription.errors import ErrorCategory
+
+from ..db.loading import defer
+from .base import ServiceBase
+from .errors import CandidatePromotionError
+from .errors import TranscriptionNotFoundError
+
+
+@dataclass(frozen=True, slots=True)
+class LatestExecutionAttempt:
+    """One execution attempt plus the loader facts a caller needs to render it."""
+
+    attempt: ExecutionAttempt
+    transport_body_deferred: bool
+
+
+class EvidenceService(ServiceBase):
+    """Read, project, and export execution attempt evidence."""
+
+    async def read_latest_execution_attempt(
+        self,
+        *,
+        job_source_id: UUID,
+        session: AsyncSession | None = None,
+    ) -> LatestExecutionAttempt | None:
+        """Read only the latest immutable attempt for one compatibility projection.
+
+        The transport body is deferred because it can be arbitrarily large; the
+        returned read model reports that as a plain flag so callers never have to
+        inspect ORM loader state.
+        """
+        async with self._session_scope(session) as _session:
+            query = (
+                select(ExecutionAttempt)
+                .options(defer(ExecutionAttempt.transport_body))
+                .where(ExecutionAttempt.job_source_id == job_source_id)
+                .order_by(
+                    col(ExecutionAttempt.attempt_number).desc(),
+                    col(ExecutionAttempt.id).desc(),
+                )
+                .limit(1)
+            )
+            attempt = (await _session.exec(query)).first()
+            if attempt is None:
+                return None
+            deferred = "transport_body" in sqlalchemy_inspect(attempt).unloaded
+            return LatestExecutionAttempt(attempt=attempt, transport_body_deferred=deferred)
+
+    async def list_execution_attempts(
+        self,
+        *,
+        source_id: UUID | None = None,
+        job_id: UUID | None = None,
+        session: AsyncSession | None = None,
+    ) -> Sequence[ExecutionAttempt]:
+        """List immutable execution evidence in stable attempt order."""
+        async with self._session_scope(session) as _session:
+            query = select(ExecutionAttempt)
+            if source_id is not None:
+                query = query.where(ExecutionAttempt.source_id == source_id)
+            if job_id is not None:
+                query = query.where(ExecutionAttempt.job_id == job_id)
+            query = query.order_by(
+                col(ExecutionAttempt.job_id),
+                col(ExecutionAttempt.source_id),
+                col(ExecutionAttempt.attempt_number),
+                col(ExecutionAttempt.id),
+            )
+            return (await _session.exec(query)).all()
+
+    async def promote_machine_attempt(
+        self,
+        *,
+        source_id: UUID,
+        execution_attempt_id: UUID,
+        session: AsyncSession | None = None,
+    ) -> Source:
+        """Atomically select one successful machine attempt as the Source projection."""
+        async with self._session_scope(session) as _session:
+            source = await self._read_source(
+                session=_session,
+                source_id=source_id,
+                suggestion="Refresh Source Detail and retry.",
+            )
+            attempt = await _session.get(ExecutionAttempt, execution_attempt_id)
+            if (
+                attempt is None
+                or attempt.source_id != source_id
+                or attempt.status != JobSourceStatus.TRANSCRIBED
+                or not attempt.raw_transcription
+            ):
+                raise CandidatePromotionError(
+                    "Only a successful transcription attempt belonging to this Source can be selected",
+                    category=ErrorCategory.VALIDATION,
+                    suggestion="Select an available successful candidate from Source Detail.",
+                )
+            source.preferred_execution_attempt_id = attempt.id
+            source.raw_transcription = attempt.raw_transcription
+            await self._finalize(session=_session, caller_session=session, refresh=(source,))
+            return source
+
+    async def build_evidence_export(
+        self,
+        *,
+        source_id: UUID,
+        session: AsyncSession | None = None,
+    ) -> dict[str, JsonValue]:
+        """Build a versioned, source-reference-only evidence export."""
+        async with self._session_scope(session) as _session:
+            source = await self._read_source(session=_session, source_id=source_id)
+            attempts = list(await self.list_execution_attempts(source_id=source_id, session=_session))
+
+        attempt_payloads = [
+            {
+                "id": str(attempt.id),
+                "job_id": str(attempt.job_id),
+                "source_id": str(attempt.source_id),
+                "attempt_number": attempt.attempt_number,
+                "status": attempt.status.value,
+                "provider": attempt.provider,
+                "model": attempt.model,
+                "request_manifest": attempt.request_manifest,
+                "request_manifest_sha256": attempt.request_manifest_sha256,
+                "request_manifest_schema_version": attempt.request_manifest_schema_version,
+                "transport": {
+                    "response_received": attempt.response_received,
+                    "status_code": attempt.transport_status_code,
+                    "body_base64": (
+                        base64.b64encode(attempt.transport_body).decode("ascii")
+                        if attempt.transport_body is not None
+                        else None
+                    ),
+                    "body_sha256": (
+                        hashlib.sha256(attempt.transport_body).hexdigest()
+                        if attempt.transport_body is not None
+                        else None
+                    ),
+                    "content_type": attempt.transport_content_type,
+                    "content_encoding": attempt.transport_content_encoding,
+                    "safe_headers": attempt.transport_safe_headers,
+                    "request_id": attempt.router_request_id,
+                    "generation_id": attempt.router_generation_id,
+                },
+                "sdk_response_snapshot": attempt.sdk_response_snapshot,
+                "normalized_metadata": attempt.normalized_metadata,
+                "software_context": attempt.software_context,
+                "raw_transcription": attempt.raw_transcription,
+                "error_category": attempt.error_category,
+                "error_detail": attempt.error_detail,
+                "failure_phase": attempt.failure_phase,
+                "started_at": attempt.started_at.isoformat(),
+                "finished_at": attempt.finished_at.isoformat(),
+                "duration_ms": attempt.duration_ms,
+            }
+            for attempt in attempts
+        ]
+        return {
+            "schema_name": "transcription.evidence-export",
+            "schema_version": "1",
+            "source": {
+                "id": str(source.id),
+                "digest_sha256": source.file_hash,
+                "byte_size": source.file_size_bytes,
+                "page_number": source.page_number,
+                "upload_name": source.upload_name,
+            },
+            "attempts": attempt_payloads,
+        }
+
+    async def _read_source(
+        self,
+        *,
+        session: AsyncSession,
+        source_id: UUID,
+        suggestion: str = "Verify the source id and retry.",
+    ) -> Source:
+        return await self._get_or_raise(
+            Source,
+            source_id,
+            session=session,
+            error=TranscriptionNotFoundError,
+            noun="Source",
+            suggestion=suggestion,
+        )

src/transcription/services/jobs.py

@@ -1,17 +1,46 @@
+import logging
 from collections.abc import Sequence
 from datetime import UTC
 from datetime import datetime
 from uuid import UUID
 
-from sqlalchemy.orm import selectinload
+from sqlalchemy import func
+from sqlalchemy import update
+from sqlmodel import col
 from sqlmodel import select
 from sqlmodel.ext.asyncio.session import AsyncSession
 
-from ..models import Job
-from ..models import JobStatus
-from ..models import Source
+from ..db.loading import orm_attribute
+from ..db.loading import selectinload
+from ..db.models import ExecutionAttempt
+from ..db.models import Document
+from ..db.models import Job
+from ..db.models import JobSource
+from ..db.models import JobSourceStatus
+from ..db.models import JobStatus
+from ..db.models import Source
+from ..errors import AppError
+from ..errors import ErrorCategory
 from .base import ServiceBase
 
+logger = logging.getLogger(__name__)
+
+
+class JobDeleteBlockedError(AppError):
+    """Raised when a job delete operation is blocked by lifecycle policy."""
+
+
+class JobCancelBlockedError(AppError):
+    """Raised when a job cancel operation is blocked by lifecycle policy."""
+
+
+class JobResubmitBlockedError(AppError):
+    """Raised when a job resubmit operation is blocked by lifecycle policy."""
+
+
+class JobNotFoundError(AppError):
+    """Raised when a requested Job does not exist."""
+
 
 class JobService(ServiceBase):
     """Thin service class for managing jobs in the database."""
@@ -37,15 +66,15 @@ class JobService(ServiceBase):
             query = (
                 select(Job)
                 .options(
-                    selectinload(Job.document),  # pyright: ignore[reportArgumentType]
-                    selectinload(Job.sources).selectinload(Source.revision),  # pyright: ignore[reportArgumentType]
+                    selectinload(Job.document).selectinload(orm_attribute(Document.sources)),
+                    selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
                 )
                 .where(Job.id == job_id)
                 .execution_options(populate_existing=True)
             )
             job = (await _session.exec(query)).first()
         if job is None:
-            raise ValueError(f"Job with id {job_id} not found")
+            raise self._not_found(job_id)
         return job
 
     async def update_job(self, job: Job, session: AsyncSession | None = None) -> Job:
@@ -73,28 +102,28 @@ class JobService(ServiceBase):
         """Query jobs from the database based on provided filters."""
         async with self._session_scope(session) as _session:
             query = select(Job).options(
-                selectinload(Job.document),  # pyright: ignore[reportArgumentType]
-                selectinload(Job.sources),  # pyright: ignore[reportArgumentType]
+                selectinload(Job.document),
+                selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
             )
             if status is not None:
                 query = query.where(Job.status == status)
             if filename is not None:
-                query = query.where(Job.sources.any(Source.filename == filename))
+                query = query.where(
+                    col(Job.job_sources).any(col(JobSource.source).has(col(Source.filename) == filename))
+                )
             result = await _session.exec(query)
             return result.all()
 
     async def list_jobs(
         self,
         *,
-        load_docs: bool = False,
         session: AsyncSession | None = None,
     ) -> Sequence[Job]:
         """List all jobs in the database with eagerly loaded documents."""
-        _ = load_docs
         async with self._session_scope(session) as _session:
             query = select(Job).options(
-                selectinload(Job.document),  # pyright: ignore[reportArgumentType]
-                selectinload(Job.sources),  # pyright: ignore[reportArgumentType]
+                selectinload(Job.document),
+                selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
             )
             result = await _session.exec(query)
             return result.all()
@@ -126,37 +155,77 @@ class JobService(ServiceBase):
         async with self._session_scope(session) as _session:
             query = (
                 select(Job)
-                .options(selectinload(Job.document))  # pyright: ignore[reportArgumentType]
+                .options(
+                    selectinload(Job.document),
+                    selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
+                )
                 .where(Job.id == job_id)
                 .execution_options(populate_existing=True)
             )
             job = (await _session.exec(query)).first()
             if job is None:
-                raise ValueError(f"Job with id {job_id} not found")
+                raise self._not_found(job_id)
             job.status = status
             if retry_count_increment:
                 job.retry_count += retry_count_increment
-            job.date_updated = datetime.now(UTC)
             await self._finalize(session=_session, caller_session=session, refresh=(job,))
             return job
 
-    async def read_next_queued_job(
+    async def claim_next_queued_job(
         self,
         *,
         session: AsyncSession | None = None,
     ) -> Job | None:
-        """Read the next queued job ordered by creation time."""
+        """Atomically claim the oldest queued job by transitioning it to PROCESSING.
+
+        The selection is deliberately unadorned: no eager loads are applied to the
+        hot poll, because callers re-read the claimed job with the relationships
+        they actually need. On PostgreSQL the row is locked with ``SKIP LOCKED`` so
+        concurrent workers never contend for the same job.
+        """
         async with self._session_scope(session) as _session:
-            query = (
-                select(Job)
-                .options(
-                    selectinload(Job.document),  # pyright: ignore[reportArgumentType]
-                    selectinload(Job.sources).selectinload(Source.revision),  # pyright: ignore[reportArgumentType]
+            dialect = _session.get_bind().dialect.name
+            if dialect == "postgresql":
+                query = (
+                    select(Job)
+                    .where(Job.status == JobStatus.QUEUED)
+                    # Break ties by id so "next" is stable when two rows share close timestamps.
+                    .order_by(col(Job.date_created), col(Job.id))
+                    .limit(1)
+                    .with_for_update(skip_locked=True)
                 )
-                .where(Job.status == JobStatus.QUEUED)
-                .order_by(Job.date_created)  # pyright: ignore[reportArgumentType]
+                job = (await _session.exec(query)).first()
+                if job is None:
+                    return None
+                job.status = JobStatus.PROCESSING
+                await self._finalize(session=_session, caller_session=session, refresh=(job,))
+                return job
+
+            now = datetime.now(UTC)
+            queued_job_id = (
+                select(col(Job.id))
+                .where(col(Job.status) == JobStatus.QUEUED)
+                .order_by(col(Job.date_created), col(Job.id))
+                .limit(1)
+                .scalar_subquery()
             )
-            return (await _session.exec(query)).first()
+            claim_statement = (
+                update(Job)
+                .where(col(Job.id) == queued_job_id)
+                .where(col(Job.status) == JobStatus.QUEUED)
+                .values(status=JobStatus.PROCESSING, date_updated=now)
+                .returning(col(Job.id))
+            )
+            claimed_row = (await _session.exec(claim_statement)).first()
+            if claimed_row is None:
+                return None
+            claimed_job_id = claimed_row if isinstance(claimed_row, UUID) else claimed_row[0]
+
+            job = (await _session.exec(select(Job).where(Job.id == claimed_job_id))).first()
+            if job is None:
+                return None
+            await self._finalize(session=_session, caller_session=session, refresh=(job,))
+            return job
 
     async def requeue_stale_processing_jobs(
         self,
@@ -170,11 +239,7 @@ class JobService(ServiceBase):
         ``stale_before`` are considered stale and re-queued.
         """
         async with self._session_scope(session) as _session:
-            query = (
-                select(Job)
-                .where(Job.status == JobStatus.PROCESSING)
-                .where(Job.date_updated < stale_before)
-            )
+            query = select(Job).where(Job.status == JobStatus.PROCESSING).where(Job.date_updated < stale_before)
             stale_jobs = (await _session.exec(query)).all()
             if not stale_jobs:
                 return 0
@@ -186,3 +251,173 @@ class JobService(ServiceBase):
 
             await self._finalize(session=_session, caller_session=session, refresh=stale_jobs)
             return len(stale_jobs)
+
+    async def delete_job_with_guardrails(self, *, job_id: UUID, session: AsyncSession | None = None) -> None:
+        """Delete a job with lifecycle guardrails and dependent cleanup policy.
+
+        Policy:
+        - Block when the job is actively processing.
+        - Otherwise remove related JobSource rows, then delete the job.
+        """
+        async with self._session_scope(session) as _session:
+            query = (
+                select(Job)
+                .options(selectinload(Job.job_sources))
+                .where(Job.id == job_id)
+                .execution_options(populate_existing=True)
+            )
+            job = (await _session.exec(query)).first()
+            if job is None:
+                raise self._not_found(job_id)
+
+            if job.status == JobStatus.PROCESSING:
+                raise JobDeleteBlockedError(
+                    "Job delete blocked while status is processing",
+                    category=ErrorCategory.VALIDATION,
+                    suggestion="Wait for processing to complete, or move the job out of processing before deleting.",
+                )
+            attempt_count = (
+                await _session.exec(
+                    select(func.count())
+                    .select_from(ExecutionAttempt)
+                    .where(ExecutionAttempt.job_id == job_id)
+                )
+            ).one()
+            if attempt_count:
+                raise JobDeleteBlockedError(
+                    "Job delete blocked because immutable execution evidence exists",
+                    category=ErrorCategory.CONFLICT,
+                    suggestion=(
+                        "Retain the Job as processing history. Evidence deletion requires "
+                        "an explicit retention workflow."
+                    ),
+                )
+
+            for job_source in list(job.job_sources):
+                await _session.delete(job_source)
+
+            await _session.delete(job)
+            await self._finalize(session=_session, caller_session=session)
+
+    async def delete_job_and_evidence(self, *, job_id: UUID) -> None:
+        """Explicitly delete a terminal job and all evidence owned by its attempts."""
+        async with self._session_scope() as session:
+            job = (
+                await session.exec(
+                    select(Job)
+                    .options(selectinload(Job.job_sources))
+                    .where(Job.id == job_id)
+                    .execution_options(populate_existing=True)
+                )
+            ).first()
+            if job is None:
+                raise self._not_found(job_id)
+            if job.status == JobStatus.PROCESSING:
+                raise JobDeleteBlockedError(
+                    "Job delete blocked while status is processing",
+                    category=ErrorCategory.VALIDATION,
+                    suggestion="Wait for processing to complete, or cancel it before deleting evidence.",
+                )
+
+            attempts = list(
+                (
+                    await session.exec(
+                        select(ExecutionAttempt).where(ExecutionAttempt.job_id == job_id)
+                    )
+                ).all()
+            )
+            for attempt in attempts:
+                await session.delete(attempt)
+            await session.flush()
+
+            for job_source in list(job.job_sources):
+                await session.delete(job_source)
+            await session.flush()
+            await session.delete(job)
+            await self._finalize(session=session, caller_session=None)
+
+    async def cancel_job(self, *, job_id: UUID, session: AsyncSession | None = None) -> Job:
+        """Cancel a queued/processing job and stop remaining source work."""
+        async with self._session_scope(session) as _session:
+            query = (
+                select(Job)
+                .options(
+                    selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
+                )
+                .where(Job.id == job_id)
+                .execution_options(populate_existing=True)
+            )
+            job = (await _session.exec(query)).first()
+            if job is None:
+                raise self._not_found(job_id)
+
+            if job.status == JobStatus.TRANSCRIBED:
+                raise JobCancelBlockedError(
+                    "Job cancel is not allowed for transcribed jobs",
+                    category=ErrorCategory.VALIDATION,
+                    suggestion="Use resubmit for reprocessing needs, or leave the terminal job unchanged.",
+                )
+
+            now = datetime.now(UTC)
+            job.status = JobStatus.FAILED
+            job.date_updated = now
+
+            for job_source in job.job_sources:
+                if job_source.status == JobSourceStatus.TRANSCRIBED:
+                    continue
+                job_source.status = JobSourceStatus.CANCELLED
+
+            await self._finalize(session=_session, caller_session=session, refresh=(job,))
+            return job
+
+    async def resubmit_failed_sources(self, *, job_id: UUID, session: AsyncSession | None = None) -> int:
+        """Reset failed source executions and queue the job for reprocessing."""
+        async with self._session_scope(session) as _session:
+            query = (
+                select(Job)
+                .options(
+                    selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
+                )
+                .where(Job.id == job_id)
+                .execution_options(populate_existing=True)
+            )
+            job = (await _session.exec(query)).first()
+            if job is None:
+                raise self._not_found(job_id)
+
+            if job.status == JobStatus.PROCESSING:
+                raise JobResubmitBlockedError(
+                    "Job resubmit is blocked while processing is active",
+                    category=ErrorCategory.VALIDATION,
+                    suggestion="Cancel processing first, then resubmit remaining sources.",
+                )
+
+            # Cancelled pages are re-attemptable: older flows wrote FAILED,
+            # so resubmit already reset them. Excluding CANCELLED here would make
+            # cancelled work permanently unrecoverable.
+            resubmittable = {JobSourceStatus.FAILED, JobSourceStatus.CANCELLED}
+            candidates = [job_source for job_source in job.job_sources if job_source.status in resubmittable]
+            if not candidates:
+                raise JobResubmitBlockedError(
+                    "Job has no failed or cancelled sources to resubmit",
+                    category=ErrorCategory.VALIDATION,
+                    suggestion="Only failed or cancelled sources can be resubmitted.",
+                )
+
+            now = datetime.now(UTC)
+            for job_source in candidates:
+                job_source.status = JobSourceStatus.PENDING
+
+            job.status = JobStatus.QUEUED
+            job.date_updated = now
+
+            await self._finalize(session=_session, caller_session=session, refresh=(job,))
+            return len(candidates)
+
+    @staticmethod
+    def _not_found(job_id: UUID) -> JobNotFoundError:
+        return JobNotFoundError(
+            f"Job with id {job_id} not found",
+            category=ErrorCategory.NOT_FOUND,
+            suggestion="Verify the Job id and retry.",
+        )

src/transcription/services/media_storage.py

@@ -0,0 +1,87 @@
+"""Single implementation for persisting uploaded media bytes to disk.
+
+Source pages, Person portraits, and homepage images all follow the same
+sequence: resolve a target directory, create it, write the bytes, and translate
+an ``OSError`` into a domain error. The write itself runs on a worker thread so
+it never blocks the event loop.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+from pathlib import Path
+from uuid import uuid4
+
+from ..errors import AppError
+from ..errors import ErrorCategory
+
+logger = logging.getLogger(__name__)
+
+
+def build_stored_filename(*, filename: str, filename_stem: str | None = None) -> str:
+    """Return a safe stored filename preserving the submitted extension."""
+    safe_name = Path(filename).name
+    suffix = Path(safe_name).suffix.lower()
+    stem = filename_stem or str(uuid4())
+    return f"{stem}{suffix}"
+
+
+async def write_media_bytes(
+    *,
+    target_dir: Path,
+    stored_name: str,
+    file_bytes: bytes,
+    error: type[AppError],
+    failure_message: str,
+    failure_suggestion: str,
+    log_label: str,
+) -> Path:
+    """Create ``target_dir`` and write ``file_bytes`` into it off the event loop."""
+    stored_path = target_dir / stored_name
+    try:
+        await asyncio.to_thread(_write, stored_path, file_bytes)
+    except OSError as exc:
+        raise error(
+            failure_message,
+            category=ErrorCategory.INFRA_PERSISTENT,
+            suggestion=failure_suggestion,
+        ) from exc
+
+    logger.info("Stored %s: %s", log_label, stored_path)
+    return stored_path
+
+
+async def persist_named_media(
+    *,
+    root: Path,
+    filename: str,
+    file_bytes: bytes,
+    error: type[AppError],
+    failure_message: str,
+    failure_suggestion: str,
+    log_label: str,
+    namespace: Path | str | None = None,
+    filename_stem: str | None = None,
+    preserve_original_name: bool = False,
+) -> Path:
+    """Resolve a target directory/name and persist media bytes safely."""
+    target_dir = root if namespace is None else root / Path(namespace)
+    stored_name = Path(filename).name if preserve_original_name else build_stored_filename(
+        filename=filename,
+        filename_stem=filename_stem,
+    )
+    return await write_media_bytes(
+        target_dir=target_dir,
+        stored_name=stored_name,
+        file_bytes=file_bytes,
+        error=error,
+        failure_message=failure_message,
+        failure_suggestion=failure_suggestion,
+        log_label=log_label,
+    )
+
+
+def _write(stored_path: Path, file_bytes: bytes) -> None:
+    stored_path.parent.mkdir(parents=True, exist_ok=True)
+    stored_path.write_bytes(file_bytes)

src/transcription/services/normalization.py

@@ -0,0 +1,117 @@
+"""Metadata-directed orientation normalization applied to image bytes at ingest.
+
+Uploaded pages are stored upright, so nothing downstream has to derive a
+rotated copy: every stored byte is already the byte the provider is sent.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import io
+import logging
+from dataclasses import dataclass
+
+from PIL import Image
+from PIL import JpegImagePlugin
+from PIL import UnidentifiedImageError
+from PIL.TiffImagePlugin import TiffImageFile
+
+from transcription.errors import AppError
+from transcription.errors import ErrorCategory
+
+logger = logging.getLogger(__name__)
+
+ORIENTATION_TAG = 274
+
+NORMALIZED_MEDIA_TYPES = frozenset({"image/jpeg", "image/png", "image/tiff"})
+
+_TRANSPOSE_BY_ORIENTATION = {
+    3: (Image.Transpose.ROTATE_180, 180),
+    6: (Image.Transpose.ROTATE_270, 90),
+    8: (Image.Transpose.ROTATE_90, 270),
+}
+
+
+class OrientationNormalizationError(AppError):
+    """Raised when a supported raster image cannot be normalized safely."""
+
+
+@dataclass(frozen=True, slots=True)
+class OrientationNormalization:
+    """Upright image bytes and the rotation that produced them."""
+
+    content: bytes
+    original_orientation: int
+    applied_rotation_degrees: int
+
+
+def normalize_orientation(content: bytes, *, media_type: str) -> OrientationNormalization | None:
+    """Physically apply supported EXIF rotation, returning None for a safe no-op.
+
+    JPEG output reuses the source quantization tables and chroma subsampling
+    rather than re-quantizing at a fixed quality. Measured across the corpus
+    that is better on both axes at once - 51.5-55.0 dB PSNR against 50.0-53.5,
+    and slightly smaller output against 38% larger - and it imposes no
+    constraint on image dimensions.
+
+    Blocking. Async callers must use :func:`normalize_orientation_async`.
+    """
+    if media_type not in NORMALIZED_MEDIA_TYPES:
+        return None
+    try:
+        image_file = Image.open(io.BytesIO(content))
+    except (OSError, ValueError, UnidentifiedImageError):
+        # Undecodable content is not this function's business to reject. Ingest
+        # accepted such bytes before orientation moved here, and decision A
+        # forbids changing what an upload does.
+        logger.info("Skipped orientation normalization for undecodable content (%s)", media_type)
+        return None
+    try:
+        with image_file as image:
+            orientation = int(image.getexif().get(ORIENTATION_TAG, 1))
+            transformation = _TRANSPOSE_BY_ORIENTATION.get(orientation)
+            if transformation is None:
+                return None
+
+            transpose, rotation = transformation
+            # Pillow applies TIFF orientation while decoding; copying freezes those upright pixels.
+            normalized = image.copy() if isinstance(image, TiffImageFile) else image.transpose(transpose)
+            output = io.BytesIO()
+            exif = normalized.getexif()
+            if ORIENTATION_TAG in exif:
+                del exif[ORIENTATION_TAG]
+            save_kwargs: dict[str, object] = {"format": image.format}
+            if image.format in {"JPEG", "PNG"}:
+                save_kwargs["exif"] = exif.tobytes()
+            if isinstance(image, JpegImagePlugin.JpegImageFile):
+                # Reusing the source quantization tables and subsampling preserves fidelity
+                # at a smaller size than any re-encode quality setting.
+                save_kwargs.update(
+                    {
+                        "qtables": image.quantization,
+                        "subsampling": JpegImagePlugin.get_sampling(image),
+                        "optimize": True,
+                    }
+                )
+            normalized.save(output, **save_kwargs)
+    except (OSError, ValueError, UnidentifiedImageError) as exc:
+        raise OrientationNormalizationError(
+            "Source image orientation could not be normalized",
+            category=ErrorCategory.VALIDATION,
+            suggestion="Verify that the uploaded Source is a valid supported raster image.",
+        ) from exc
+
+    return OrientationNormalization(
+        content=output.getvalue(),
+        original_orientation=orientation,
+        applied_rotation_degrees=rotation,
+    )
+
+
+async def normalize_orientation_async(content: bytes, *, media_type: str) -> OrientationNormalization | None:
+    """Run :func:`normalize_orientation` off the event loop.
+
+    Pillow decode, transpose, and re-encode are CPU-bound and scale with page
+    size, so they must not run on the request or worker event loop ([MED-01]).
+    """
+    return await asyncio.to_thread(normalize_orientation, content, media_type=media_type)

src/transcription/services/people.py

@@ -0,0 +1,515 @@
+"""People, relationship role, and document-person link services."""
+
+from __future__ import annotations
+
+import logging
+import re
+from collections.abc import Sequence
+from dataclasses import dataclass
+from typing import Any
+from uuid import UUID
+
+from sqlalchemy.exc import IntegrityError
+from sqlalchemy.ext.asyncio import async_sessionmaker
+from sqlmodel import SQLModel
+from sqlmodel import col
+from sqlmodel import select
+from sqlmodel.ext.asyncio.session import AsyncSession
+
+from ..config import Settings
+from ..db.loading import orm_attribute
+from ..db.loading import selectinload
+from ..db.models import Document
+from ..db.models import DocumentPerson
+from ..db.models import Photo
+from ..db.models import Person
+from ..db.models import PersonRole
+from ..errors import AppError
+from ..errors import ErrorCategory
+from .base import ServiceBase
+from .registry import RegistryService
+from .registry import RegistrySummary
+
+logger = logging.getLogger(__name__)
+
+FAMILY_SEARCH_ID_PATTERN = re.compile(r"^[A-Z0-9]{4}-[A-Z0-9]{3}$")
+
+
+class PeopleError(AppError):
+    """Raised when a Person or document-person relationship operation fails."""
+
+
+class PersonRoleError(PeopleError):
+    """Raised when Person Role maintenance fails."""
+
+
+class PersonRoleRegistry(RegistryService[PersonRole]):
+    """Person Role registry maintenance."""
+
+    model = PersonRole
+    error = PersonRoleError
+    noun = "Person Role"
+    short_noun = "role"
+    referenced_retainer = "historical relationships"
+
+    def reference_model(self) -> type[SQLModel]:
+        return DocumentPerson
+
+    def reference_id_column(self) -> Any:
+        return col(DocumentPerson.id)
+
+    def reference_key_column(self) -> Any:
+        return col(DocumentPerson.role_id)
+
+
+def normalize_family_search_id(value: str | None) -> str | None:
+    """Normalize and validate a FamilySearch tree person identifier."""
+    normalized = (value or "").strip().upper()
+    if not normalized:
+        return None
+    if not FAMILY_SEARCH_ID_PATTERN.fullmatch(normalized):
+        raise PeopleError(
+            "FamilySearch ID must use the format XXXX-XXX",
+            category=ErrorCategory.VALIDATION,
+            suggestion="Enter the seven-character FamilySearch person ID, including its hyphen.",
+        )
+    return normalized
+
+
+type PersonRoleSummary = RegistrySummary
+
+
+@dataclass(frozen=True, slots=True)
+class DocumentPersonInput:
+    """Complete desired relationship for one Person on a Document."""
+
+    person_id: UUID
+    role_id: UUID
+
+
+class PeopleService(ServiceBase):
+    """Manage People, relationship roles, and document-person links."""
+
+    def __init__(
+        self,
+        session_factory: async_sessionmaker[AsyncSession] | None = None,
+        settings: Settings | None = None,
+    ) -> None:
+        super().__init__(session_factory, settings)
+        self._person_roles = PersonRoleRegistry(self.session_factory, self.settings)
+
+    async def create_person(self, person: Person, *, session: AsyncSession | None = None) -> Person:
+        async with self._session_scope(session) as _session:
+            person.family_search_id = normalize_family_search_id(person.family_search_id)
+            _session.add(person)
+            try:
+                await self._finalize(session=_session, caller_session=session, refresh=(person,))
+            except IntegrityError as exc:
+                raise self._family_search_conflict(person.family_search_id) from exc
+            return person
+
+    async def read_person(self, person_id: UUID, *, session: AsyncSession | None = None) -> Person:
+        async with self._session_scope(session) as _session:
+            person = await _session.get(Person, person_id)
+        if person is None:
+            raise self._not_found(f"Person with id {person_id} not found")
+        return person
+
+    async def update_person(self, person: Person, *, session: AsyncSession | None = None) -> Person:
+        async with self._session_scope(session) as _session:
+            person.family_search_id = normalize_family_search_id(person.family_search_id)
+            merged = await _session.merge(person)
+            try:
+                await self._finalize(session=_session, caller_session=session, refresh=(merged,))
+            except IntegrityError as exc:
+                raise self._family_search_conflict(person.family_search_id) from exc
+            return merged
+
+    async def delete_person(self, person: Person, *, session: AsyncSession | None = None) -> None:
+        async with self._session_scope(session) as _session:
+            existing = await _session.get(
+                Person,
+                person.id,
+                options=(selectinload(Person.document_people), selectinload(Person.photos)),
+            )
+            if existing is None:
+                raise self._not_found(f"Person with id {person.id} not found")
+            if existing.photos:
+                raise PeopleError(
+                    "Person delete blocked by related records: Photos",
+                    category=ErrorCategory.VALIDATION,
+                    suggestion="Delete or reassign Person photos before deleting this record.",
+                )
+            for link in list(existing.document_people):
+                await _session.delete(link)
+            await _session.delete(existing)
+            await self._finalize(session=_session, caller_session=session)
+
+    async def create_document_person(
+        self,
+        document_person: DocumentPerson,
+        *,
+        session: AsyncSession | None = None,
+    ) -> DocumentPerson:
+        async with self._session_scope(session) as _session:
+            await self._validate_role(session=_session, role_id=document_person.role_id, require_active=True)
+            _session.add(document_person)
+            return await self._finalize_link(session=_session, caller_session=session, link=document_person)
+
+    async def read_document_person(
+        self,
+        document_person_id: UUID,
+        *,
+        session: AsyncSession | None = None,
+    ) -> DocumentPerson:
+        async with self._session_scope(session) as _session:
+            link = await _session.get(DocumentPerson, document_person_id)
+        if link is None:
+            raise self._not_found(f"DocumentPerson with id {document_person_id} not found")
+        return link
+
+    async def update_document_person(
+        self,
+        document_person: DocumentPerson,
+        *,
+        session: AsyncSession | None = None,
+    ) -> DocumentPerson:
+        async with self._session_scope(session) as _session:
+            existing = await _session.get(DocumentPerson, document_person.id)
+            if existing is None:
+                raise self._not_found(f"DocumentPerson with id {document_person.id} not found")
+            await self._validate_role(
+                session=_session,
+                role_id=document_person.role_id,
+                require_active=existing.role_id != document_person.role_id,
+            )
+            merged = await _session.merge(document_person)
+            return await self._finalize_link(session=_session, caller_session=session, link=merged)
+
+    async def delete_document_person(
+        self,
+        document_person: DocumentPerson,
+        *,
+        session: AsyncSession | None = None,
+    ) -> None:
+        async with self._session_scope(session) as _session:
+            await _session.delete(document_person)
+            await self._finalize(session=_session, caller_session=session)
+
+    async def read_person_detail(self, person_id: UUID, *, session: AsyncSession | None = None) -> Person:
+        async with self._session_scope(session) as _session:
+            query = (
+                select(Person)
+                .options(
+                    selectinload(Person.document_people)
+                    .selectinload(orm_attribute(DocumentPerson.document))
+                    .selectinload(orm_attribute(Document.sources)),
+                    selectinload(Person.document_people).selectinload(orm_attribute(DocumentPerson.role_ref)),
+                    selectinload(Person.photos),
+                )
+                .where(Person.id == person_id)
+                .execution_options(populate_existing=True)
+            )
+            person = (await _session.exec(query)).first()
+        if person is None:
+            raise self._not_found(f"Person with id {person_id} not found")
+        return person
+
+    async def list_people(self, *, session: AsyncSession | None = None) -> Sequence[Person]:
+        async with self._session_scope(session) as _session:
+            query = select(Person).options(selectinload(Person.document_people))
+            return (await _session.exec(query)).all()
+
+    async def list_person_roles(
+        self,
+        *,
+        active_only: bool = True,
+        session: AsyncSession | None = None,
+    ) -> Sequence[PersonRole]:
+        return await self._person_roles.list_entries(active_only=active_only, session=session)
+
+    async def list_person_role_summaries(
+        self,
+        *,
+        session: AsyncSession | None = None,
+    ) -> Sequence[PersonRoleSummary]:
+        """List Person Roles alphabetically with current link counts."""
+        rows = await self._person_roles.list_entries_with_counts(session=session)
+        return [
+            RegistrySummary(
+                id=role.id,
+                label=role.label,
+                is_active=role.is_active,
+                is_built_in=role.semantic_key is not None,
+                reference_count=link_count,
+            )
+            for role, link_count in rows
+        ]
+
+    async def create_person_role(
+        self,
+        *,
+        label: str,
+        is_active: bool = True,
+        session: AsyncSession | None = None,
+    ) -> PersonRole:
+        """Create a custom Person Role with a unique label."""
+        return await self._person_roles.create_entry(label=label, is_active=is_active, session=session)
+
+    async def read_person_role(
+        self,
+        person_role_id: UUID,
+        *,
+        session: AsyncSession | None = None,
+    ) -> PersonRole:
+        """Read a Person Role by id."""
+        return await self._person_roles.read_entry(person_role_id, session=session)
+
+    async def update_person_role(
+        self,
+        person_role_id: UUID,
+        *,
+        label: str,
+        is_active: bool,
+        session: AsyncSession | None = None,
+    ) -> PersonRole:
+        """Update mutable Person Role fields without changing semantic identity."""
+        return await self._person_roles.update_entry(
+            person_role_id,
+            label=label,
+            is_active=is_active,
+            session=session,
+        )
+
+    async def delete_person_role(
+        self,
+        person_role_id: UUID,
+        *,
+        session: AsyncSession | None = None,
+    ) -> None:
+        """Delete an unreferenced Person Role without cascade behavior."""
+        await self._person_roles.delete_entry(person_role_id, session=session)
+
+    async def is_person_role_referenced(
+        self,
+        person_role_id: UUID,
+        *,
+        session: AsyncSession | None = None,
+    ) -> bool:
+        """Return whether a document-person link references a Person Role."""
+        return await self._person_roles.is_referenced(person_role_id, session=session)
+
+    async def read_person_role_by_semantic_key(
+        self,
+        semantic_key: str,
+        *,
+        session: AsyncSession | None = None,
+    ) -> PersonRole:
+        """Resolve one application-defined built-in role."""
+        async with self._session_scope(session) as _session:
+            role = (await _session.exec(select(PersonRole).where(PersonRole.semantic_key == semantic_key))).first()
+        if role is None:
+            raise PersonRoleError(
+                f"Built-in Person Role {semantic_key!r} is unavailable",
+                category=ErrorCategory.NOT_FOUND,
+                suggestion="Recreate the built-in registry rows and retry.",
+            )
+        return role
+
+    async def list_document_people(
+        self,
+        *,
+        document_id: UUID | None = None,
+        person_id: UUID | None = None,
+        session: AsyncSession | None = None,
+    ) -> Sequence[DocumentPerson]:
+        async with self._session_scope(session) as _session:
+            query = select(DocumentPerson).options(
+                selectinload(DocumentPerson.document),
+                selectinload(DocumentPerson.person),
+                selectinload(DocumentPerson.role_ref),
+            )
+            if document_id is not None:
+                query = query.where(DocumentPerson.document_id == document_id)
+            if person_id is not None:
+                query = query.where(DocumentPerson.person_id == person_id)
+            return (await _session.exec(query)).all()
+
+    async def add_document_person_link(
+        self,
+        *,
+        document_id: UUID,
+        person_id: UUID,
+        role_id: UUID,
+        session: AsyncSession | None = None,
+    ) -> DocumentPerson:
+        async with self._session_scope(session) as _session:
+            await self._require_document(session=_session, document_id=document_id)
+            if await _session.get(Person, person_id) is None:
+                raise self._not_found(f"Person with id {person_id} not found")
+
+            await self._validate_role(session=_session, role_id=role_id, require_active=True)
+            link = DocumentPerson(document_id=document_id, person_id=person_id, role_id=role_id)
+            _session.add(link)
+            return await self._finalize_link(session=_session, caller_session=session, link=link)
+
+    async def set_document_person_role(
+        self,
+        *,
+        document_person_id: UUID,
+        role_id: UUID,
+        session: AsyncSession | None = None,
+    ) -> DocumentPerson:
+        async with self._session_scope(session) as _session:
+            link = await _session.get(DocumentPerson, document_person_id)
+            if link is None:
+                raise self._not_found(f"DocumentPerson with id {document_person_id} not found")
+            await self._validate_role(
+                session=_session,
+                role_id=role_id,
+                require_active=link.role_id != role_id,
+            )
+            link.role_id = role_id
+            return await self._finalize_link(session=_session, caller_session=session, link=link)
+
+    async def remove_document_person_link(
+        self,
+        *,
+        document_person_id: UUID,
+        session: AsyncSession | None = None,
+    ) -> None:
+        async with self._session_scope(session) as _session:
+            link = await _session.get(DocumentPerson, document_person_id)
+            if link is None:
+                raise self._not_found(f"DocumentPerson with id {document_person_id} not found")
+            await _session.delete(link)
+            await self._finalize(session=_session, caller_session=session)
+
+    async def sync_document_people(
+        self,
+        *,
+        document_id: UUID,
+        links: Sequence[DocumentPersonInput],
+        session: AsyncSession | None = None,
+    ) -> Sequence[DocumentPerson]:
+        """Synchronize one Document's complete Person link set."""
+        person_ids = [link.person_id for link in links]
+        if len(person_ids) != len(set(person_ids)):
+            raise PeopleError(
+                "A Person can be linked to a Document only once",
+                category=ErrorCategory.CONFLICT,
+                suggestion="Edit the existing Linked People row instead of adding another.",
+            )
+
+        async with self._session_scope(session) as _session:
+            await self._require_document(session=_session, document_id=document_id)
+            existing_links = (
+                await _session.exec(select(DocumentPerson).where(DocumentPerson.document_id == document_id))
+            ).all()
+            existing_by_person = {link.person_id: link for link in existing_links}
+            desired_by_person = {link.person_id: link for link in links}
+
+            roles: dict[UUID, PersonRole] = {}
+            for desired in links:
+                if await _session.get(Person, desired.person_id) is None:
+                    raise self._not_found(f"Person with id {desired.person_id} not found")
+                role = roles.get(desired.role_id)
+                if role is None:
+                    role = await self._validate_role(session=_session, role_id=desired.role_id)
+                    roles[desired.role_id] = role
+                current = existing_by_person.get(desired.person_id)
+                if (current is None or current.role_id != desired.role_id) and not role.is_active:
+                    raise PeopleError(
+                        f"Inactive Person Role {role.label!r} cannot be assigned",
+                        category=ErrorCategory.VALIDATION,
+                        suggestion="Select an active Person Role and retry.",
+                    )
+
+            for person_id, existing in existing_by_person.items():
+                if person_id not in desired_by_person:
+                    await _session.delete(existing)
+
+            synchronized: list[DocumentPerson] = []
+            for desired in links:
+                existing = existing_by_person.get(desired.person_id)
+                if existing is None:
+                    existing = DocumentPerson(
+                        document_id=document_id,
+                        person_id=desired.person_id,
+                        role_id=desired.role_id,
+                    )
+                    _session.add(existing)
+                elif existing.role_id != desired.role_id:
+                    existing.role_id = desired.role_id
+                synchronized.append(existing)
+
+            try:
+                await self._finalize(session=_session, caller_session=session, refresh=synchronized)
+            except IntegrityError as exc:
+                raise PeopleError(
+                    "A Person can be linked to a Document only once",
+                    category=ErrorCategory.CONFLICT,
+                    suggestion="Edit the existing Linked People row instead of adding another.",
+                ) from exc
+            return synchronized
+
+    async def _validate_role(
+        self,
+        *,
+        session: AsyncSession,
+        role_id: UUID,
+        require_active: bool = False,
+    ) -> PersonRole:
+        role = await session.get(PersonRole, role_id)
+        if role is None:
+            raise PeopleError(
+                f"Person role with id {role_id} not found",
+                category=ErrorCategory.VALIDATION,
+                suggestion="Select a valid relationship role and retry.",
+            )
+        if require_active and not role.is_active:
+            raise PeopleError(
+                f"Inactive Person Role {role.label!r} cannot be assigned",
+                category=ErrorCategory.VALIDATION,
+                suggestion="Select an active Person Role and retry.",
+            )
+        return role
+
+    async def _finalize_link(
+        self,
+        *,
+        session: AsyncSession,
+        caller_session: AsyncSession | None,
+        link: DocumentPerson,
+    ) -> DocumentPerson:
+        try:
+            await self._finalize(session=session, caller_session=caller_session, refresh=(link,))
+        except IntegrityError as exc:
+            raise PeopleError(
+                "This Person is already linked to the Document",
+                category=ErrorCategory.CONFLICT,
+                suggestion="Edit the existing relationship instead of adding another one.",
+            ) from exc
+        # Relationships load explicitly; the models declare lazy="raise".
+        await session.refresh(link, attribute_names=["document", "person", "role_ref"])
+        return link
+
+    async def _require_document(self, *, session: AsyncSession, document_id: UUID) -> None:
+        if await session.get(Document, document_id) is None:
+            raise self._not_found(f"Document with id {document_id} not found")
+
+    @staticmethod
+    def _family_search_conflict(family_search_id: str | None) -> PeopleError:
+        return PeopleError(
+            f"FamilySearch ID {family_search_id} is already assigned to another person",
+            category=ErrorCategory.CONFLICT,
+            suggestion="Open the existing person record or enter a different FamilySearch ID.",
+        )
+
+    @staticmethod
+    def _not_found(message: str) -> PeopleError:
+        return PeopleError(
+            message,
+            category=ErrorCategory.NOT_FOUND,
+            suggestion="Verify the requested Person or relationship id and retry.",
+        )

src/transcription/services/photos.py

@@ -0,0 +1,209 @@
+"""Photo service for Person and homepage image records."""
+
+from __future__ import annotations
+
+import asyncio
+import random
+from pathlib import Path
+from uuid import UUID
+from uuid import uuid4
+
+from sqlalchemy.ext.asyncio import async_sessionmaker
+from sqlmodel import select
+from sqlmodel.ext.asyncio.session import AsyncSession
+
+from ..config import Settings
+from ..db.models import Photo
+from ..errors import AppError
+from ..errors import ErrorCategory
+from .base import ServiceBase
+from .media_storage import persist_named_media
+
+PHOTO_EXTENSIONS = frozenset({".jpg", ".jpeg", ".png", ".gif", ".webp", ".bmp", ".tif", ".tiff"})
+
+
+class PhotoError(AppError):
+    """Raised when photo operations fail."""
+
+
+class PhotosService(ServiceBase):
+    """Manage homepage and Person photos."""
+
+    def __init__(
+        self,
+        session_factory: async_sessionmaker[AsyncSession] | None = None,
+        settings: Settings | None = None,
+    ) -> None:
+        super().__init__(session_factory, settings)
+
+    async def create_photo(
+        self,
+        *,
+        filename: str,
+        file_bytes: bytes,
+        person_id: UUID | None,
+        description: str | None = None,
+        is_primary: bool | None = None,
+        session: AsyncSession | None = None,
+    ) -> Photo:
+        if not file_bytes:
+            raise PhotoError(
+                "Photo content is empty",
+                category=ErrorCategory.VALIDATION,
+                suggestion="Select a non-empty image file and retry.",
+            )
+
+        suffix = Path(filename).suffix.lower()
+        if suffix not in PHOTO_EXTENSIONS:
+            raise PhotoError(
+                f"Unsupported photo format: {suffix or '<none>'}",
+                category=ErrorCategory.USER_INPUT,
+                suggestion="Use JPG, JPEG, PNG, GIF, WEBP, BMP, or TIFF image files.",
+            )
+
+        photo_id = uuid4()
+        async with self._session_scope(session) as _session:
+            existing = await self._list_owner_photos(session=_session, person_id=person_id)
+            should_be_primary = bool(is_primary) if is_primary is not None else len(existing) == 0
+            if should_be_primary:
+                await self._clear_owner_primary(session=_session, person_id=person_id)
+
+            stored_path = await persist_named_media(
+                root=self.settings.upload_dir,
+                namespace=Path("photos"),
+                filename=filename,
+                filename_stem=str(photo_id),
+                file_bytes=file_bytes,
+                error=PhotoError,
+                failure_message="Failed to persist photo media",
+                failure_suggestion="Check media directory permissions and available disk space, then retry.",
+                log_label="photo media",
+            )
+            relative_path = self._relative_upload_path(stored_path)
+
+            photo = Photo(
+                id=photo_id,
+                person_id=person_id,
+                path=relative_path,
+                description=(description or "").strip() or None,
+                is_primary=should_be_primary,
+            )
+            _session.add(photo)
+            await self._finalize(session=_session, caller_session=session, refresh=(photo,))
+            return photo
+
+    async def list_photos(
+        self,
+        *,
+        person_id: UUID | None,
+        session: AsyncSession | None = None,
+    ) -> list[Photo]:
+        async with self._session_scope(session) as _session:
+            photos = await self._list_owner_photos(session=_session, person_id=person_id)
+
+        primary = [photo for photo in photos if photo.is_primary]
+        non_primary = [photo for photo in photos if not photo.is_primary]
+        random.shuffle(non_primary)
+        return [*primary[:1], *non_primary]
+
+    async def set_primary(
+        self,
+        *,
+        photo_id: UUID,
+        session: AsyncSession | None = None,
+    ) -> Photo:
+        async with self._session_scope(session) as _session:
+            photo = await self._get_or_raise(
+                Photo,
+                photo_id,
+                session=_session,
+                error=PhotoError,
+                noun="Photo",
+                suggestion="Refresh and retry with a valid photo record.",
+            )
+            await self._clear_owner_primary(session=_session, person_id=photo.person_id)
+            photo.is_primary = True
+            await self._finalize(session=_session, caller_session=session, refresh=(photo,))
+            return photo
+
+    async def update_description(
+        self,
+        *,
+        photo_id: UUID,
+        description: str | None,
+        session: AsyncSession | None = None,
+    ) -> Photo:
+        async with self._session_scope(session) as _session:
+            photo = await self._get_or_raise(
+                Photo,
+                photo_id,
+                session=_session,
+                error=PhotoError,
+                noun="Photo",
+                suggestion="Refresh and retry with a valid photo record.",
+            )
+            photo.description = (description or "").strip() or None
+            await self._finalize(session=_session, caller_session=session, refresh=(photo,))
+            return photo
+
+    async def delete_photo(
+        self,
+        *,
+        photo_id: UUID,
+        session: AsyncSession | None = None,
+    ) -> None:
+        async with self._session_scope(session) as _session:
+            photo = await self._get_or_raise(
+                Photo,
+                photo_id,
+                session=_session,
+                error=PhotoError,
+                noun="Photo",
+                suggestion="Refresh and retry with a valid photo record.",
+            )
+            owner_person_id = photo.person_id
+            deleted_primary = photo.is_primary
+            media_path = self.settings.upload_dir / Path(photo.path)
+            await _session.delete(photo)
+
+            if deleted_primary:
+                replacement = await self._owner_oldest_photo(session=_session, person_id=owner_person_id)
+                if replacement is not None:
+                    replacement.is_primary = True
+
+            await self._finalize(session=_session, caller_session=session)
+
+        await asyncio.to_thread(media_path.unlink, missing_ok=True)
+
+    async def _list_owner_photos(self, *, session: AsyncSession, person_id: UUID | None) -> list[Photo]:
+        query = select(Photo)
+        if person_id is None:
+            query = query.where(Photo.person_id.is_(None))
+        else:
+            query = query.where(Photo.person_id == person_id)
+        query = query.order_by(Photo.created_at.asc(), Photo.id.asc())
+        return list((await session.exec(query)).all())
+
+    async def _owner_oldest_photo(self, *, session: AsyncSession, person_id: UUID | None) -> Photo | None:
+        query = select(Photo)
+        if person_id is None:
+            query = query.where(Photo.person_id.is_(None))
+        else:
+            query = query.where(Photo.person_id == person_id)
+        query = query.order_by(Photo.created_at.asc(), Photo.id.asc()).limit(1)
+        return (await session.exec(query)).first()
+
+    async def _clear_owner_primary(self, *, session: AsyncSession, person_id: UUID | None) -> None:
+        query = select(Photo).where(Photo.is_primary.is_(True))
+        if person_id is None:
+            query = query.where(Photo.person_id.is_(None))
+        else:
+            query = query.where(Photo.person_id == person_id)
+        for current in (await session.exec(query)).all():
+            current.is_primary = False
+
+    def _relative_upload_path(self, absolute_path: Path) -> str:
+        try:
+            return absolute_path.resolve().relative_to(self.settings.upload_dir.resolve()).as_posix()
+        except ValueError:
+            return absolute_path.name

src/transcription/services/prompts.py

@@ -0,0 +1,191 @@
+"""Constrained storage for mutable prompt Markdown artifacts."""
+
+from __future__ import annotations
+
+import os
+from dataclasses import dataclass
+from pathlib import Path
+from uuid import uuid4
+
+from ..config import Settings
+from ..config import get_settings
+from ..errors import AppError
+from ..errors import ErrorCategory
+
+PROMPT_EXTENSION = ".md"
+BACKUP_SUFFIX = ".bak"
+
+
+class PromptStoreError(AppError):
+    """Raised when prompt storage validation or persistence fails."""
+
+
+@dataclass(frozen=True, slots=True)
+class PromptSummary:
+    """Read model for one editable prompt artifact."""
+
+    name: str
+    is_default: bool
+    has_backup: bool
+
+
+class PromptStore:
+    """List, read, atomically update, and recover existing prompt files."""
+
+    def __init__(self, settings: Settings | None = None) -> None:
+        self.settings = settings or get_settings()
+
+    def list_prompts(self) -> tuple[PromptSummary, ...]:
+        """List editable direct-child Markdown prompts by filename."""
+        root = self._prompt_root()
+        try:
+            candidates = tuple(root.iterdir())
+        except OSError as exc:
+            raise self._filesystem_error("Prompt directory could not be read", exc) from exc
+
+        summaries: list[PromptSummary] = []
+        for candidate in candidates:
+            if candidate.suffix.lower() != PROMPT_EXTENSION or not candidate.is_file():
+                continue
+            resolved = candidate.resolve()
+            if resolved.parent != root:
+                continue
+            summaries.append(
+                PromptSummary(
+                    name=candidate.name,
+                    is_default=candidate.name == self.settings.default_prompt_name,
+                    has_backup=self._backup_path(candidate).is_file(),
+                )
+            )
+        return tuple(sorted(summaries, key=lambda item: item.name.casefold()))
+
+    def read_prompt(self, name: str) -> str:
+        """Read one existing UTF-8 prompt."""
+        path = self._resolve_existing_prompt(name)
+        return self._read_nonempty_text(path, description="Prompt")
+
+    def write_prompt(self, name: str, content: str) -> None:
+        """Atomically replace an existing prompt and retain one prior version."""
+        path = self._resolve_existing_prompt(name)
+        normalized_content = content.strip()
+        if not normalized_content:
+            raise PromptStoreError(
+                "Prompt content cannot be empty",
+                category=ErrorCategory.VALIDATION,
+                suggestion="Enter prompt text before saving.",
+            )
+        self._atomic_write(path=path, content=f"{normalized_content}\n", preserve_current=True)
+
+    def recover_prompt(self, name: str) -> None:
+        """Restore the sole previous-version backup as an explicit operation."""
+        path = self._resolve_existing_prompt(name)
+        backup_path = self._backup_path(path)
+        if not backup_path.is_file():
+            raise PromptStoreError(
+                f"No previous version is available for {path.name}",
+                category=ErrorCategory.NOT_FOUND,
+                suggestion="Save a prompt edit before attempting recovery.",
+            )
+        backup_content = self._read_nonempty_text(backup_path, description="Prompt backup")
+        self._atomic_write(path=path, content=backup_content, preserve_current=True)
+
+    def _prompt_root(self) -> Path:
+        try:
+            root = self.settings.prompt_dir.resolve()
+        except OSError as exc:
+            raise self._filesystem_error("Prompt directory could not be resolved", exc) from exc
+        if not root.is_dir():
+            raise PromptStoreError(
+                f"Prompt directory is unavailable: {root}",
+                category=ErrorCategory.INFRA_PERSISTENT,
+                suggestion="Restore the configured prompt directory and its permissions.",
+            )
+        return root
+
+    def _resolve_existing_prompt(self, name: str) -> Path:
+        normalized_name = name.strip()
+        # Path().name is platform-dependent: POSIX treats "\" as an ordinary filename
+        # character, so reject both separators explicitly to match config.PromptFilename.
+        if (
+            not normalized_name
+            or any(separator in normalized_name for separator in ("/", "\\"))
+            or Path(normalized_name).name != normalized_name
+            or Path(normalized_name).suffix.lower() != PROMPT_EXTENSION
+        ):
+            raise PromptStoreError(
+                "Prompt name must be a direct-child Markdown filename",
+                category=ErrorCategory.VALIDATION,
+                suggestion="Select an existing .md prompt from Settings.",
+            )
+        root = self._prompt_root()
+        try:
+            path = (root / normalized_name).resolve()
+        except OSError as exc:
+            raise self._filesystem_error("Prompt path could not be resolved", exc) from exc
+        if path.parent != root:
+            raise PromptStoreError(
+                "Prompt path must remain inside the configured prompt directory",
+                category=ErrorCategory.VALIDATION,
+                suggestion="Select an existing prompt from Settings.",
+            )
+        if not path.is_file():
+            raise PromptStoreError(
+                f"Prompt file not found: {normalized_name}",
+                category=ErrorCategory.NOT_FOUND,
+                suggestion="Refresh Settings and select an existing prompt.",
+            )
+        return path
+
+    def _read_nonempty_text(self, path: Path, *, description: str) -> str:
+        try:
+            content = path.read_text(encoding="utf-8")
+        except UnicodeError as exc:
+            raise PromptStoreError(
+                f"{description} is not valid UTF-8: {path.name}",
+                category=ErrorCategory.VALIDATION,
+                suggestion="Restore a valid UTF-8 Markdown prompt.",
+            ) from exc
+        except OSError as exc:
+            raise self._filesystem_error(f"{description} could not be read", exc) from exc
+        if not content.strip():
+            raise PromptStoreError(
+                f"{description} is empty: {path.name}",
+                category=ErrorCategory.VALIDATION,
+                suggestion="Restore non-empty prompt content.",
+            )
+        return content
+
+    def _atomic_write(self, *, path: Path, content: str, preserve_current: bool) -> None:
+        temporary_path = path.with_name(f".{path.name}.{uuid4().hex}.tmp")
+        backup_path = self._backup_path(path)
+        backup_temporary_path = backup_path.with_name(f".{backup_path.name}.{uuid4().hex}.tmp")
+        try:
+            self._write_synced(temporary_path, content.encode("utf-8"))
+            if preserve_current:
+                self._write_synced(backup_temporary_path, path.read_bytes())
+                backup_temporary_path.replace(backup_path)
+            temporary_path.replace(path)
+        except (OSError, UnicodeError) as exc:
+            raise self._filesystem_error(f"Prompt {path.name} could not be saved", exc) from exc
+        finally:
+            temporary_path.unlink(missing_ok=True)
+            backup_temporary_path.unlink(missing_ok=True)
+
+    @staticmethod
+    def _write_synced(path: Path, content: bytes) -> None:
+        with path.open("wb") as stream:
+            stream.write(content)
+            stream.flush()
+            os.fsync(stream.fileno())
+
+    @staticmethod
+    def _backup_path(path: Path) -> Path:
+        return path.with_name(f"{path.name}{BACKUP_SUFFIX}")
+
+    @staticmethod
+    def _filesystem_error(message: str, exc: Exception) -> PromptStoreError:
+        return PromptStoreError(
+            f"{message}: {exc}",
+            category=ErrorCategory.INFRA_PERSISTENT,
+            suggestion="Check prompt directory permissions and available disk space, then retry.",
+        )

src/transcription/services/quality.py

@@ -0,0 +1,90 @@
+"""Deterministic, provider-neutral transcription quality warnings."""
+
+from __future__ import annotations
+
+import re
+from enum import StrEnum
+
+from pydantic import BaseModel
+from pydantic import ConfigDict
+
+QUALITY_ANALYSIS_SCHEMA = "transcription.quality-warnings"
+QUALITY_ANALYSIS_VERSION = "1"
+QUALITY_ANALYSIS_PRODUCER = "transcription.quality"
+QUALITY_ANALYSIS_PRODUCER_VERSION = "1"
+
+_BODY_MARKER_RE = re.compile(
+    r"\[document body (?:handwritten|typewritten|typeset|mixed)\]",
+    flags=re.IGNORECASE,
+)
+_HANDWRITTEN_LINE_RE = re.compile(r"(?m)^\s*\[handwritten:\s*.+\]\s*$", flags=re.IGNORECASE)
+_HTML_ENTITY_RE = re.compile(r"&(?:#[0-9]{1,7}|#x[0-9a-f]{1,6}|[a-z][a-z0-9]{1,31});", flags=re.IGNORECASE)
+
+
+class QualityWarningCode(StrEnum):
+    """Stable identifiers for output warning rules."""
+
+    REPLACEMENT_CHARACTER = "replacement_character"
+    MULTIPLE_BODY_MARKERS = "multiple_body_markers"
+    REDUNDANT_HANDWRITING_WRAPPERS = "redundant_handwriting_wrappers"
+    UNRESOLVED_HTML_ENTITY = "unresolved_html_entity"
+
+
+class QualityWarning(BaseModel):
+    """One immutable warning produced without changing transcription text."""
+
+    model_config = ConfigDict(extra="forbid", frozen=True)
+
+    code: QualityWarningCode
+    detail: str
+
+
+def analyze_transcription_quality(text: str) -> tuple[QualityWarning, ...]:
+    """Return deterministic warnings in stable rule order."""
+    warnings: list[QualityWarning] = []
+    if "\ufffd" in text:
+        warnings.append(
+            QualityWarning(
+                code=QualityWarningCode.REPLACEMENT_CHARACTER,
+                detail="Transcription contains one or more Unicode replacement characters.",
+            )
+        )
+
+    body_markers = _BODY_MARKER_RE.findall(text)
+    if len(body_markers) > 1:
+        warnings.append(
+            QualityWarning(
+                code=QualityWarningCode.MULTIPLE_BODY_MARKERS,
+                detail=f"Transcription contains {len(body_markers)} document-body markers; exactly one is expected.",
+            )
+        )
+
+    if re.search(r"\[document body handwritten\]", text, flags=re.IGNORECASE):
+        wrappers = _HANDWRITTEN_LINE_RE.findall(text)
+        if len(wrappers) > 1:
+            warnings.append(
+                QualityWarning(
+                    code=QualityWarningCode.REDUNDANT_HANDWRITING_WRAPPERS,
+                    detail=(
+                        "A wholly handwritten document also uses repeated whole-line handwriting wrappers."
+                    ),
+                )
+            )
+
+    if _HTML_ENTITY_RE.search(text):
+        warnings.append(
+            QualityWarning(
+                code=QualityWarningCode.UNRESOLVED_HTML_ENTITY,
+                detail="Transcription contains a likely unresolved HTML entity.",
+            )
+        )
+    return tuple(warnings)
+
+
+def quality_warning_payload(warnings: tuple[QualityWarning, ...]) -> dict:
+    """Build the versioned JSON artifact payload."""
+    return {
+        "schema_name": QUALITY_ANALYSIS_SCHEMA,
+        "schema_version": QUALITY_ANALYSIS_VERSION,
+        "warnings": [warning.model_dump(mode="json") for warning in warnings],
+    }

src/transcription/services/registry.py

@@ -0,0 +1,266 @@
+"""Shared implementation for label-keyed registry tables.
+
+Document Types and Person Roles are the same shape: a UUID-identified row with a
+user-facing ``label``, a casefolded ``normalized_label`` uniqueness key, an
+``is_active`` flag, and an optional ``semantic_key`` marking built-in entries
+that may be deactivated but never deleted. This module owns that behavior once
+so the two registries cannot drift apart.
+"""
+
+from __future__ import annotations
+
+from abc import abstractmethod
+from collections.abc import Sequence
+from dataclasses import dataclass
+from typing import Any
+from typing import Protocol
+from uuid import UUID
+
+from sqlalchemy import func
+from sqlalchemy.exc import IntegrityError
+from sqlmodel import SQLModel
+from sqlmodel import col
+from sqlmodel import select
+from sqlmodel.ext.asyncio.session import AsyncSession
+
+from ..errors import AppError
+from ..errors import ErrorCategory
+from .base import ServiceBase
+
+
+class RegistryEntry(Protocol):
+    """Structural contract every registry table row satisfies.
+
+    Bounding ``RegistryService`` by this protocol rather than by bare ``SQLModel``
+    lets the shared implementation read ``id``/``label``/``normalized_label``/
+    ``is_active`` off the model class without suppressions.
+    """
+
+    id: UUID
+    label: str
+    normalized_label: str
+    is_active: bool
+
+    def __init__(self, /, **data: Any) -> None: ...
+
+
+@dataclass(frozen=True, slots=True)
+class RegistrySummary:
+    """Shared settings read model for label-keyed registries and usage counts."""
+
+    id: UUID
+    label: str
+    is_active: bool
+    is_built_in: bool
+    reference_count: int
+
+
+class RegistryService[ModelT: RegistryEntry](ServiceBase):
+    """Generic create/read/update/delete behavior for a registry table.
+
+    Subclasses declare the model, the error type, the user-facing noun, and the
+    reference query used to decide whether an entry may be deleted.
+    """
+
+    #: Registry table this service maintains.
+    model: type[ModelT]
+    #: Error raised for every failure mode of this registry.
+    error: type[AppError]
+    #: User-facing singular noun, e.g. ``"Document Type"``.
+    noun: str
+    #: Lowercase noun used inside remediation suggestions, e.g. ``"type"``.
+    short_noun: str
+    #: Subject that retains a referenced entry, e.g. ``"historical Documents"``.
+    referenced_retainer: str
+
+    @abstractmethod
+    def reference_model(self) -> type[SQLModel]:
+        """Return the table whose rows reference this registry."""
+
+    @abstractmethod
+    def reference_id_column(self) -> Any:
+        """Return the primary key column of the referencing table."""
+
+    @abstractmethod
+    def reference_key_column(self) -> Any:
+        """Return the foreign key column pointing at this registry.
+
+        Declared as methods rather than class attributes because a mapped
+        column stored on a plain class would be re-invoked as a descriptor.
+        """
+
+    #
+    # Message templates
+    #
+
+    def _not_found(self, entry_id: UUID) -> AppError:
+        return self.error(
+            f"{self.noun} with id {entry_id} not found",
+            category=ErrorCategory.NOT_FOUND,
+            suggestion=f"Refresh Settings and select an available {self.noun}.",
+        )
+
+    def _duplicate_label(self, label: str) -> AppError:
+        return self.error(
+            f"{self.noun} label {label!r} already exists",
+            category=ErrorCategory.CONFLICT,
+            suggestion=f"Choose a different label or edit the existing {self.short_noun}.",
+        )
+
+    def normalize_label(self, label: str) -> str:
+        """Strip a submitted label, rejecting blank input."""
+        normalized = label.strip()
+        if not normalized:
+            raise self.error(
+                f"{self.noun} label is required",
+                category=ErrorCategory.VALIDATION,
+                suggestion="Enter a user-facing label and retry.",
+            )
+        return normalized
+
+    def label_key(self, label: str) -> str:
+        """Return the casefolded uniqueness key for a submitted label."""
+        return self.normalize_label(label).casefold()
+
+    #
+    # Reads
+    #
+
+    async def list_entries(
+        self,
+        *,
+        active_only: bool = True,
+        session: AsyncSession | None = None,
+    ) -> Sequence[ModelT]:
+        """List registry entries alphabetically by normalized label."""
+        async with self._session_scope(session) as _session:
+            query = select(self.model)
+            if active_only:
+                query = query.where(col(self.model.is_active).is_(True))
+            query = query.order_by(col(self.model.normalized_label), col(self.model.id))
+            return (await _session.exec(query)).all()
+
+    async def list_entries_with_counts(
+        self,
+        *,
+        session: AsyncSession | None = None,
+    ) -> Sequence[tuple[ModelT, int]]:
+        """List every entry alphabetically with its current reference count."""
+        async with self._session_scope(session) as _session:
+            query = (
+                select(self.model, func.count(self.reference_id_column()))
+                .outerjoin(self.reference_model(), self.reference_key_column() == col(self.model.id))
+                .group_by(col(self.model.id))
+                .order_by(col(self.model.normalized_label), col(self.model.id))
+            )
+            rows = (await _session.exec(query)).all()
+            return [(entry, int(count)) for entry, count in rows]
+
+    async def read_entry(
+        self,
+        entry_id: UUID,
+        *,
+        session: AsyncSession | None = None,
+    ) -> ModelT:
+        """Read a registry entry by id."""
+        async with self._session_scope(session) as _session:
+            entry = await _session.get(self.model, entry_id)
+        if entry is None:
+            raise self._not_found(entry_id)
+        return entry
+
+    async def is_referenced(
+        self,
+        entry_id: UUID,
+        *,
+        session: AsyncSession | None = None,
+    ) -> bool:
+        """Return whether any row references the registry entry."""
+        async with self._session_scope(session) as _session:
+            entry = await _session.get(self.model, entry_id)
+            if entry is None:
+                raise self._not_found(entry_id)
+            return await self._is_referenced(session=_session, entry=entry)
+
+    async def _is_referenced(self, *, session: AsyncSession, entry: ModelT) -> bool:
+        query = select(self.reference_id_column()).where(self.reference_key_column() == entry.id)
+        return (await session.exec(query)).first() is not None
+
+    #
+    # Writes
+    #
+
+    async def create_entry(
+        self,
+        *,
+        label: str,
+        is_active: bool = True,
+        session: AsyncSession | None = None,
+    ) -> ModelT:
+        """Create a UUID-identified entry with a unique label."""
+        entry = self.model(
+            label=self.normalize_label(label),
+            normalized_label=self.label_key(label),
+            is_active=is_active,
+        )
+        async with self._session_scope(session) as _session:
+            _session.add(entry)
+            try:
+                await self._finalize(session=_session, caller_session=session, refresh=(entry,))
+            except IntegrityError as exc:
+                raise self._duplicate_label(entry.label) from exc
+            return entry
+
+    async def update_entry(
+        self,
+        entry_id: UUID,
+        *,
+        label: str,
+        is_active: bool,
+        session: AsyncSession | None = None,
+    ) -> ModelT:
+        """Update mutable fields without changing semantic identity."""
+        async with self._session_scope(session) as _session:
+            entry = await _session.get(self.model, entry_id)
+            if entry is None:
+                raise self._not_found(entry_id)
+            entry.label = self.normalize_label(label)
+            entry.normalized_label = self.label_key(label)
+            entry.is_active = is_active
+            try:
+                await self._finalize(session=_session, caller_session=session, refresh=(entry,))
+            except IntegrityError as exc:
+                raise self._duplicate_label(entry.label) from exc
+            return entry
+
+    async def delete_entry(
+        self,
+        entry_id: UUID,
+        *,
+        session: AsyncSession | None = None,
+    ) -> None:
+        """Delete an unreferenced, non-built-in entry without cascade behavior."""
+        async with self._session_scope(session) as _session:
+            entry = await _session.get(self.model, entry_id)
+            if entry is None:
+                raise self._not_found(entry_id)
+            if entry.semantic_key is not None:
+                raise self.error(
+                    f"Built-in {self.noun} {entry.label!r} cannot be deleted",
+                    category=ErrorCategory.CONFLICT,
+                    suggestion=(
+                        f"Deactivate the {self.short_noun} instead; "
+                        "its built-in meaning must remain available."
+                    ),
+                )
+            if await self._is_referenced(session=_session, entry=entry):
+                raise self.error(
+                    f"{self.noun} {entry.label!r} is referenced and cannot be deleted",
+                    category=ErrorCategory.CONFLICT,
+                    suggestion=(
+                        f"Deactivate the {self.short_noun} instead; "
+                        f"{self.referenced_retainer} will retain it."
+                    ),
+                )
+            await _session.delete(entry)
+            await self._finalize(session=_session, caller_session=session)