doc updates for pydantic

docs/Intent.md

@@ -1,4 +1,4 @@
-# Historical Document Transcription Design Intent
+# Historical Document Transcription
 I have several thousand pages of family history told through letters, postcards, books, and other documents that I want to transcribe to text. 
 
 ---
@@ -20,5 +20,4 @@ I have several thousand pages of family history told through letters, postcards,
 
 ## Methodology
 
-1. Follow current best practices per "A Guide to Documentary Editing" by Mary-Jo Kline. (See [Transcription Methodology](transcription_methodology.md))
-
+See [transcription_methodology.md](transcription_methodology.md) for details on the transcription methodology.

docs/architecture.md

@@ -0,0 +1,130 @@
+# Architecture (V1 Baseline)
+
+This document describes the current architecture of the personal historical-document transcription system and serves as the V1 technical baseline.
+
+## Architecture Objectives
+
+- preserve source material as transcribed text
+- keep operational complexity low for personal-scale deployment
+- support asynchronous processing without external queue infrastructure
+- maintain clear module boundaries for incremental extension
+
+## Runtime Topology
+
+V1 runtime is a modular monolith:
+
+- one FastAPI + NiceGUI application process
+- one in-process async worker loop
+- relational persistence via SQLModel (SQLite baseline)
+
+```mermaid
+flowchart LR
+    U[Browser User] --> A[FastAPI + NiceGUI App]
+    A --> W[In-process Worker]
+    A --> DB[(SQLite via SQLModel)]
+    W --> P[OpenRouter Provider]
+    W --> DB
+```
+
+## Lifecycle Ownership
+
+Application lifespan owns runtime setup/teardown:
+
+- configure logging
+- initialize and dispose DB runtime resources
+- optional schema bootstrap by environment policy
+- recover stale processing jobs
+- start/stop worker consumer lifespan
+
+## Layered Module Structure
+
+### Interface Layer
+
+- `src/transcription/ui/**` (NiceGUI pages/components)
+- `src/transcription/api/**` (FastAPI routes and error handlers)
+
+### Application/Workflow Layer
+
+- `src/transcription/services/workflows.py`
+- `src/transcription/worker.py`
+
+Responsibilities:
+
+- orchestration and status transitions
+- retry/timeout behavior
+- provider call coordination
+
+### Service Layer
+
+- `src/transcription/services/*.py`
+
+Responsibilities:
+
+- CRUD and transactional boundaries
+- domain-aligned persistence operations
+
+### Infrastructure Layer
+
+- `src/transcription/db/**` (runtime/session/bootstrap)
+- `src/transcription/providers/**` (OpenRouter adapter)
+
+## Processing Workflow
+
+1. User uploads a source file from the UI.
+2. App persists `Document`, `Job(queued)`, and `Source`.
+3. Worker claims next queued job and marks `processing`.
+4. Worker calls provider with prompt + source bytes.
+5. On success, app writes immutable `Job.text` and marks `transcribed`.
+6. On failure, app writes `Job.error_detail` and marks `failed`.
+7. UI exposes job detail, original transcription, and optional revision.
+
+## Domain Ownership Invariants
+
+- `Job.text` is immutable original provider output.
+- `Revision` is optional, user-authored, and linked to `Source`.
+- `Revision` does not overwrite original job transcription.
+- Status lifecycle is fixed to: `queued -> processing -> transcribed|failed`.
+
+## Data Model Summary
+
+- `Document` has many `Source` and many `Job`.
+- `Source` belongs to one `Document` and one `Job`.
+- `Source` has optional `Revision` (`0..1`) enforced by unique `revision.source_id`.
+
+## Simplicity Guardrails (V1)
+
+- no external queue/broker required
+- no search engine required
+- no distributed worker fleet required
+- keep provider integration behind adapter boundary
+
+## Extension Path
+
+### V1 (current)
+
+- SQLite baseline
+- OpenRouter provider
+- in-process worker
+- optional single revision workflow
+
+### V2 (planned)
+
+- PostgreSQL as relational baseline
+- optional MongoDB adjunct store for scoped use cases
+- migration-first schema evolution
+
+See [ver2/ver2.md](ver2/ver2.md) for roadmap details.
+
+## Test Strategy
+
+- unit tests for model/service behaviors
+- integration tests for upload/workflow reliability
+- UI integration tests for page/render contracts
+- external provider tests opt-in via marker/config
+
+## Related References
+
+- [index.md](index.md)
+- [requirements.md](requirements.md)
+- [schema.md](schema.md)
+- [error_handling.md](error_handling.md)

docs/architecture_v2.md

@@ -1,136 +0,0 @@
-# System Architecture (Version 2)
-
-This document describes the V2 production architecture of the personal historical-document transcription system.
-
-## Architecture Objectives
-
-* Preserve source material as immutable transcribed text alongside page-level spatial AI metadata.
-* Support batching multi-image and folder uploads cleanly into sequential pages (`page_number`).
-* Leverage asynchronous worker pools (`asyncio`) for parallel single-image API execution bounded by rate limiters (`asyncio.Semaphore`).
-* Migrate persistence to PostgreSQL using native `UUID`, `TIMESTAMPTZ`, and `JSONB` document storage.
-* Standardize all data validation, API parsing, and database models on **Pydantic V2**.
-* Support rich historical attribution (multi-author and multi-recipient relationships).
-
-## Runtime Topology
-
-The V2 runtime operates as an asynchronous Python application:
-
-* FastAPI + NiceGUI web application process.
-* In-process `asyncio` background task orchestrator for parallel API execution.
-* Relational persistence via PostgreSQL (using `asyncpg` or `psycopg3`).
-* Pydantic V2 validation layer wrapping API payloads and PostgreSQL `JSONB` schemas.
-
-```mermaid
-flowchart LR
-U[Browser User] --> A[FastAPI + NiceGUI App]
-A --> W[Asyncio Worker Engine]
-A --> DB[(PostgreSQL Database)]
-W --> P[Vision Provider APIs\nOpenAI / Claude]
-W --> DB
-```
-
-## Lifecycle Ownership
-
-Application lifespan owns runtime setup/teardown:
-
-* Initialize environment logging and Pydantic configuration.
-* Manage asynchronous PostgreSQL connection pools (`asyncpg` / `psycopg3`).
-* Execute database migrations and index initialization.
-* Recover stale processing jobs on startup.
-* Manage graceful shutdown of active `asyncio` worker pools.
-
-## Layered Module Structure
-
-### Interface Layer
-
-* `src/transcription/ui/**` (NiceGUI pages, multi-page renderers, person cards)
-* `src/transcription/api/**` (FastAPI routes and JSON error handlers)
-
-### Application & Async Worker Layer
-
-* `src/transcription/services/workflows.py`
-* `src/transcription/worker.py`
-
-Responsibilities:
-
-* Batch orchestration and status transitions (`queued` -> `processing` -> `completed` | `partial_success` | `failed`).
-* Parallel single-image API execution using `asyncio.gather` bounded by `asyncio.Semaphore`.
-* Pydantic schema parsing (`PageAIMetadata`) and validation prior to database storage.
-
-### Domain & Service Layer
-
-* `src/transcription/models/*.py` (Pydantic V2 schemas and entity definitions)
-* `src/transcription/services/*.py` (Transactional operations for `Document`, `Person`, `Source`, `Job`, and `JobSource`)
-
-### Infrastructure Layer
-
-* `src/transcription/db/**` (PostgreSQL connection pooling and raw parameterized SQL execution)
-* `src/transcription/providers/**` (OpenAI & Anthropic Vision SDK adapters)
-
-## Processing Workflow
-
-1. User uploads a folder or batch of images for a `Document`.
-2. System creates `Document`, `Job(status='queued')`, and ordered `Source` pages (`page_number = 1..N`).
-3. Worker claims job, sets `Job.status = 'processing'`, and spawns parallel `asyncio` tasks bounded by semaphore.
-4. Each task calls Vision API for a **single** `Source` image.
-5. On task completion:
-* Writes a `JobSource` record containing `status='transcribed'`, `raw_transcription`, `ai_metadata` (bounding boxes/confidence), and `raw_api_response`.
-* Caches active text to `Source.raw_transcription`.
-
-
-6. On page failure:
-* Writes `JobSource` record with `status='failed'` and `error_detail`.
-
-
-7. Once all page tasks resolve:
-* Marks `Job.status` as `completed` (100% success), `partial_success` (at least 1 success, 1 failure), or `failed` (all failed).
-
-
-
-## Domain Ownership & Invariants
-
-* **Immutable AI Outputs:** `source.raw_transcription` and `job_source.raw_transcription` store original, point-in-time machine output and are immutable.
-* **Inlined Revisions:** Human corrections occur on `source.revised_text`. UI renders `COALESCE(revised_text, raw_transcription)`.
-* **Sequential Integrity:** Multi-page documents are strictly ordered by `source.page_number ASC`.
-* **Page Execution Isolation:** A failure on one page image does not invalidate successful transcriptions on sister pages in the same batch job.
-
-## Data Model Summary
-
-* `Document` has many `Source` pages, many `Job` runs, and many `Person` records via `DocumentPerson` junction (`author` or `recipient`).
-* `Source` belongs to one `Document` and can be processed across many `JobSource` executions.
-* `Job` has many `JobSource` execution records.
-
-## Test Strategy
-
-* Unit tests for Pydantic V2 schemas, custom validators, and JSONB serialization.
-* Integration tests for async PostgreSQL connection handling and parameterized queries.
-* Async workflow tests using mock AI providers to verify `partial_success` and retry logic.
-* UI integration tests for multi-page rendering and person management.
-
----
-
-## Technology References
-
-- [FastAPI documentation](https://fastapi.tiangolo.com/)
-- [NiceGUI documentation](https://nicegui.io/documentation)
-- [PostgreSQL documentation](https://www.postgresql.org/docs/)
-- [Python asyncio](https://docs.python.org/3/library/asyncio.html#module-asyncio)
-- [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)
-- [Pydantic AI](https://pydantic.dev/docs/ai/overview/)
- 
-
-## Related Local References
-
-- [System Overview](index_v1.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- System Architecture (this document)
-- [System Requirements](requirements_v1.md)
-- [Data model](schema_v1.md)
-- [Error Handling Policy](error_handling_v1.md)
-- [Implementation Plan](implementation_plan_v1.md)
-
-
-
-
-

docs/archive/README.md

@@ -0,0 +1,23 @@
+# V2 Archive
+
+This folder preserves pre-V1-alignment versions of core documentation that included planned target-state architecture material.
+
+Archived snapshots:
+
+- `index.pre-v1-alignment.md`
+- `requirements.pre-v1-alignment.md`
+- `architecture.pre-v1-alignment.md`
+
+Purpose:
+
+- keep a durable reference for planned architecture language
+- reduce risk of losing useful V2 direction while V1 docs stay implementation-aligned
+
+Notes:
+
+- These files are historical snapshots, not the active V1 source of truth.
+- Active V1 docs remain at:
+  - `docs/index.md`
+  - `docs/requirements.md`
+  - `docs/architecture.md`
+- V2 planning should continue in `docs/ver2/ver2.md` and related V2 artifacts.

docs/archive/architecture.pre-v1-alignment.md

@@ -1,4 +1,4 @@
- # System Architecture (Version 1)
+ # Architecture
 
 This document describes the production architecture of the personal historical-document transcription system. The system is intentionally optimized for single-user operation, low operational overhead, and clean internal boundaries that support future growth without rewrites.
 
@@ -265,8 +265,6 @@ Control:
 
 - first-class human review and immutable revision history
 
----
-
 ## Technology References
 
 - [FastAPI documentation](https://fastapi.tiangolo.com/)
@@ -277,14 +275,7 @@ Control:
 
 ## Related Local References
 
-- [System Overview](index_v1.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- System Architecture (this document)
-- [System Requirements](requirements_v1.md)
-- [Data model](schema_v1.md)
-- [Error Handling Policy](error_handling_v1.md)
-- [Implementation Plan](implementation_plan_v1.md)
+- [System overview](index.md)
 
 ## Glossary
 

docs/archive/index.pre-v1-alignment.md

@@ -1,10 +1,10 @@
-## Document Transcription System Overview
+## Document Transcription System
 
 This project is a production application for transcribing and preserving historical family documents. It is intentionally designed for personal-scale use, with a simplicity-first architecture that is easy to operate and easy to extend.
 
 ## Start Here
 
-Read [architecture_v1.md](architecture_v1.md) first.
+Read [architecture.md](architecture.md) first.
 
 The architecture page is the primary technical reference and defines:
 
@@ -17,7 +17,7 @@ The architecture page is the primary technical reference and defines:
 
 At a high level, users upload images or PDFs as content sources for handwritten, typed, or typeset documents, run asynchronous transcription jobs, review optional revisions, and search across accepted text.
 
-### Core capabilities:
+Core capabilities:
 
 - document grouping with one or more content sources and metadata capture
 - asynchronous transcription with visible job status
@@ -38,18 +38,16 @@ The system runs with minimal operational overhead:
 
 This operating model keeps deployment and maintenance simple while preserving clean boundaries for future scale.
 
----
-
 ## Documentation Map
 
-- System Overview (this document)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v1.md)
-- [System Requirements](requirements_v1.md)
-- [Data model](schema_v1.md)
-- [Error Handling Policy](error_handling_v1.md)
-- [Implementation Plan](implementation_plan_v1.md)
+- Architecture and technical design: [architecture.md](architecture.md)
+- Runtime and deployment requirements: [requirements.md](requirements.md)
+- Error handling policy and operational guidance: [error_handling.md](error_handling.md)
+- Domain context and transcription policy: [intent.md](intent.md)
+- Transcription Methodology: [transcription_methodology.md](transcription_methodology.md)
+- Data model: [schema.md](schema.md)
+
+
 
 ## Glossary
 

docs/archive/requirements.pre-v1-alignment.md

@@ -1,6 +1,6 @@
 ## Document Transcription System Requirements
 
-This page captures a SysML v1.6-style requirements baseline for the production system described in [index_v1.md](index_v1.md). The model is represented as concise tables and traceability lists that preserve SysML-style IDs and relationship semantics.
+This page captures a SysML v1.6-style requirements baseline for the production system described in [index.md](index.md). The model is represented as concise tables and traceability lists that preserve SysML-style IDs and relationship semantics.
 
 ## Scope
 
@@ -77,19 +77,6 @@ This page captures a SysML v1.6-style requirements baseline for the production s
 - Analysis: evaluate asynchronous execution behavior and design sufficiency.
 - Test: automate behavioral checks through pytest suites and service-level tests.
 
----
-
-## Related Local References
-
-- [System Overview](index_v1.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v1.md)
-- System Requirements (this document)
-- [Data model](schema_v1.md)
-- [Error Handling Policy](error_handling_v1.md)
-- [Implementation Plan](implementation_plan_v1.md)
-
 ## Glossary
 
 - Document-oriented persistence: A storage approach that uses flexible document structures for variable data shapes.

docs/error_handling.md

@@ -1,4 +1,4 @@
-# Error Handling Policy
+# Error Handling
 
 This document defines the canonical error-handling policy for the document transcription system. It is the single source of truth for how errors are classified, surfaced to users, logged for diagnosis, and handled across UI, API, service, worker, and provider boundaries.
 
@@ -266,18 +266,12 @@ Change requirements:
 - preserve taxonomy stability; if changed, document migration impact
 - record noteworthy policy changes in project release notes or changelog
 
----
+## Related Pages
 
-## Related Local References
-
-- [System Overview](index_v1.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v1.md)
-- [System Requirements](requirements_v1.md)
-- [Data model](schema_v1.md)
-- Error Handling Policy (this document)
-- [Implementation Plan](implementation_plan_v1.md)
+- [System overview](index.md)
+- [Architecture](architecture.md)
+- [Requirements](requirements.md)
+- [Intent](intent.md)
 
 ## Glossary
 

docs/error_handling_v2.md

@@ -1,88 +0,0 @@
-# Error Handling Policy (Version 2)
-
-This document defines the canonical error-handling policy for the V2 document transcription system.
-
-## Error Handling Objectives
-
-* Make failures visible in clear, actionable language at both the document and individual page levels.
-* Support **isolated failure handling** in multi-image batches so single page errors do not crash an entire batch job.
-* Preserve diagnostic detail (Pydantic validation errors, raw provider responses) in PostgreSQL `JSONB` for fast troubleshooting.
-* Ensure consistent error envelope structure across API, UI, and async worker boundaries.
-
-## Scope And Authority
-
-Governs error behavior across NiceGUI pages, FastAPI routes, service orchestration, `asyncio` background tasks, PostgreSQL interactions, and AI provider adapters.
-
-## Error Taxonomy
-
-| Category | Definition | Retriable |
-| --- | --- | --- |
-| `validation_error` | Pydantic payload or parameter schema validation failure | no |
-| `user_input_error` | Unacceptable user file (unsupported image type, corrupt file) | no |
-| `not_found_error` | Requested resource (`Document`, `Source`, `Person`, `Job`) missing | no |
-| `conflict_error` | Operation violates state constraints (e.g., duplicate `document_person` role) | no |
-| `external_provider_error` | AI Provider API failure (rate limit, vision execution error) | yes |
-| `infrastructure_transient_error` | Temporary DB connection reset or HTTP timeout | yes |
-| `infrastructure_persistent_error` | Database down, missing API credentials, misconfiguration | no |
-| `internal_unexpected_error` | Uncaught Python exception or logic defect | no |
-
-## Async Batch & Page-Level Error Behavior
-
-In multi-image `asyncio` batch processing:
-
-1. **Page Isolation:** Exceptions caught during individual page calls are caught within the `asyncio` task wrapper.
-2. **Page Record Logging:** Page failure detail is written directly to `job_source.error_detail` and `job_source.status = 'failed'`.
-3. **Batch Aggregate State:**
-* If **all** page tasks succeed -> `job.status = 'completed'`.
-* If **some** page tasks fail -> `job.status = 'partial_success'`.
-* If **all** page tasks fail -> `job.status = 'failed'`.
-
-
-4. **Retry Strategy:** The UI exposes a "Retry Failed Pages" option for `partial_success` jobs, which spawns a new targeted `Job` containing *only* the `Source` IDs marked as `failed`.
-
-## API Error Response Contract
-
-API error responses return a structured JSON envelope:
-```json
-{
-"error_id": "err_uuid_12345",
-"category": "validation_error",
-"message": "The uploaded payload failed schema validation.",
-"suggestion": "Check file format and metadata fields, then try again.",
-"details": {
-"pydantic_errors": [...]
-},
-"timestamp": "2026-07-31T07:55:00Z"
-}
-```
-
-HTTP Status Mappings:
-
-* `validation_error`, `user_input_error` -> `400`
-* `not_found_error` -> `404`
-* `conflict_error` -> `409`
-* `external_provider_error` -> `502` / `503`
-* `infrastructure_transient_error` -> `503`
-* `infrastructure_persistent_error`, `internal_unexpected_error` -> `500`
-
----
-
-## Technology References
-
-- [FastAPI documentation](https://fastapi.tiangolo.com/)
-- [NiceGUI documentation](https://nicegui.io/documentation)
-- [PostgreSQL documentation](https://www.postgresql.org/docs/)
-- [Python asyncio](https://docs.python.org/3/library/asyncio.html#module-asyncio)
-- [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)
-- [Pydantic AI](https://pydantic.dev/docs/ai/overview/)
-
-## Related Local References
-
-- [System Overview](index_v2.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v2.md)
-- [System Requirements](requirements_v2.md)
-- [Data model](schema_v2.md)
-- Error Handling Policy (this document)
-- [Implementation Plan](implementation_plan_v2.md)

docs/implementation_plan_v2.md

@@ -1,248 +0,0 @@
-# Implementation Plan (Version 2)
-
-This plan defines the path from the V1 baseline to **Version 2 complete**, aligned to the updated multi-image and multi-person relational domain model:
-
-* `Document` acts as a logical parent container for physical artifacts, supporting multi-author and multi-recipient relationships via `DocumentPerson`.
-* `Source` represents an individual image page within a document, maintaining sequential order (`page_number`), cached active machine output (`raw_transcription`), and inline single user revisions (`revised_text`).
-* `Job` acts as an overarching batch orchestrator for multi-page async processing tasks.
-* `JobSource` records individual point-in-time API executions per image page, storing Pydantic-validated `ai_metadata` and raw REST envelopes (`raw_api_response`).
-* **Pydantic V2** acts as the single source of truth for runtime validation, API payload parsing, and PostgreSQL JSONB serialization.
-
-The objective is to complete the V2 scope with production readiness while keeping non-V2 enhancements out of active delivery.
-
----
-
-## V2 Completion Definition
-
-V2 is complete when all of the following are true:
-
-1. **Functional complete**
-* Multi-image and whole-folder uploads assign sequential page numbers to `Source` records under a single `Document`.
-* Batch jobs process pages concurrently using an `asyncio` worker pool with semaphore rate limiting.
-* Partial job failures resolve cleanly to `partial_success`, allowing single-page retries without re-running successful pages.
-* Multi-author and multi-recipient tagging is supported on `Document`.
-
-
-2. **Data-model complete**
-* SQLite is fully replaced with PostgreSQL (using `asyncpg` or `psycopg3`).
-* Pydantic V2 models validate all API payloads, database row mappings, and `JSONB` structures.
-
-
-3. **Operational complete**
-* Concurrency controls, worker pool metrics, and database connections operate safely under batch load.
-
-
-4. **Documentation complete**
-* `schema_v2.md`, `DDL_v2.sql`, Pydantic model contracts are updated and consistent.
-
-
-
----
-
-## Phase 1 — Data Contract Stabilization & Pydantic Baseline
-
-**Goal:** Lock the PostgreSQL schema, DDL, and Pydantic V2 models before refactoring service logic.
-
-### Tasks
-
-1. Finalize DDL for PostgreSQL native types (`UUID`, `TIMESTAMPTZ`, `JSONB`) and junction tables (`document_person`, `job_source`).
-2. Build core Pydantic V2 schemas (`Person`, `Document`, `Source`, `Job`, `JobSource`, `PageAIMetadata`).
-3. Confirm and document data invariants:
-* `source.raw_transcription` and `job_source.raw_transcription` are immutable machine outputs.
-* `source.revised_text` holds user edits. UI renders `COALESCE(revised_text, raw_transcription)`.
-* Page sequence is strictly ordered by `source.page_number ASC`.
-
-
-4. Freeze V2 job status values (`queued`, `processing`, `completed`, `partial_success`, `failed`) and page execution status values (`pending`, `transcribed`, `failed`).
-
-### Deliverables
-
-* Canonical `docs/schema_v2.md` and `docs/DDL_v2.sql`.
-* Centralized Pydantic validation suite in `models/schemas_v2.py`.
-
-### Exit Criteria
-
-* All database tables, relationships, and JSONB structures have corresponding Pydantic V2 models passing unit validation tests.
-
----
-
-## Phase 2 — Persistence Layer Transition (SQLite to PostgreSQL)
-
-**Goal:** Replace the SQLite storage layer with an asynchronous PostgreSQL driver (`asyncpg` or `psycopg3`).
-
-### Tasks
-
-1. Configure PostgreSQL database connection pooling and environment configuration.
-2. Refactor `services/store.py` / repository layers to execute parameterized async SQL queries (`$1`, `$2`).
-3. Implement JSONB serialization and deserialization helpers using Pydantic's `.model_dump_json()` and `.model_validate()`.
-4. Implement database bootstrap routines for PostgreSQL table creation and index initialization.
-
-### Deliverables
-
-* PostgreSQL-native database connection and query service modules.
-* Integration test suite confirming connection pooling and JSONB CRUD operations.
-
-### Exit Criteria
-
-* All database reads/writes run asynchronously against PostgreSQL with zero remaining SQLite driver dependencies.
-
----
-
-## Phase 3 — Service Layer & `asyncio` Engine Refactor
-
-**Goal:** Implement batch orchestration and parallel single-image API execution.
-
-### Tasks
-
-1. Refactor upload service to process folder/multi-image input:
-* Group files into a single `Document`.
-* Create ordered `Source` rows (`page_number = 1..N`).
-
-
-2. Refactor `services/workflows.py` with `asyncio` worker pools:
-* Use `asyncio.Semaphore` to enforce API provider rate limits.
-* Issue parallel single-image requests to Vision APIs (OpenAI/Claude).
-* Parse API responses directly into Pydantic models (`PageAIMetadata`).
-
-
-3. Update execution tracking:
-* Create a `JobSource` row per page call to record `raw_transcription`, `ai_metadata`, and `raw_api_response`.
-* Update active `source.raw_transcription` upon task completion.
-* Calculate aggregate batch status (`completed`, `partial_success`, `failed`) on the parent `Job`.
-
-
-4. Refactor `services/person.py` and `services/documents.py` to handle multi-person roles via `document_person`.
-
-### Deliverables
-
-* Asynchronous batch execution engine in `services/workflows.py`.
-* Service routines for multi-person tagging and page-level retries.
-
-### Exit Criteria
-
-* Executing a folder upload of 10+ images processes concurrently, populates page-level `JobSource` entries, and handles partial worker errors without crashing the batch.
-
----
-
-## Phase 4 — UI & API Contract Alignment
-
-**Goal:** Update API endpoints and frontend/UI views to render multi-page documents and person roles.
-
-### Tasks
-
-1. Update document and job API endpoints to accept batch file arrays and multi-person ID payloads.
-2. Update UI document views:
-* Render multi-page document transcriptions sequentially by `page_number`.
-* Display author and recipient chips/cards linked from `document_person`.
-
-
-3. Update job detail UI to show page-level execution statuses (`transcribed` vs. `failed`) and provide a "Retry Failed Pages" action for `partial_success` jobs.
-4. Align inline page editing controls to update `source.revised_text` and `source.date_revised`.
-
-### Deliverables
-
-* Refactored API routes and UI components supporting multi-page rendering and person management.
-
-### Exit Criteria
-
-* UI successfully displays multi-page document text, allows per-page human revisions, and shows author/recipient metadata.
-
----
-
-## Phase 5 — Test Suite Realignment & Concurrency Testing
-
-**Goal:** Ensure end-to-end system stability under concurrent async execution and load.
-
-### Tasks
-
-1. Write unit tests for Pydantic models, custom validators, and JSONB conversions.
-2. Write integration tests for async database operations:
-* CRUD for `Document`, `Person`, `DocumentPerson`, `Source`, `Job`, and `JobSource`.
-
-
-3. Write mock-backed async workflow tests:
-* Verify `asyncio.Semaphore` bounds concurrent tasks properly.
-* Validate state transition logic for `completed`, `partial_success`, and `failed` jobs.
-* Confirm retry routines process only targeted `JobSource` records marked as `failed`.
-
-
-4. Re-enable CI quality gates (linting, type checking with Pyright/mypy, pytest).
-
-### Deliverables
-
-* Passing asynchronous test suite covering core workflows, edge cases, and failure recoveries.
-
-### Exit Criteria
-
-* CI pipeline is green with comprehensive coverage across database operations, Pydantic models, and worker queues.
-
----
-
-## Phase 6 — Reliability, Operations, and Release Readiness
-
-**Goal:** Prepare V2 for production deployment and operator management.
-
-### Tasks
-
-1. Verify structured logging includes `job_id`, `document_id`, `source_id`, and `person_id`.
-2. Tune PostgreSQL connection pool limits and `asyncio` concurrency thresholds for production infrastructure.
-3. Update operational documentation:
-* Review and update `docs/schema_v2.md` as needed.
-* Create `docs/runbook_v2.md` detailing PostgreSQL maintenance, JSONB index management, and worker queue monitoring.
-* Create `docs/release_checklist_v2.md` for launch sign-off.
-
-
-
-### Deliverables
-
-* Updated project documentation and operational runbooks.
-* V2 release sign-off checklist.
-
-### Exit Criteria
-
-* All documentation reflects V2 architecture; launch checklist is fully verified.
-
----
-
-## Requirement Traceability Focus
-
-Maintain evidence against these V2 requirement groups:
-
-* **Batch & Multi-Image Pipeline:** Folder ingestion, page ordering, async worker execution.
-* **Database & Persistence:** PostgreSQL, native UUIDs, JSONB execution storage, `asyncpg` pooling.
-* **Validation & Schemas:** Pydantic V2 models for DB rows, API requests, and AI vision responses.
-* **Attribution & Metadata:** Multi-author and multi-recipient tagging, biographical entity management.
-* **Error Recovery:** Partial success states, page-level status flags, isolated retry execution.
-
----
-
-## Scope Discipline Rule (V2 Focus)
-
-* Only tasks required for V2 scope (PostgreSQL, Pydantic V2, folder/async processing, multi-person roles) enter this plan.
-* V3 candidate features (such as side-by-side multi-provider model output comparison) remain strictly in the future backlog.
-* Any schema adjustments during implementation require immediate updates to `DDL_v2.sql`, Pydantic models, and `schema_v2.md`.
-
----
-
-## Technology References
-
-- [FastAPI documentation](https://fastapi.tiangolo.com/)
-- [NiceGUI documentation](https://nicegui.io/documentation)
-- [PostgreSQL documentation](https://www.postgresql.org/docs/)
-- [Python asyncio](https://docs.python.org/3/library/asyncio.html#module-asyncio)
-- [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)
-- [Pydantic AI](https://pydantic.dev/docs/ai/overview/)
-
-## Related Local References
-
-- [System Overview](index_v2.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v2.md)
-- [System Requirements](requirements_v2.md)
-- [Data model](schema_v2.md)
-- [Error Handling Policy](error_handling_v2.md)
-- Implementation Plan (this document)
- 
-
-

docs/index.md

@@ -0,0 +1,57 @@
+## Document Transcription System (V1)
+
+This project is a personal-scale application for transcribing and preserving historical family documents.
+
+## Start Here
+
+Read [architecture.md](architecture.md) first.
+
+The architecture page is the primary technical reference for:
+
+- runtime topology and infrastructure assumptions
+- module boundaries and dependency flow
+- processing lifecycle and data ownership
+- test strategy and extension path
+
+## What The Application Does
+
+At a high level, users upload images/PDFs, jobs are processed asynchronously, and users review original transcriptions plus optional revisions.
+
+Core V1 capabilities:
+
+- upload supported source files (`.jpg`, `.jpeg`, `.png`, `.tif`, `.tiff`, `.pdf`)
+- asynchronous job processing with visible status (`queued`, `processing`, `transcribed`, `failed`)
+- immutable original transcription stored on `Job.text`
+- optional single user-authored revision per source (`0..1`)
+- prompt artifacts stored as Markdown files in `prompts/`
+
+## Current Operating Model (V1 Baseline)
+
+- application service: FastAPI + NiceGUI
+- persistence baseline: SQLModel with SQLite
+- worker: in-process async background loop
+- deployment baseline: lightweight Docker Compose app runtime
+
+> Planned persistence evolution (PostgreSQL and optional MongoDB) belongs to V2 planning and is tracked separately.
+
+## Documentation Map
+
+- Architecture and technical design: [architecture.md](architecture.md)
+- V1 runtime and requirement baseline: [requirements.md](requirements.md)
+- Data model and constraints: [schema.md](schema.md)
+- Error handling policy and operational guidance: [error_handling.md](error_handling.md)
+- V1 requirement evidence matrix: [traceability_v1.md](traceability_v1.md)
+- Operations runbook: [runbook.md](runbook.md)
+- V1 migration and rollback guidance: [migration_v1.md](migration_v1.md)
+- V1 release checklist: [release_checklist_v1.md](release_checklist_v1.md)
+- Domain context and transcription policy: [intent.md](intent.md)
+- Transcription methodology: [transcription_methodology.md](transcription_methodology.md)
+- V1 execution plan: [ver1/ver1.md](ver1/ver1.md)
+- V2 roadmap: [ver2/ver2.md](ver2/ver2.md)
+
+## Glossary
+
+- Prompt artifact: a Markdown file containing one transcription prompt.
+- Original transcription: immutable provider output stored on `Job.text`.
+- Revision: optional user-authored text linked to a `Source`.
+- System of record: the authoritative persistent store for canonical application data.

docs/index_v2.md

@@ -1,47 +0,0 @@
-# Document Transcription System Overview (Version 2)
-
-This project is a personal-scale application for transcribing, indexing, and preserving historical family documents, letters, postcards, and journals.
-
-## Start Here
-
-Read [architecture_v2.md](architecture_v2.md) first for technical overview and system design.
-
-## Core V2 Capabilities
-
-* **Folder & Multi-Image Ingestion:** Upload whole folders or image batches that map sequentially (`page_number`) under a single `Document`.
-* **Parallel Async AI Vision Engine:** Concurrently process single-page image transcriptions using Python `asyncio` bounded by rate limiters.
-* **Robust PostgreSQL Storage:** Relational storage for entities with native `UUID`, `TIMESTAMPTZ`, and `JSONB` for deep AI spatial metadata and raw envelopes.
-* **Pydantic V2 Validation:** End-to-end type safety, DB row mapping, and JSONB payload validation.
-* **Historical Person Management:** Track authors and recipients across documents with rich biographical entities (`Person`).
-* **Page-Level Execution Auditing & Revisions:** Store immutable point-in-time machine output per run while enabling inline human corrections (`revised_text`).
-* **Partial Failure Recovery:** Bounded batch execution that isolates single-page API errors (`partial_success`) for simple retries.
-
-## Technical Stack
-
-* **Application Web Framework:** FastAPI + NiceGUI
-* **Persistence Engine:** PostgreSQL 13+
-* **Data Validation & Schemas:** Pydantic V2
-* **Concurrency & Workers:** Python `asyncio` worker pool with `asyncio.Semaphore`
-* **Vision Providers:** OpenAI (GPT-4o) and Anthropic (Claude 3.5 Sonnet) via native SDKs
-
----
-
-## Technology References
-
-- [FastAPI documentation](https://fastapi.tiangolo.com/)
-- [NiceGUI documentation](https://nicegui.io/documentation)
-- [PostgreSQL documentation](https://www.postgresql.org/docs/)
-- [Python asyncio](https://docs.python.org/3/library/asyncio.html#module-asyncio)
-- [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)
-- [Pydantic AI](https://pydantic.dev/docs/ai/overview/)
- 
-## Documentation Index
-
-- System Overview (this document)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v2.md)
-- [System Requirements](requirements_v2.md)
-- [Data model](schema_v2.md)
-- [Error Handling Policy](error_handling_v2.md)
-- [Implementation Plan](implementation_plan_v2.md)

docs/requirements.md

@@ -0,0 +1,86 @@
+## Document Transcription System Requirements (V1 Baseline)
+
+This page captures the **Version 1 baseline requirements** for the currently implemented system. It is the source of truth for V1 acceptance and test traceability.
+
+Forward-looking architecture changes (for example PostgreSQL/Mongo adoption) are intentionally out of this document and should be tracked in a V2 planning/backlog artifact.
+
+## Scope
+
+- System of interest: a single Python application service (NiceGUI + FastAPI) with SQLModel persistence.
+- Runtime/persistence baseline: local-first execution using SQLite (default `sqlite:///./transcription.db`), with Docker Compose support.
+- Primary concern: end-to-end transcription lifecycle from upload through terminal state plus optional single revision editing.
+
+## Requirements Model (Concise Text Form)
+
+### Requirements
+
+| ID | Category | Requirement | Risk | Verify Method |
+| --- | --- | --- | --- | --- |
+| REQ-0 | System | Provide end-to-end document transcription with persistent, inspectable lifecycle state. | medium | demonstration |
+| REQ-1 | Functional | Allow users to upload supported image/PDF files as sources from the web UI. | low | test |
+| REQ-2 | Functional | Process uploads asynchronously and return either original transcription output or explicit failure. | high | test |
+| REQ-3 | Functional | Persist and expose job states: `queued`, `processing`, `transcribed`, `failed`. | high | inspection |
+| REQ-4 | Functional | Persist original provider output (`Job.text`) and failure detail (`Job.error_detail`) for each job. | medium | test |
+| REQ-5 | Interface | Expose API/UI views for status inspection and transcription reading. | medium | demonstration |
+| REQ-6 | Performance | Trigger background processing on upload to preserve UI responsiveness. | medium | analysis |
+| REQ-7 | Design Constraint | Keep lifespan-owned runtime resources (engine/session factory/worker resources) initialized and disposed at application boundaries. | medium | inspection |
+| REQ-8 | Design Constraint | Initialize configuration and logging once at startup through centralized mechanisms. | low | inspection |
+| REQ-9 | Design Constraint | Support containerized app runtime via Docker Compose using the same V1 persistence model. | medium | demonstration |
+| REQ-10 | Design Constraint | Keep schema bootstrap explicit and opt-in for production safety. | high | inspection |
+| REQ-11 | Design Constraint | Route persistence changes through service/workflow orchestration boundaries. | medium | inspection |
+| REQ-12 | Design Constraint | Store transcription prompts as individual Markdown artifacts for iterative refinement. | medium | inspection |
+| REQ-13 | Functional | Allow users to create/update one optional revision derived from the original job transcription and view/delete it from the job detail flow. | low | test |
+
+### Requirement Relationships
+
+- Contains: REQ-0 contains REQ-1 through REQ-13.
+- Derives: REQ-2 -> REQ-3, REQ-3 -> REQ-4.
+- Traces: REQ-5 -> REQ-3.
+- Refines: REQ-6 -> REQ-2.
+
+### Architecture Elements
+
+| Element | Type | Doc Reference |
+| --- | --- | --- |
+| UI | NiceGUI pages/components | `src/transcription/ui/pages`, `src/transcription/ui/components` |
+| API | FastAPI routes and handlers | `src/transcription/api`, `src/transcription/app.py` |
+| WORKER | Async queued-job processing workflow | `src/transcription/worker.py`, `src/transcription/services/workflows.py` |
+| DBREL | SQLModel relational persistence (SQLite in V1 baseline) | `src/transcription/models.py`, `src/transcription/db` |
+| SERVICES | Service-layer persistence orchestration | `src/transcription/services` |
+| OPS | Containerized runtime baseline | `docker-compose.yml`, `Dockerfile` |
+| PROMPTS | Transcription prompt artifacts | `prompts/` |
+| TESTS | Pytest verification suite | `tests/` |
+
+### Satisfaction Mapping
+
+- UI satisfies REQ-1, REQ-5, REQ-13.
+- API satisfies REQ-5.
+- WORKER satisfies REQ-2, REQ-6.
+- DBREL satisfies REQ-3, REQ-4, REQ-10, REQ-13.
+- SERVICES satisfies REQ-4, REQ-11.
+- OPS satisfies REQ-9.
+- PROMPTS satisfies REQ-12.
+
+### Verification Mapping
+
+- TESTS verifies REQ-1, REQ-2, REQ-3, REQ-4, REQ-5, REQ-10, REQ-11, REQ-12, REQ-13.
+
+## Requirement Notes
+
+- Requirement IDs (`REQ-*`) are stable references for planning, implementation, and traceability.
+- This document is intentionally **implementation-aligned** for V1 completion and release sign-off.
+- Planned storage evolution (PostgreSQL and optional MongoDB) is a **V2 concern** and should be tracked outside this V1 baseline.
+
+## Verification Intent
+
+- Demonstration: validate end-to-end behavior through operator-visible flows.
+- Inspection: verify architecture and startup/runtime policies in code and configuration.
+- Analysis: evaluate asynchronous execution behavior and design sufficiency.
+- Test: automate behavioral checks through pytest suites and service/UI integration tests.
+
+## Glossary
+
+- Original transcription: immutable provider output stored on `Job.text`.
+- Revision: optional user-authored editable text tied to a `Source` (`0..1` in V1).
+- Prompt artifact: a Markdown file containing instructions used for transcription.
+- System of record: the authoritative relational store for canonical V1 data.

docs/requirements_v2.md

@@ -1,42 +0,0 @@
-# Document Transcription System Requirements (Version 2)
-
-This document captures the **Version 2 baseline requirements** for the production implementation.
-
-## Requirements Model
-
-| ID | Category | Requirement | Verify Method |
-| --- | --- | --- | --- |
-| REQ-0 | System | Provide end-to-end multi-page document transcription with persistent, inspectable async job states. | demonstration |
-| REQ-1 | Functional | Allow users to upload folders or multi-image batches as sequential `Source` pages under a `Document`. | test |
-| REQ-2 | Functional | Process multi-page jobs asynchronously using an `asyncio` worker pool bounded by rate limits. | test |
-| REQ-3 | Functional | Persist page-level execution outputs (`raw_transcription`, `ai_metadata`, `raw_api_response`) on `JobSource`. | test |
-| REQ-4 | Functional | Support job states (`queued`, `processing`, `completed`, `partial_success`, `failed`) and page states (`pending`, `transcribed`, `failed`). | inspection |
-| REQ-5 | Functional | Allow users to manage historical `Person` records and link multiple authors/recipients to a `Document` via `DocumentPerson`. | test |
-| REQ-6 | Functional | Maintain immutable original machine output on `Source.raw_transcription` while permitting inline human edits on `Source.revised_text`. | test |
-| REQ-7 | Data Constraint | Store all persistent domain data in PostgreSQL using native `UUID`, `TIMESTAMPTZ`, and `JSONB` columns. | inspection |
-| REQ-8 | Data Constraint | Validate all API requests, database rows, and JSONB structures using Pydantic V2 schemas. | test |
-| REQ-9 | Interface | Render multi-page transcriptions sequentially by `page_number` in the web UI with author/recipient metadata. | demonstration |
-| REQ-10 | Operations | Allow operators to retry only failed pages for jobs in a `partial_success` state. | test |
-
-## Element Satisfaction Mapping
-
-* **UI (NiceGUI):** Satisfies REQ-1, REQ-5, REQ-6, REQ-9, REQ-10.
-* **API (FastAPI):** Satisfies REQ-1, REQ-4, REQ-5, REQ-8.
-* **WORKER (asyncio):** Satisfies REQ-2, REQ-3, REQ-4, REQ-10.
-* **PERSISTENCE (PostgreSQL):** Satisfies REQ-3, REQ-6, REQ-7.
-* **MODELS (Pydantic V2):** Satisfies REQ-8.
-
----
-
-## Related Local References
-
-- [System Overview](index_v2.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v2.md)
-- System Requirements (this document)
-- [Data model](schema_v2.md)
-- [Error Handling Policy](error_handling_v2.md)
-- [Implementation Plan](implementation_plan_v2.md)
-
-

docs/schema.md

@@ -79,17 +79,6 @@ erDiagram
 
 ---
 
-## Related Local References
-
-- [System Overview](index_v1.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v1.md)
-- [System Requirements](requirements_v1.md)
-- Data model (this document)
-- [Error Handling Policy](error_handling_v1.md)
-- [Implementation Plan](implementation_plan_v1.md)
-
 ## Glossary
 
 - **Document**: logical grouping for one or more transcribed sources.

docs/test_block-beta.md

@@ -1,92 +0,0 @@
-```mermaid
-block-beta
-  columns 3
-
-  %% UI Component Column
-  block:UI["UI COMPONENTS / WIREFRAME"]:1
-    columns 1
-    
-    block:HeaderUI["Header & Nav"]:1
-      columns 1
-      h_title["[Text] Document Name & Type"]
-      h_date["[Text] Date & Origin Location"]
-    end
-
-    block:EditorUI["Page Transcription Editor"]:1
-      columns 1
-      ed_img["[Image Viewer] Source Image"]
-      ed_page["[Badge] Page Number"]
-      ed_raw["[Read-Only] AI Raw Output"]
-      ed_rev["[Textarea] Human Revised Text"]
-    end
-
-    block:PeopleUI["Attribution Sidebar"]:1
-      columns 1
-      p_author["[List] Authors (Full Name)"]
-      p_recip["[List] Recipients (Full Name)"]
-      p_bio["[Card] Person Biography & Dates"]
-    end
-
-    block:JobUI["AI Processing Drawer"]:1
-      columns 1
-      j_status["[Badge] Job Status"]
-      j_model["[Text] Provider & Model"]
-      j_tokens["[JSON View] AI Token Usage"]
-    end
-  end
-
-  %% Directional Mapping / Connectors
-  block:FLOW["MAPPING / FLOW"]:1
-    columns 1
-    f1["Reads / Updates -->"]
-    f2["Renders Active Page -->"]
-    f3["Joins via Role -->"]
-    f4["Executes & Logs -->"]
-  end
-
-  %% Postgres Schema Column
-  block:DB["POSTGRES SQL SCHEMA"]:1
-    columns 1
-
-    block:DocTbl["Table: document"]:1
-      columns 1
-      d_id["id : UUID (PK)"]
-      d_name["name : TEXT"]
-      d_type["document_type : TEXT"]
-      d_date["document_date : DATE"]
-    end
-
-    block:SrcTbl["Table: source"]:1
-      columns 1
-      s_id["id : UUID (PK)"]
-      s_page["page_number : INT"]
-      s_path["file_path : TEXT"]
-      s_raw["raw_transcription : TEXT"]
-      s_rev["revised_text : TEXT"]
-    end
-
-    block:PersonTbl["Table: person & document_person"]:1
-      columns 1
-      p_id["id : UUID (PK)"]
-      p_name["full_name : TEXT"]
-      p_role["role : 'author' | 'recipient'"]
-    end
-
-    block:JobTbl["Table: job & job_source"]:1
-      columns 1
-      j_id["id : UUID (PK)"]
-      j_stat["status : VARCHAR"]
-      j_prov["provider / model : TEXT"]
-      j_meta["ai_metadata : JSONB"]
-    end
-  end
-
-  %% Connections
-  HeaderUI --> DocTbl
-  ed_img --> s_path
-  ed_page --> s_page
-  ed_raw --> s_raw
-  ed_rev --> s_rev
-  PeopleUI --> PersonTbl
-  JobUI --> JobTbl
-```

docs/test_flowchart.md

@@ -1,53 +0,0 @@
-```mermaid
-flowchart LR
-    subgraph UI["UI Components / Wireframe"]
-        direction TB
-        subgraph HeaderUI["Header & Nav"]
-            h_title["[Text] Document Name & Type"]
-            h_date["[Text] Date & Origin Location"]
-        end
-        subgraph EditorUI["Page Transcription Editor"]
-            ed_img["[Image Viewer] Source Image"]
-            ed_page["[Badge] Page Number"]
-            ed_raw["[Read-Only] AI Raw Output"]
-            ed_rev["[Textarea] Human Revised Text"]
-        end
-        subgraph PeopleUI["Attribution Sidebar"]
-            p_author["[List] Authors / Recipients"]
-        end
-        subgraph JobUI["AI Processing Drawer"]
-            j_status["[Badge] Job Status"]
-        end
-    end
-
-    subgraph DB["Postgres SQL Schema"]
-        direction TB
-        subgraph DocTbl["Table: document"]
-            d_name["name : TEXT"]
-            d_type["document_type : TEXT"]
-        end
-        subgraph SrcTbl["Table: source"]
-            s_path["file_path : TEXT"]
-            s_page["page_number : INT"]
-            s_raw["raw_transcription : TEXT"]
-            s_rev["revised_text : TEXT"]
-        end
-        subgraph PersonTbl["Table: person & document_person"]
-            p_name["full_name : TEXT"]
-            p_role["role : author | recipient"]
-        end
-        subgraph JobTbl["Table: job & job_source"]
-            j_stat["status : VARCHAR"]
-            j_meta["ai_metadata : JSONB"]
-        end
-    end
-
-    %% Mappings
-    HeaderUI --> DocTbl
-    ed_img --> s_path
-    ed_page --> s_page
-    ed_raw --> s_raw
-    ed_rev --> s_rev
-    PeopleUI --> PersonTbl
-    JobUI --> JobTbl
-```

docs/ver1/migration_v1.md

@@ -0,0 +1,92 @@
+# V1 Data Migration and Recovery Guidance
+
+This document defines migration/backfill and rollback guidance for the V1 SQLite baseline.
+
+## Purpose
+
+- provide safe procedures for local schema evolution and recovery
+- reduce data-loss risk during version upgrades
+- establish repeatable pre-change and post-change checks
+
+## Current Baseline
+
+- canonical relational store: SQLite
+- default DB path: `./transcription.db`
+- schema bootstrap may apply compatibility updates for dev/test scenarios
+
+## Pre-Change Checklist
+
+Before changing runtime version or schema behavior:
+
+1. Stop the app process.
+2. Create a timestamped DB backup copy.
+3. Capture current app commit/version.
+4. Export a quick status inventory:
+   - job counts by status
+   - total documents/sources/revisions
+5. Ensure sufficient disk space.
+
+## Backup Procedure (SQLite)
+
+Minimum procedure:
+
+1. Stop app.
+2. Copy DB file to a safe location with timestamp.
+3. Store backup path in release notes or change log.
+
+## Upgrade Procedure (V1)
+
+1. Perform pre-change checklist.
+2. Deploy updated app version.
+3. Start app and observe startup logs.
+4. Verify schema bootstrap completes (if enabled).
+5. Run smoke flow:
+   - upload valid file
+   - observe terminal status
+   - open job detail
+
+## Backfill Guidance
+
+V1 backfill is limited and conservative:
+
+- for records missing newly introduced non-null defaults, use explicit one-time SQL updates only after backup
+- avoid destructive rewrites of `Job.text` or `Revision.text`
+- never backfill by overwriting original immutable transcription output
+
+## Rollback Procedure
+
+If upgrade fails or causes data inconsistency:
+
+1. Stop app.
+2. Restore prior DB backup file.
+3. Revert app version to last known-good commit.
+4. Restart app.
+5. Run smoke flow and confirm stability.
+
+## Recovery Scenarios
+
+### Stale processing jobs after crash/restart
+
+- restart app and allow stale-job recovery to re-queue timed-out `processing` jobs
+- monitor for terminal progression
+
+### Schema mismatch symptoms
+
+- errors during startup or writes indicating missing columns/indexes
+- rollback to last good DB + app version
+- reattempt with documented upgrade path
+
+## Validation Evidence
+
+For each upgrade rehearsal, capture:
+
+- backup filename/path
+- pre and post job status counts
+- smoke test result
+- rollback rehearsal result (recommended)
+
+## Operational Constraints
+
+- treat DB backups as required before non-trivial upgrades
+- do not perform in-place DB edits while app is running
+- do not skip post-upgrade smoke validation

docs/ver1/release_checklist_v1.md

@@ -18,8 +18,8 @@ Use this checklist before declaring V1 operationally complete.
 
 ## C) Operational Readiness
 
-- [ ] `runbook_v1.md` reviewed and current.
-- [ ] `migration_v1.md` reviewed and current.
+- [ ] `docs/runbook.md` reviewed and current.
+- [ ] `docs/migration_v1.md` reviewed and current.
 - [ ] Backup and rollback procedures tested at least once.
 - [ ] Incident escalation packet template is known to operators.
 
@@ -28,14 +28,14 @@ Use this checklist before declaring V1 operationally complete.
 - [ ] Lint/type checks pass.
 - [ ] `pytest -m "not external" -q` passes.
 - [ ] Targeted external/provider checks executed (if credentials available).
-- [ ] Release evidence recorded in `release_evidence_v1.md`.
+- [ ] Release evidence recorded in `docs/release_evidence_v1.md`.
 
 ## E) Traceability and Documentation
 
-- [ ] `requirements_v1.md` aligns with implemented V1 behavior.
-- [ ] `architecture_v1.md`, `schema_v1.md`, and `error_handling_v1.md` are consistent.
-- [ ] `traceability_v1.md` is updated with current implementation and test evidence.
-- [ ] `implementation_plan_v1.md` phase status updated with evidence references.
+- [ ] `docs/requirements.md` aligns with implemented V1 behavior.
+- [ ] `docs/architecture.md`, `docs/schema.md`, and `docs/error_handling.md` are consistent.
+- [ ] `docs/traceability_v1.md` is updated with current implementation and test evidence.
+- [ ] `docs/ver1/ver1.md` phase status updated with evidence references.
 - [ ] REQ traceability evidence links recorded (tests/runbook/checks).
 
 ## Release Sign-Off

docs/ver1/traceability_v1.md

@@ -21,7 +21,7 @@ Status values:
 | REQ-6 | done | Background processing trigger/worker notifier and non-blocking workflow in `src/transcription/ui/components/upload.py`, `src/transcription/worker.py` | `tests/test_app.py`, `tests/services/test_workflows_reliability.py` |
 | REQ-7 | done | Lifespan-owned runtime resources in `src/transcription/app.py`, `src/transcription/db/runtime.py` | `tests/test_app.py`, `tests/test_db.py` |
 | REQ-8 | done | Centralized settings/logging initialization in `src/transcription/config.py`, `src/transcription/app.py` | `tests/test_config.py`, `tests/test_app.py` |
-| REQ-9 | done | Containerized runtime baseline in `docker-compose.yml`, `Dockerfile` | `release_checklist_v1.md` (Ops checklist), manual demonstration step |
+| REQ-9 | done | Containerized runtime baseline in `docker-compose.yml`, `Dockerfile` | `docs/release_checklist_v1.md` (Ops checklist), manual demonstration step |
 | REQ-10 | done | Explicit schema bootstrap policy + runtime controls in `src/transcription/config.py`, `src/transcription/app.py`, `src/transcription/db/operations.py` | `tests/test_db.py`, `tests/test_config.py` |
 | REQ-11 | done | Service/workflow persistence boundaries in `src/transcription/services/*.py`, `src/transcription/services/workflows.py` | `tests/services/test_job_service.py`, `tests/services/test_transcription_service.py` |
 | REQ-12 | done | Prompt artifacts in `prompts/` and loading/validation in `src/transcription/services/transcription.py` | `tests/test_prompts.py` |
@@ -29,9 +29,9 @@ Status values:
 
 ## Operational Evidence (Step 3 Artifacts)
 
-- Runbook: `runbook_v1.md`
-- Migration/backfill/rollback guidance: `migration_v1.md`
-- Release readiness checklist: `release_checklist_v1.md`
+- Runbook: `docs/runbook.md`
+- Migration/backfill/rollback guidance: `docs/migration_v1.md`
+- Release readiness checklist: `docs/release_checklist_v1.md`
 
 ## Verification Cadence
 

docs/ver1/ver1.md

@@ -44,7 +44,7 @@ V1 is complete when all of the following are true:
 4. Freeze V1 status lifecycle to current implementation (`queued`, `processing`, `transcribed`, `failed`).
 
 ### Deliverables
-- Updated `schema_v1.md` and `requirements.md` traceability alignment.
+- Updated `docs/schema.md` and `docs/requirements.md` traceability alignment.
 - Explicit V1 data invariants section in architecture docs.
 
 ### Exit Criteria
@@ -150,8 +150,9 @@ V1 is complete when all of the following are true:
 
 ### Deliverables
 - V1 release checklist and acceptance evidence.
-- `runbook_v1.md` for incident response and operator workflows.
-- `release_checklist_v1.md` for release sign-off.
+- `docs/runbook.md` for incident response and operator workflows.
+- `docs/migration_v1.md` for V1 migration/backfill/rollback guidance.
+- `docs/release_checklist_v1.md` for release sign-off.
 
 ### Exit Criteria
 - Stakeholder sign-off and launch readiness achieved.
@@ -186,18 +187,4 @@ A lightweight traceability table should be maintained with:
 
 - Only work required to satisfy V1 requirements enters this plan.
 - Nice-to-have enhancements are captured in a separate backlog document.
-- Schema or contract changes after Phase 1 require explicit approval and traceability impact review.
-
----
-
-## Related Local References
-
-- [System Overview](index_v1.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v1.md)
-- [System Requirements](requirements_v1.md)
-- [Data model](schema_v1.md)
-- [Error Handling Policy](error_handling_v1.md)
-- Implementation Plan (this document)
-
+- Schema or contract changes after Phase 1 require explicit approval and traceability impact review.

docs/ver2/V2 Context for DB Updates.md

@@ -0,0 +1,35 @@
+# AI Coding Assistant Project Briefing & Context
+
+## Project Mission
+This application is a family history archival and transcription platform. Its primary goal is to accept scanned document images (letters, postcards, logbooks, diaries), execute OCR and structured transcription via AI vision models (OpenAI GPT-4o, Anthropic Claude 3.5 Sonnet), and manage historical metadata (authors, recipients, dates, and locations).
+
+---
+
+## Technical Stack & Architecture
+* **Database:** PostgreSQL 13+ with native `UUID` (`gen_random_uuid()`) and `JSONB` columns.
+* **Backend Runtime / Concurrency:** Python utilizing `asyncio` for concurrent HTTP API calls to AI providers, with strict rate-limiting via `asyncio.Semaphore`.
+* **Validation & Types:** Python with **Pydantic** model definitions. Incoming AI responses must be parsed and validated with Pydantic models *before* database insertion.
+* **ORM / Database Access:** SQLModel and SQLAlchemy, using parameterized statements and PostgreSQL-native types.
+
+---
+
+## Core System Directives for AI Code Generation
+
+### 1. Data Immutability vs. Human Corrections
+* `job_source.raw_transcription` and `source.raw_transcription` represent original, point-in-time machine outputs and are **immutable**.
+* Human corrections occur on `source.revised_text`.
+* When fetching text for the UI, always display `COALESCE(source.revised_text, source.raw_transcription)`.
+
+### 2. Async Execution & Batching Rules
+* A `job` represents an overarching execution run for a folder/group of images belonging to a single `document`.
+* Images are submitted to AI APIs **one at a time in rapid succession** using `asyncio` worker pools.
+* Each single-image API call populates a row in `job_source` with its own `status`, `raw_transcription`, `ai_metadata`, and `raw_api_response`.
+* If 9 of 10 pages succeed and 1 fails, `job_source.status` for the failed image becomes `'failed'`, while `job.status` becomes `'partial_success'`. Do not mark the entire batch as failed if partial results exist.
+
+### 3. Entity Relationships
+* **Authors/Recipients:** A `document` can have multiple authors and recipients. Do NOT put direct `author_id` foreign keys on `document`. Query authors/recipients via `document_person` where `role = 'author'` or `role = 'recipient'`.
+* **Page Ordering:** Multi-page documents must always be queried using `ORDER BY page_number ASC`.
+
+### 4. Database Mutations
+* Always use parameterized SQL queries (`$1`, `$2`) to prevent SQL injection.
+* Store datetimes using UTC ISO 8601 strings or native PostgreSQL `TIMESTAMPTZ`.

docs/ver2/V2 DB Schema.md

@@ -1,4 +1,4 @@
-# Database Schema (Version 2)
+# Database Schema (V2 Architecture)
 
 This document describes the PostgreSQL relational schema for the transcription platform. It incorporates multi-image batch orchestration via `asyncio`, page-level execution tracking, many-to-many author/recipient attribution, and JSONB document storage for AI vision outputs.
 
@@ -119,18 +119,4 @@ erDiagram
 ### Attribution & Person Roles
 
 * Multi-Person Roles: Documents support zero, one, or many authors and recipients linked via document_person.
-* Role Uniqueness: (document_id, person_id, role) must be unique to prevent duplicate role tagging.
-
----
-
-## Related Local References
-
-- [System Overview](index_v2.md)
-- [System Design Intent](intent.md)
-- [Transcription Methodology](transcription_methodology.md)
-- [System Architecture](architecture_v2.md)
-- [System Requirements](requirements_v2.md)
-- Data model (this document)
-- [Error Handling Policy](error_handling_v2.md)
-- [Implementation Plan](implementation_plan_v2.md)
-
+* Role Uniqueness: (document_id, person_id, role) must be unique to prevent duplicate role tagging.

docs/ver2/V2 PostgreSQL DDL Specification.md

@@ -1,4 +1,4 @@
-## PostgreSQL DDL Specification (Version 2)
+## PostgreSQL DDL Specification
 
 ```sql
 -- Enable pgcrypto for UUID generation if on PostgreSQL < 13

docs/ver2/V2 Python Pydantic Models.md

@@ -0,0 +1,408 @@
+# SQLModel Table Models
+
+Each V2 table is represented by one `SQLModel` class. Because `SQLModel` is built on Pydantic and SQLAlchemy, these classes provide application validation and PostgreSQL mappings without parallel row and create models.
+
+Database-generated UUIDs and timestamps are `None` until PostgreSQL supplies their values during insert. The database columns remain non-nullable. `Person.metadata_` maps to the `metadata` column because `metadata` is reserved by SQLAlchemy's declarative API.
+
+```python
+from datetime import date
+from datetime import datetime
+from enum import StrEnum
+from uuid import UUID
+
+from pydantic import JsonValue
+from sqlalchemy import Column
+from sqlalchemy import Date
+from sqlalchemy import DateTime
+from sqlalchemy import ForeignKey
+from sqlalchemy import Index
+from sqlalchemy import Integer
+from sqlalchemy import String
+from sqlalchemy import Text
+from sqlalchemy import UniqueConstraint
+from sqlalchemy import text
+from sqlalchemy.dialects.postgresql import JSONB
+from sqlalchemy.dialects.postgresql import UUID as PostgreSQLUUID
+from sqlmodel import Field
+from sqlmodel import Relationship
+from sqlmodel import SQLModel
+
+
+class PersonRole(StrEnum):
+    AUTHOR = "author"
+    RECIPIENT = "recipient"
+
+
+class JobStatus(StrEnum):
+    QUEUED = "queued"
+    PROCESSING = "processing"
+    COMPLETED = "completed"
+    PARTIAL_SUCCESS = "partial_success"
+    FAILED = "failed"
+
+
+class JobSourceStatus(StrEnum):
+    PENDING = "pending"
+    TRANSCRIBED = "transcribed"
+    FAILED = "failed"
+
+
+class Person(SQLModel, table=True):
+    __tablename__ = "person"
+    __table_args__ = (Index("idx_person_full_name", "full_name"),)
+
+    id: UUID | None = Field(
+        default=None,
+        sa_column=Column(
+            PostgreSQLUUID(as_uuid=True),
+            primary_key=True,
+            server_default=text("gen_random_uuid()"),
+        ),
+    )
+    full_name: str = Field(sa_column=Column(Text, nullable=False))
+    display_name: str | None = Field(default=None, sa_column=Column(Text))
+    maiden_name: str | None = Field(default=None, sa_column=Column(Text))
+    birth_date: date | None = Field(default=None, sa_column=Column(Date))
+    birth_date_raw: str | None = Field(default=None, sa_column=Column(Text))
+    birth_place: str | None = Field(default=None, sa_column=Column(Text))
+    death_date: date | None = Field(default=None, sa_column=Column(Date))
+    death_date_raw: str | None = Field(default=None, sa_column=Column(Text))
+    death_place: str | None = Field(default=None, sa_column=Column(Text))
+    biography: str | None = Field(default=None, sa_column=Column(Text))
+    portrait_path: str | None = Field(default=None, sa_column=Column(Text))
+    metadata_: JsonValue | None = Field(
+        default_factory=dict,
+        sa_column=Column(
+            "metadata",
+            JSONB,
+            server_default=text("'{}'::jsonb"),
+        ),
+    )
+    created_at: datetime | None = Field(
+        default=None,
+        sa_column=Column(
+            DateTime(timezone=True),
+            nullable=False,
+            server_default=text("now()"),
+        ),
+    )
+    updated_at: datetime | None = Field(
+        default=None,
+        sa_column=Column(
+            DateTime(timezone=True),
+            nullable=False,
+            server_default=text("now()"),
+        ),
+    )
+
+    document_people: list["DocumentPerson"] = Relationship(
+        back_populates="person",
+        sa_relationship_kwargs={"lazy": "raise", "passive_deletes": True},
+    )
+
+
+class Document(SQLModel, table=True):
+    __tablename__ = "document"
+    __table_args__ = (Index("idx_document_date", "document_date"),)
+
+    id: UUID | None = Field(
+        default=None,
+        sa_column=Column(
+            PostgreSQLUUID(as_uuid=True),
+            primary_key=True,
+            server_default=text("gen_random_uuid()"),
+        ),
+    )
+    name: str = Field(sa_column=Column(Text, nullable=False))
+    document_type: str | None = Field(default=None, sa_column=Column(Text))
+    document_date: date | None = Field(default=None, sa_column=Column(Date))
+    document_date_raw: str | None = Field(default=None, sa_column=Column(Text))
+    location_created: str | None = Field(default=None, sa_column=Column(Text))
+    notes: str | None = Field(default=None, sa_column=Column(Text))
+    archive_identifier: str | None = Field(default=None, sa_column=Column(Text))
+    created_at: datetime | None = Field(
+        default=None,
+        sa_column=Column(
+            DateTime(timezone=True),
+            nullable=False,
+            server_default=text("now()"),
+        ),
+    )
+    updated_at: datetime | None = Field(
+        default=None,
+        sa_column=Column(
+            DateTime(timezone=True),
+            nullable=False,
+            server_default=text("now()"),
+        ),
+    )
+
+    document_people: list["DocumentPerson"] = Relationship(
+        back_populates="document",
+        sa_relationship_kwargs={"lazy": "raise", "passive_deletes": True},
+    )
+    jobs: list["Job"] = Relationship(
+        back_populates="document",
+        sa_relationship_kwargs={"lazy": "raise", "passive_deletes": True},
+    )
+    sources: list["Source"] = Relationship(
+        back_populates="document",
+        sa_relationship_kwargs={"lazy": "raise", "passive_deletes": True},
+    )
+
+
+class DocumentPerson(SQLModel, table=True):
+    __tablename__ = "document_person"
+    __table_args__ = (
+        UniqueConstraint(
+            "document_id",
+            "person_id",
+            "role",
+            name="unique_document_person_role",
+        ),
+        Index("idx_document_person_doc", "document_id"),
+        Index("idx_document_person_per", "person_id"),
+    )
+
+    id: UUID | None = Field(
+        default=None,
+        sa_column=Column(
+            PostgreSQLUUID(as_uuid=True),
+            primary_key=True,
+            server_default=text("gen_random_uuid()"),
+        ),
+    )
+    document_id: UUID = Field(
+        sa_column=Column(
+            PostgreSQLUUID(as_uuid=True),
+            ForeignKey("document.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+    )
+    person_id: UUID = Field(
+        sa_column=Column(
+            PostgreSQLUUID(as_uuid=True),
+            ForeignKey("person.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+    )
+    role: PersonRole = Field(sa_column=Column(String(20), nullable=False))
+    created_at: datetime | None = Field(
+        default=None,
+        sa_column=Column(
+            DateTime(timezone=True),
+            nullable=False,
+            server_default=text("now()"),
+        ),
+    )
+
+    document: Document | None = Relationship(
+        back_populates="document_people",
+        sa_relationship_kwargs={"lazy": "raise"},
+    )
+    person: Person | None = Relationship(
+        back_populates="document_people",
+        sa_relationship_kwargs={"lazy": "raise"},
+    )
+
+
+class Job(SQLModel, table=True):
+    __tablename__ = "job"
+    __table_args__ = (Index("idx_job_document", "document_id"),)
+
+    id: UUID | None = Field(
+        default=None,
+        sa_column=Column(
+            PostgreSQLUUID(as_uuid=True),
+            primary_key=True,
+            server_default=text("gen_random_uuid()"),
+        ),
+    )
+    document_id: UUID = Field(
+        sa_column=Column(
+            PostgreSQLUUID(as_uuid=True),
+            ForeignKey("document.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+    )
+    status: JobStatus = Field(
+        default=JobStatus.QUEUED,
+        sa_column=Column(
+            String(50),
+            nullable=False,
+            server_default=text("'queued'"),
+        ),
+    )
+    retry_count: int = Field(
+        default=0,
+        sa_column=Column(
+            Integer,
+            nullable=False,
+            server_default=text("0"),
+        ),
+    )
+    provider: str = Field(sa_column=Column(Text, nullable=False))
+    model: str = Field(sa_column=Column(Text, nullable=False))
+    prompt_name: str | None = Field(default=None, sa_column=Column(Text))
+    date_created: datetime | None = Field(
+        default=None,
+        sa_column=Column(
+            DateTime(timezone=True),
+            nullable=False,
+            server_default=text("now()"),
+        ),
+    )
+    date_updated: datetime | None = Field(
+        default=None,
+        sa_column=Column(
+            DateTime(timezone=True),
+            nullable=False,
+            server_default=text("now()"),
+        ),
+    )
+
+    document: Document | None = Relationship(
+        back_populates="jobs",
+        sa_relationship_kwargs={"lazy": "raise"},
+    )
+    job_sources: list["JobSource"] = Relationship(
+        back_populates="job",
+        sa_relationship_kwargs={"lazy": "raise", "passive_deletes": True},
+    )
+
+
+class Source(SQLModel, table=True):
+    __tablename__ = "source"
+    __table_args__ = (
+        Index("idx_source_document", "document_id"),
+        Index("idx_source_page_order", "document_id", "page_number"),
+    )
+
+    id: UUID | None = Field(
+        default=None,
+        sa_column=Column(
+            PostgreSQLUUID(as_uuid=True),
+            primary_key=True,
+            server_default=text("gen_random_uuid()"),
+        ),
+    )
+    document_id: UUID = Field(
+        sa_column=Column(
+            PostgreSQLUUID(as_uuid=True),
+            ForeignKey("document.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+    )
+    page_number: int = Field(
+        default=1,
+        sa_column=Column(
+            Integer,
+            nullable=False,
+            server_default=text("1"),
+        ),
+    )
+    upload_name: str = Field(sa_column=Column(Text, nullable=False))
+    filename: str = Field(sa_column=Column(Text, nullable=False))
+    file_path: str = Field(sa_column=Column(Text, nullable=False))
+    raw_transcription: str | None = Field(default=None, sa_column=Column(Text))
+    revised_text: str | None = Field(default=None, sa_column=Column(Text))
+    date_uploaded: datetime | None = Field(
+        default=None,
+        sa_column=Column(
+            DateTime(timezone=True),
+            nullable=False,
+            server_default=text("now()"),
+        ),
+    )
+    date_revised: datetime | None = Field(
+        default=None,
+        sa_column=Column(DateTime(timezone=True)),
+    )
+
+    document: Document | None = Relationship(
+        back_populates="sources",
+        sa_relationship_kwargs={"lazy": "raise"},
+    )
+    job_sources: list["JobSource"] = Relationship(
+        back_populates="source",
+        sa_relationship_kwargs={"lazy": "raise", "passive_deletes": True},
+    )
+
+
+class JobSource(SQLModel, table=True):
+    __tablename__ = "job_source"
+    __table_args__ = (
+        UniqueConstraint("job_id", "source_id", name="unique_job_source"),
+        Index("idx_job_source_job", "job_id"),
+        Index("idx_job_source_source", "source_id"),
+        Index(
+            "idx_job_source_ai_metadata",
+            "ai_metadata",
+            postgresql_using="gin",
+        ),
+    )
+
+    id: UUID | None = Field(
+        default=None,
+        sa_column=Column(
+            PostgreSQLUUID(as_uuid=True),
+            primary_key=True,
+            server_default=text("gen_random_uuid()"),
+        ),
+    )
+    job_id: UUID = Field(
+        sa_column=Column(
+            PostgreSQLUUID(as_uuid=True),
+            ForeignKey("job.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+    )
+    source_id: UUID = Field(
+        sa_column=Column(
+            PostgreSQLUUID(as_uuid=True),
+            ForeignKey("source.id", ondelete="CASCADE"),
+            nullable=False,
+        ),
+    )
+    status: JobSourceStatus = Field(
+        default=JobSourceStatus.PENDING,
+        sa_column=Column(
+            String(50),
+            nullable=False,
+            server_default=text("'pending'"),
+        ),
+    )
+    raw_transcription: str | None = Field(default=None, sa_column=Column(Text))
+    ai_metadata: JsonValue | None = Field(
+        default=None,
+        sa_column=Column(JSONB),
+    )
+    raw_api_response: JsonValue | None = Field(
+        default=None,
+        sa_column=Column(JSONB),
+    )
+    error_detail: str | None = Field(default=None, sa_column=Column(Text))
+    executed_at: datetime | None = Field(
+        default=None,
+        sa_column=Column(
+            DateTime(timezone=True),
+            nullable=False,
+            server_default=text("now()"),
+        ),
+    )
+
+    job: Job | None = Relationship(
+        back_populates="job_sources",
+        sa_relationship_kwargs={"lazy": "raise"},
+    )
+    source: Source | None = Relationship(
+        back_populates="job_sources",
+        sa_relationship_kwargs={"lazy": "raise"},
+    )
+```
+
+The enum annotations validate application values while the mapped columns retain the `VARCHAR` types specified by the DDL. PostgreSQL owns generated UUIDs and timestamps through `server_default`; call `session.refresh(instance)` after a flush or commit when those generated values are needed immediately.
+
+`ai_metadata`, `raw_api_response`, and `metadata_` accept any JSON value supported by `JSONB`. Validate provider-specific payload structure before assigning it to these fields, while preserving the complete raw response in `raw_api_response`.
+
+Relationships use `lazy="raise"` to prevent implicit database I/O in async code. Queries must explicitly load relationships they need, for example with `selectinload()`.

docs/ver2/ver2.md

@@ -0,0 +1,183 @@
+# Version 2 Plan
+
+## Purpose
+
+Version 2 updates the existing SQLModel domain schema to support multi-page documents, page-level transcription results, richer document metadata, and author/recipient attribution.
+
+PostgreSQL support is already present in the database runtime. V2 does not require a database-layer rewrite or a general SQLite-to-PostgreSQL migration system. PostgreSQL adoption consists primarily of selecting the existing PostgreSQL settings, provisioning the database, creating the V2 schema, and verifying the application against it.
+
+The main implementation effort is the schema update and the application changes that depend on it.
+
+---
+
+## Current State
+
+- The application uses Python 3.12, Pydantic v2, SQLModel, and async SQLAlchemy sessions.
+- The database engine already supports both SQLite and PostgreSQL through `SqliteSettings` and `PostgresSettings`.
+- The PostgreSQL async driver is installed and the engine already builds `postgresql+asyncpg` connections.
+- Schema bootstrap currently uses `SQLModel.metadata.create_all()`.
+- SQLite remains the default local configuration and the current Compose configuration still selects SQLite.
+- The current V1 domain contains `Document`, `Source`, `Job`, and `Revision` tables.
+- The V2 target is defined in [V2 DB Schema](V2%20DB%20Schema.md) and [V2 PostgreSQL DDL Specification](V2%20PostgreSQL%20DDL%20Specification.md).
+
+---
+
+## V2 Outcomes
+
+1. **V2 schema implemented**
+    - SQLModel models, relationships, enums, constraints, and indexes match the approved V2 schema.
+1. **Page-level batch processing supported**
+    - A job can process multiple sources and retain an independent result for each source through `JobSource`.
+1. **Document metadata expanded**
+    - Documents support ordered pages, descriptive metadata, and multiple authors and recipients.
+1. **Raw provider data retained**
+    - Complete provider payloads are stored in PostgreSQL `JSONB` without flattening or discarding fields.
+    - Stored documents remain suitable for a future MongoDB import if one is ever needed.
+1. **PostgreSQL enabled through configuration**
+    - The application starts against a provisioned PostgreSQL database using the existing runtime path.
+1. **Existing workflows remain reliable**
+    - Upload, worker execution, status inspection, and transcription revision work with the new schema.
+
+---
+
+## Non-Goals
+
+- Rewriting the database engine or session layer
+- Building a general-purpose SQLite-to-PostgreSQL migration utility
+- Rehearsing a production database cutover when no production dataset requires preservation
+- Running or integrating MongoDB in V2
+- Building MongoDB projections, synchronization, or fallback behavior
+- Replacing Python, Pydantic, SQLModel, or SQLAlchemy
+- Supporting more than one active human revision per source
+
+If an existing SQLite dataset must be retained, define a small one-time import task separately. It is not part of the default V2 implementation path.
+
+---
+
+## Scope
+
+### A) SQLModel schema update (primary)
+
+- Add `Person`, `DocumentPerson`, and `JobSource` models.
+- Expand `Document` with type, date, location, notes, archive identifier, and timestamps.
+- Update `Source` with page ordering, active raw transcription, revised text, and revision timestamp.
+- Update `Job` for batch execution and the `partial_success` terminal state.
+- Replace the standalone `Revision` table with revision fields on `Source`.
+- Remove the direct `Source.job_id` relationship; connect sources to jobs through `JobSource`.
+- Add role, job status, and job-source status enums.
+- Add required uniqueness constraints, foreign-key delete behavior, lookup indexes, and PostgreSQL JSON indexes.
+- Keep model definitions aligned with [V2 Python Pydantic Models](V2%20Python%20Pydantic%20Models.md).
+
+### B) Raw document storage
+
+- Store structured AI metadata in `job_source.ai_metadata` as `JSONB`.
+- Store the complete raw provider response in `job_source.raw_api_response` as `JSONB`.
+- Preserve the original document structure, field names, nested values, and unknown fields in the raw response.
+- Keep validation of extracted application fields separate from retention of the raw response.
+- Serialize UUIDs and datetimes using portable string representations.
+- Use MongoDB Extended JSON representations only if a provider value cannot be represented faithfully in standard JSON.
+- Do not store opaque BSON bytes in PostgreSQL unless a future payload contains BSON-only values that cannot be preserved in `JSONB`.
+
+### C) Schema creation and verification
+
+- Use a fresh V2 database during development unless preservation of existing data becomes a requirement.
+- Create the schema from SQLModel metadata and verify it against the approved DDL.
+- Keep SQLite available for fast unit tests where its behavior is equivalent.
+- Add focused PostgreSQL integration tests for native UUIDs, JSON storage, constraints, indexes, and transactions.
+- Introduce migration tooling only if V2 must update a populated deployed database in place.
+
+### D) Service and worker alignment
+
+- Update document, source, job, and store operations for the new relationships.
+- Create one `JobSource` row per source included in a job.
+- Persist page-level status, transcription, AI metadata, raw provider response, and errors on `JobSource`.
+- Derive the parent job status from its page results:
+  - `completed` when all pages succeed
+  - `partial_success` when successful and failed pages are mixed
+  - `failed` when all pages fail or a job-level failure prevents execution
+- Update `Source.raw_transcription` after a successful page result while keeping the original `JobSource.raw_transcription` immutable.
+- Read `Source.revised_text` in preference to `Source.raw_transcription` when presenting active text.
+
+### E) Document and multi-image workflows
+
+- Require a document before associating uploaded sources.
+- Support uploading multiple images into one document.
+- Preserve page order through `Source.page_number`.
+- Allow a job to include one or more sources from the same document.
+- Update job details to show the document, each source filename, page order, page status, and page-level errors.
+- Define a practical upload limit and split oversized selections into manageable batches if needed.
+
+### F) PostgreSQL configuration
+
+- Provision PostgreSQL for local and deployed environments.
+- Configure the existing `Settings.database` field with PostgreSQL host, port, database, user, and password values.
+- Update Compose and environment configuration to stop selecting SQLite.
+- Decide whether schema bootstrap is enabled for local development or performed as a separate deployment step.
+- Run a connectivity and schema smoke test against PostgreSQL.
+- Keep uploaded files on a persistent, backup-capable path outside the application image.
+
+---
+
+## Milestones
+
+### M1 - Schema models
+
+- Implement the V2 SQLModel models and enums.
+- Implement relationships, constraints, indexes, and JSON column types.
+- Update the Pydantic data contracts where model decisions change.
+- Add schema-focused tests.
+
+**Exit criteria:** SQLModel metadata represents the approved V2 schema and schema tests pass.
+
+### M2 - Persistence and worker behavior
+
+- Update database operations and services for the V2 entities.
+- Implement page-level `JobSource` execution records.
+- Preserve complete raw provider responses in `JSONB`.
+- Implement aggregate job status calculation.
+- Add transaction, partial-success, and failure-isolation tests.
+
+**Exit criteria:** single-page and multi-page jobs persist correct page, raw payload, and aggregate states.
+
+### M3 - Document and upload workflows
+
+- Update document creation and source association flows.
+- Add ordered multi-image upload.
+- Update job and document detail views for page-level results.
+- Add focused UI and service tests.
+
+**Exit criteria:** a user can create a document, upload ordered pages, run a job, and inspect each result.
+
+### M4 - PostgreSQL verification and release
+
+- Switch local or test configuration to the existing PostgreSQL runtime path.
+- Create the V2 schema in a fresh PostgreSQL database.
+- Run PostgreSQL-specific schema and workflow tests.
+- Document startup, backup, and recovery settings.
+- Run the final regression suite.
+
+**Exit criteria:** V2 workflows pass against PostgreSQL and release checks are complete.
+
+---
+
+## Risks and Mitigations
+
+- **Model and DDL drift** -> compare generated metadata with the approved schema and test named constraints and indexes.
+- **Raw payload loss** -> retain the complete provider response separately from validated and extracted fields.
+- **Cross-database differences** -> retain fast SQLite tests but verify PostgreSQL-native UUID, JSON, and index behavior in integration tests.
+- **Batch state errors** -> test all-success, mixed-result, and all-failed jobs explicitly.
+- **Page ordering errors** -> enforce uniqueness and ordering rules for document pages.
+- **Unexpected data-preservation need** -> confirm whether existing SQLite data matters before implementation; add a one-time importer only when required.
+- **Worker regressions** -> preserve terminal-state and retry reliability tests while changing persistence ownership.
+
+---
+
+## Suggested First Tasks
+
+1. Update `src/transcription/db/models.py` to represent the approved V2 schema.
+2. Add schema tests for tables, columns, relationships, constraints, indexes, and enums.
+3. Define and test lossless raw provider response storage in `job_source.raw_api_response`.
+4. Update database operations and services to use `JobSource` and source-level revisions.
+5. Add page-result aggregation tests before changing the worker workflow.
+6. Update document and multi-image upload flows.
+7. Select PostgreSQL in configuration and run the integration suite against a fresh V2 database.

src/transcription/__main__.py

@@ -0,0 +1,20 @@
+import uvicorn
+
+from .config import LOGGING_CONFIG
+from .config import get_settings
+
+
+def main() -> None:
+    settings = get_settings()
+    uvicorn.run(
+        "transcription.app:create_app",
+        factory=True,
+        host=settings.host,
+        port=settings.port,
+        log_level=LOGGING_CONFIG.get("root", {}).get("level", "info").lower(),
+        reload=settings.reload,
+    )
+
+
+if __name__ == "__main__":
+    main()

src/transcription/app.py

@@ -16,11 +16,14 @@ from fastapi.staticfiles import StaticFiles
 
 from .api.errors import register_error_handlers
 from .api.health import router as health_router
+from .config import Settings
 from .config import configure_logging
 from .config import get_settings
 from .db import create_all
-from .db import dispose_database_runtime
 from .db import initialize_database_runtime
+from .db.engine import get_database_url
+from .db.engine import resolve_engine
+from .db.session import dispose_session_factory
 from .services import ServiceBundle
 from .services.jobs import JobService
 from .ui import register_pages
@@ -39,7 +42,7 @@ async def _lifespan(app: FastAPI):
     app.state.runtime = initialize_database_runtime(settings=settings)
 
     if settings.should_bootstrap_schema:
-        await create_all(engine=app.state.runtime.engine)
+        await create_all(engine=resolve_engine(settings=settings))
 
     settings.upload_dir.mkdir(parents=True, exist_ok=True)
     settings.prompt_dir.mkdir(parents=True, exist_ok=True)
@@ -47,7 +50,10 @@ async def _lifespan(app: FastAPI):
     await _recover_stale_processing_jobs(app)
 
     async with AsyncExitStack() as stack:
-        stack.push_async_callback(dispose_database_runtime)
+        stack.push_async_callback(
+            dispose_session_factory,
+            database_url=get_database_url(settings),
+        )
         stop_event, worker_notifier = await stack.enter_async_context(
             worker_consumer_lifespan(
                 session_factory=app.state.runtime.session_factory,
@@ -73,14 +79,14 @@ async def _recover_stale_processing_jobs(app: FastAPI) -> None:
         logger.warning("Recovered %s stale processing job(s) at startup", recovered)
 
 
-def create_app() -> FastAPI:
+def create_app(settings: Settings | None = None) -> FastAPI:
     """Create and configure the FastAPI application."""
     app = FastAPI(title="Transcription", lifespan=_lifespan)
-    settings = get_settings()
-    app.state.settings = settings
+    active_settings = settings or get_settings()
+    app.state.settings = active_settings
     app.mount(
         "/uploads",
-        StaticFiles(directory=settings.upload_dir, check_dir=False),
+        StaticFiles(directory=active_settings.upload_dir, check_dir=False),
         name="uploads",
     )
 
@@ -92,6 +98,10 @@ def create_app() -> FastAPI:
     async def ui_redirect() -> RedirectResponse:
         return RedirectResponse(url="/ui/upload", status_code=status.HTTP_307_TEMPORARY_REDIRECT)
 
+    @app.get("/healthz")
+    def health() -> dict[str, str]:
+        return {"status": "ok"}
+
     register_error_handlers(app)
     register_pages(app)
     app.include_router(health_router)

src/transcription/app_state.py

@@ -7,7 +7,7 @@ from sqlalchemy.ext.asyncio import async_sessionmaker
 from sqlmodel.ext.asyncio.session import AsyncSession
 
 from transcription.db.runtime import DatabaseRuntime
-from transcription.db.runtime import get_session_factory
+from transcription.db.session import get_session_factory
 from transcription.worker import WorkerNotifier
 from transcription.worker import resolve_worker_notifier
 

src/transcription/config.py

@@ -6,12 +6,16 @@ are resolved by the provider adapters, not here.
 """
 
 import logging.config
-from contextvars import ContextVar
 from enum import StrEnum
+from functools import cache
 from pathlib import Path
+from typing import Annotated
+from typing import Any
 from typing import Literal
 
+from pydantic import BaseModel
 from pydantic import Field
+from pydantic import SecretStr
 from pydantic_settings import BaseSettings
 from pydantic_settings import SettingsConfigDict
 
@@ -22,13 +26,42 @@ class Provider(StrEnum):
     OPENROUTER = "openrouter"
 
 
+class SqliteSettings(BaseModel):
+    driver: Literal["sqlite"] = "sqlite"
+    path: str = "app.db"
+
+
+class PostgresSettings(BaseModel):
+    driver: Literal["postgres"] = "postgres"
+    host: str
+    port: int = 5432
+    database: str
+    user: str
+    password: SecretStr
+
+
+DatabaseSettings = Annotated[
+    SqliteSettings | PostgresSettings,
+    Field(discriminator="driver"),
+]
+
+
 class Settings(BaseSettings):
     model_config = SettingsConfigDict(
         env_file=".env",
         env_file_encoding="utf-8",
         extra="ignore",
+        cli_parse_args=True,
+        cli_implicit_flags=True,
+        cli_kebab_case=True,
     )
 
+    # --- NiceGUI Server ---
+    host: str = "0.0.0.0"
+    port: int = 8000
+    log_level: Literal["critical", "error", "warning", "info", "debug", "trace"] = "info"
+    reload: bool = False
+
     # --- AI provider ---
     provider: Provider = Provider.OPENROUTER
     openrouter_api_key: str
@@ -40,8 +73,9 @@ class Settings(BaseSettings):
     environment: Literal["development", "test", "production"] = "development"
 
     # --- persistence ---
+    database: DatabaseSettings = Field(default_factory=SqliteSettings)
     database_url: str = "sqlite:///./transcription.db"
-    bootstrap_schema_on_startup: bool | None = None
+    bootstrap_schema_on_startup: bool = False
     sqlite_check_same_thread: bool = False
 
     # --- filesystem paths ---
@@ -64,18 +98,12 @@ class Settings(BaseSettings):
         return self.environment in {"development", "test"}
 
 
-_settings: ContextVar[Settings | None] = ContextVar("settings", default=None)
-
-
+@cache
 def get_settings(**kwargs) -> Settings:
-    settings = _settings.get()
-    if settings is None:
-        settings = Settings(**kwargs)  # pyright: ignore[reportCallIssue]
-        _settings.set(settings)
-    return settings
+    return Settings(**kwargs)
 
 
-LOGGING_CONFIG: dict[str, object] = {
+LOGGING_CONFIG: dict[str, Any] = {
     "version": 1,
     "disable_existing_loggers": False,
     "formatters": {
@@ -105,7 +133,10 @@ LOGGING_CONFIG: dict[str, object] = {
 }
 
 
-def configure_logging() -> None:
+def configure_logging(settings: Settings | None = None) -> None:
     """Configure root logging once at startup."""
-    logging.config.dictConfig(LOGGING_CONFIG)
+    cfg = LOGGING_CONFIG.copy()
+    active_settings = settings or get_settings()
+    cfg["loggers"]["transcription"]["level"] = active_settings.log_level.upper()
+    logging.config.dictConfig(cfg)
     logger.debug("Logging configured")

src/transcription/db/__init__.py

@@ -1,6 +1,13 @@
 from .operations import create_all
 from .runtime import dispose_database_runtime
-from .runtime import get_session
 from .runtime import initialize_database_runtime
+from .session import session_scope
+from .session import transaction_scope
 
-__all__ = ["create_all", "dispose_database_runtime", "get_session", "initialize_database_runtime"]
+__all__ = [
+    "create_all",
+    "dispose_database_runtime",
+    "initialize_database_runtime",
+    "session_scope",
+    "transaction_scope",
+]

src/transcription/db/engine.py

@@ -0,0 +1,60 @@
+from functools import cache
+from typing import Any
+
+from sqlalchemy import URL
+from sqlalchemy import StaticPool
+from sqlalchemy.ext.asyncio import AsyncEngine
+from sqlalchemy.ext.asyncio import create_async_engine
+
+from ..config import PostgresSettings
+from ..config import Settings
+from ..config import SqliteSettings
+from ..config import get_settings
+
+
+def get_database_url(settings: Settings) -> str:
+    match settings.database:
+        case SqliteSettings(path=path):
+            url = URL.create(
+                drivername="sqlite+aiosqlite",
+                database=path,
+            )
+        case PostgresSettings() as database:
+            url = URL.create(
+                drivername="postgresql+asyncpg",
+                host=database.host,
+                port=database.port,
+                database=database.database,
+                username=database.user,
+                password=database.password.get_secret_value(),
+            )
+    return url.render_as_string(hide_password=False)
+
+
+def resolve_engine(settings: Settings | None = None) -> AsyncEngine:
+    active_settings = settings or get_settings()
+    return get_engine(get_database_url(active_settings))
+
+
+@cache
+def get_engine(database_url: str) -> AsyncEngine:
+    kwargs: dict[str, Any] = {"echo": False, "pool_pre_ping": True}
+    if database_url.startswith("sqlite"):
+        kwargs["connect_args"] = {"check_same_thread": False}
+        if ":memory:" in database_url:
+            kwargs["poolclass"] = StaticPool
+
+    return create_async_engine(database_url, **kwargs)
+
+
+async def dispose_engine(database_url: str) -> None:
+    engine = get_engine(database_url)
+    try:
+        await engine.dispose()
+    finally:
+        get_engine.cache_clear()
+
+
+async def refresh_engine(database_url: str) -> AsyncEngine:
+    await dispose_engine(database_url)
+    return get_engine(database_url)

src/transcription/db/operations.py

@@ -10,13 +10,25 @@ from sqlmodel import SQLModel
 from sqlmodel import select
 from sqlmodel.ext.asyncio.session import AsyncSession
 
-from ..models import Job
-from ..models import JobStatus
-from .runtime import get_engine
+from .engine import resolve_engine
+from .models import Job
+from .models import JobStatus
 
 logger = logging.getLogger(__name__)
 
 
+async def create_all(*, engine: AsyncEngine | None = None) -> None:
+    """Create all tables on the selected engine."""
+    # Import models so SQLModel metadata is fully registered before bootstrap.
+    from transcription.db import models as _models  # noqa: F401
+
+    active_engine = engine or resolve_engine()
+    async with active_engine.begin() as connection:
+        await connection.run_sync(SQLModel.metadata.create_all)
+        await connection.run_sync(_ensure_sqlite_compat_columns)
+    logger.debug("Database schema bootstrap complete for database_url=%s", active_engine.url)
+
+
 async def get_next_queued_job(*, session: AsyncSession) -> Job | None:
     """Get the next queued job, if any."""
     result = await session.exec(
@@ -28,18 +40,6 @@ async def get_next_queued_job(*, session: AsyncSession) -> Job | None:
     return result.first()
 
 
-async def create_all(*, engine: AsyncEngine | None = None) -> None:
-    """Create all tables on the selected engine."""
-    # Import models so SQLModel metadata is fully registered before bootstrap.
-    from transcription import models as _models  # noqa: F401
-
-    active_engine = engine or get_engine()
-    async with active_engine.begin() as connection:
-        await connection.run_sync(SQLModel.metadata.create_all)
-        await connection.run_sync(_ensure_sqlite_compat_columns)
-    logger.debug("Database schema bootstrap complete for database_url=%s", active_engine.url)
-
-
 def _ensure_sqlite_compat_columns(connection: Connection) -> None:
     """Apply lightweight dev/test SQLite compatibility column patches.
 
@@ -68,12 +68,8 @@ def _ensure_sqlite_compat_columns(connection: Connection) -> None:
                     break
             if not has_unique_source:
                 connection.execute(
-                    text(
-                        "CREATE UNIQUE INDEX IF NOT EXISTS "
-                        "ux_revision_source_id ON revision(source_id)"
-                    )
+                    text("CREATE UNIQUE INDEX IF NOT EXISTS ux_revision_source_id ON revision(source_id)")
                 )
                 logger.warning(
-                    "Applied SQLite compatibility schema patch "
-                    "table=revision unique_index=ux_revision_source_id"
+                    "Applied SQLite compatibility schema patch table=revision unique_index=ux_revision_source_id"
                 )

src/transcription/db/runtime.py

@@ -1,18 +1,16 @@
 import logging
-from collections.abc import AsyncGenerator
-from contextlib import asynccontextmanager
 from contextvars import ContextVar
 from dataclasses import dataclass
-from functools import partial
 
 from sqlalchemy.ext.asyncio import AsyncEngine
 from sqlalchemy.ext.asyncio import async_sessionmaker
-from sqlalchemy.ext.asyncio import create_async_engine
 from sqlmodel.ext.asyncio.session import AsyncSession
-from sqlmodel.pool import StaticPool
 
 from ..config import Settings
 from ..config import get_settings
+from .engine import get_database_url
+from .engine import get_engine
+from .session import get_session_factory
 
 logger = logging.getLogger(__name__)
 
@@ -37,33 +35,6 @@ async def dispose_database_runtime() -> None:
     _runtime.set(None)
 
 
-def _to_async_database_url(database_url: str) -> str:
-    """Normalize configured database URL to an async SQLAlchemy driver URL."""
-    if database_url.startswith("sqlite://") and not database_url.startswith("sqlite+aiosqlite://"):
-        return database_url.replace("sqlite://", "sqlite+aiosqlite://", 1)
-    if database_url.startswith("postgresql://") and not database_url.startswith("postgresql+asyncpg://"):
-        return database_url.replace("postgresql://", "postgresql+asyncpg://", 1)
-    return database_url
-
-
-def _build_engine(settings: Settings) -> AsyncEngine:
-    database_url = _to_async_database_url(settings.database_url)
-    engine_factory = partial(
-        create_async_engine,
-        url=database_url,
-        echo=False,
-        pool_pre_ping=True,
-    )
-
-    if database_url.startswith("sqlite"):
-        sqlite_connect_settings = {"check_same_thread": settings.sqlite_check_same_thread}
-        engine_factory = partial(engine_factory, connect_args=sqlite_connect_settings)
-        if ":memory:" in database_url:
-            engine_factory = partial(engine_factory, poolclass=StaticPool)
-
-    return engine_factory()
-
-
 def initialize_database_runtime(*, settings: Settings | None = None) -> DatabaseRuntime:
     """Initialize lifespan-owned async DB resources once per process."""
     runtime = _runtime.get()
@@ -71,33 +42,10 @@ def initialize_database_runtime(*, settings: Settings | None = None) -> Database
         return runtime
 
     active_settings = settings or get_settings()
-    engine = _build_engine(active_settings)
-    session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
+    database_url = get_database_url(active_settings)
+    engine = get_engine(database_url)
+    session_factory = get_session_factory(database_url)
     runtime = DatabaseRuntime(engine=engine, session_factory=session_factory)
     _runtime.set(runtime)
     logger.debug("Initialized async database runtime for database_url=%s", engine.url)
     return runtime
-
-
-def get_engine(settings: Settings | None = None) -> AsyncEngine:
-    """Return the current async SQLAlchemy engine."""
-    runtime = _runtime.get() or initialize_database_runtime(settings=settings)
-    return runtime.engine
-
-
-def get_session_factory(settings: Settings | None = None) -> async_sessionmaker[AsyncSession]:
-    """Return the shared async session factory."""
-    runtime = _runtime.get() or initialize_database_runtime(settings=settings)
-    return runtime.session_factory
-
-
-@asynccontextmanager
-async def get_session(
-    *,
-    settings: Settings | None = None,
-    session_factory: async_sessionmaker[AsyncSession] | None = None,
-) -> AsyncGenerator[AsyncSession]:
-    """Yield a database session and ensure cleanup."""
-    active_session_factory = session_factory or get_session_factory(settings)
-    async with active_session_factory() as session:
-        yield session

src/transcription/db/session.py

@@ -0,0 +1,79 @@
+from collections.abc import AsyncGenerator
+from contextlib import asynccontextmanager
+from functools import cache
+from typing import Annotated
+
+from fastapi import Depends
+from sqlalchemy.ext.asyncio import AsyncSessionTransaction
+from sqlalchemy.ext.asyncio import async_sessionmaker
+from sqlmodel.ext.asyncio.session import AsyncSession
+
+from ..config import get_settings
+from .engine import dispose_engine
+from .engine import get_database_url
+from .engine import get_engine
+
+type SessionFactory = async_sessionmaker[AsyncSession]
+
+
+@cache
+def get_session_factory(database_url: str) -> SessionFactory:
+    return async_sessionmaker(
+        bind=get_engine(database_url),
+        class_=AsyncSession,
+        expire_on_commit=False,
+    )
+
+
+def resolve_session_factory(database_url: str | None = None) -> SessionFactory:
+    return get_session_factory(database_url or get_database_url(get_settings()))
+
+
+type SessionFactoryDep = Annotated[SessionFactory, Depends(resolve_session_factory)]
+
+
+async def dispose_session_factory(database_url: str) -> None:
+    get_session_factory.cache_clear()
+    await dispose_engine(database_url)
+
+
+@asynccontextmanager
+async def session_scope(
+    *,
+    database_url: str | None = None,
+    session: AsyncSession | None = None,
+) -> AsyncGenerator[AsyncSession]:
+    if session is not None:
+        yield session
+        return
+
+    session_factory = resolve_session_factory(database_url)
+    async with session_factory() as owned_session:
+        yield owned_session
+
+
+type SessionScopeDep = Annotated[AsyncSession, Depends(session_scope)]
+
+
+@asynccontextmanager
+async def transaction_scope(
+    *,
+    database_url: str | None = None,
+    session: AsyncSessionTransaction | None = None,
+) -> AsyncGenerator[AsyncSessionTransaction]:
+    match session:
+        case AsyncSession() as async_session:
+            if not async_session.in_transaction():
+                raise RuntimeError("A supplied session must have an active transaction")
+            yield async_session
+            return
+        case AsyncSessionTransaction() as async_transaction:
+            yield async_transaction
+            return
+
+    session_factory = resolve_session_factory(database_url)
+    async with session_factory().begin() as owned_session:
+        yield owned_session
+
+
+type TransactionScopeDep = Annotated[AsyncSessionTransaction, Depends(transaction_scope)]

src/transcription/services/base.py

@@ -8,7 +8,8 @@ from sqlmodel.ext.asyncio.session import AsyncSession
 
 from ..config import Settings
 from ..config import get_settings
-from ..db.runtime import get_session_factory
+from ..db.session import resolve_session_factory
+from ..db.session import session_scope
 
 
 class ServiceBase(ABC):
@@ -24,19 +25,14 @@ class ServiceBase(ABC):
         queue: asyncio.Queue | None = None,
     ):
         self.settings = get_settings()
-        self.session_factory = session_factory or get_session_factory()
+        self.session_factory = session_factory or resolve_session_factory()
         self.queue = queue or asyncio.Queue()
 
     @asynccontextmanager
     async def _session_scope(self, session: AsyncSession | None = None):
         """Provide a transactional scope around a series of operations."""
-        if session is not None:
-            # Reuse the provided session if one is passed in
-            yield session
-        else:
-            # Otherwise, create a new session for this scope
-            async with self.session_factory() as new_session:
-                yield new_session
+        async with session_scope(session=session) as active_session:
+            yield active_session
 
     async def _finalize(
         self,

src/transcription/services/documents.py

@@ -9,9 +9,9 @@ from sqlalchemy.orm import selectinload
 from sqlmodel import select
 from sqlmodel.ext.asyncio.session import AsyncSession
 
+from ..db.models import Document
 from ..errors import AppError
 from ..errors import ErrorCategory
-from ..models import Document
 from .base import ServiceBase
 
 logger = logging.getLogger(__name__)

src/transcription/services/jobs.py

@@ -7,9 +7,9 @@ from sqlalchemy.orm import selectinload
 from sqlmodel import select
 from sqlmodel.ext.asyncio.session import AsyncSession
 
-from ..models import Job
-from ..models import JobStatus
-from ..models import Source
+from ..db.models import Job
+from ..db.models import JobStatus
+from ..db.models import Source
 from .base import ServiceBase
 
 
@@ -170,11 +170,7 @@ class JobService(ServiceBase):
         ``stale_before`` are considered stale and re-queued.
         """
         async with self._session_scope(session) as _session:
-            query = (
-                select(Job)
-                .where(Job.status == JobStatus.PROCESSING)
-                .where(Job.date_updated < stale_before)
-            )
+            query = select(Job).where(Job.status == JobStatus.PROCESSING).where(Job.date_updated < stale_before)
             stale_jobs = (await _session.exec(query)).all()
             if not stale_jobs:
                 return 0

src/transcription/services/store.py

@@ -11,9 +11,9 @@ from transcription.config import get_settings
 from transcription.errors import AppError
 from transcription.errors import ErrorCategory
 
-from ..models import Document
-from ..models import Job
-from ..models import Source
+from ..db.models import Document
+from ..db.models import Job
+from ..db.models import Source
 from .documents import UploadJobResult
 
 logger = logging.getLogger(__name__)

src/transcription/services/transcription.py

@@ -18,11 +18,11 @@ from sqlmodel.ext.asyncio.session import AsyncSession
 
 from transcription.config import Settings
 from transcription.config import get_settings
+from transcription.db.models import Job
+from transcription.db.models import Revision
+from transcription.db.models import Source
 from transcription.errors import AppError
 from transcription.errors import ErrorCategory
-from transcription.models import Job
-from transcription.models import Revision
-from transcription.models import Source
 from transcription.providers import ProviderAuthError
 from transcription.providers import ProviderError
 from transcription.providers import ProviderResponseError

src/transcription/services/workflows.py

@@ -5,13 +5,13 @@ from sqlmodel.ext.asyncio.session import AsyncSession
 
 from ..config import Settings
 from ..config import get_settings
+from ..db.models import Job
+from ..db.models import JobStatus
+from ..db.models import Source
 from ..errors import AppError
 from ..errors import ErrorCategory
 from ..errors import classify_unexpected_error
 from ..errors import format_error_detail
-from ..models import Job
-from ..models import JobStatus
-from ..models import Source
 from ..providers import TranscriptionResult
 from . import ServiceBundle
 from .transcription import DEFAULT_PROMPT_FILE

src/transcription/ui/components/document_panzoom.py

@@ -10,7 +10,7 @@ from uuid import uuid4
 from nicegui import ui
 
 from transcription.config import get_settings
-from transcription.models import Source
+from transcription.db.models import Source
 
 PANGOZOOM_CDN_URL = "https://unpkg.com/@panzoom/[email protected]/dist/panzoom.min.js"
 UPLOADS_URL_PREFIX = "/uploads"

src/transcription/ui/components/job_detail.py

@@ -6,9 +6,9 @@ import logging
 
 from nicegui import ui
 
-from transcription.models import Job
-from transcription.models import Revision
-from transcription.models import Source
+from transcription.db.models import Job
+from transcription.db.models import Revision
+from transcription.db.models import Source
 from transcription.ui.components.document_panzoom import render_document_panzoom
 from transcription.ui.components.transcript import render_original_transcription_card
 from transcription.ui.components.transcript import render_revision_row

src/transcription/ui/components/transcript.py

@@ -9,8 +9,8 @@ from typing import Any
 
 from nicegui import ui
 
-from transcription.models import Job
-from transcription.models import Revision
+from transcription.db.models import Job
+from transcription.db.models import Revision
 
 type RevisionAction = Callable[[Revision], Awaitable[None] | None]
 

src/transcription/ui/pages/jobs_page.py

@@ -4,19 +4,18 @@ from __future__ import annotations
 
 from uuid import UUID
 
-from fastapi import Request
 from nicegui import ui
 
-from transcription.app_state import resolve_session_factory
-from transcription.models import Job
-from transcription.models import JobStatus
-from transcription.models import Source
+from transcription.db.models import Job
+from transcription.db.models import JobStatus
+from transcription.db.models import Source
 from transcription.services.jobs import JobService
 from transcription.services.transcription import TranscriptionService
 from transcription.ui.components.app_shell import render_navigation_header
 from transcription.ui.components.error_presenter import show_error
 from transcription.ui.components.table.jobs import render_jobs_table
 
+from ...db.session import SessionFactoryDep
 from ..components.document_panzoom import render_document_panzoom
 from ..components.table.jobs import JobTableRow
 from ..components.transcript import render_original_transcription_card
@@ -27,8 +26,7 @@ def register_page() -> None:  # noqa: PLR0915
     """Register jobs list and detail routes."""
 
     @ui.page("/jobs")
-    async def jobs_page(request: Request) -> None:
-        session_factory = resolve_session_factory(request.app.state)
+    async def jobs_page(session_factory: SessionFactoryDep) -> None:
         jobs_service = JobService(session_factory=session_factory)
         render_navigation_header(current_path="/jobs")
 
@@ -51,8 +49,7 @@ def register_page() -> None:  # noqa: PLR0915
         await render_table()
 
     @ui.page("/jobs/{job_id}")
-    async def job_detail_page(job_id: str, request: Request) -> None:  # noqa: PLR0915
-        session_factory = resolve_session_factory(request.app.state)
+    async def job_detail_page(job_id: str, session_factory: SessionFactoryDep) -> None:  # noqa: PLR0915
         jobs_service = JobService(session_factory=session_factory)
         transcription_service = TranscriptionService(session_factory=session_factory)
         render_navigation_header(current_path="/jobs")

src/transcription/ui/pages/upload_page.py

@@ -5,8 +5,7 @@ from __future__ import annotations
 from fastapi import Request
 from nicegui import ui
 
-from transcription.app_state import resolve_session_factory
-from transcription.db import get_session
+from transcription.db import session_scope
 from transcription.services.store import create_upload_job
 from transcription.ui.components.app_shell import render_navigation_header
 from transcription.ui.components.upload import render_upload_widget
@@ -19,10 +18,9 @@ def register_page() -> None:
     @ui.page("/upload", title="Upload Document")
     def upload_page(request: Request) -> None:
         render_navigation_header(current_path="/upload")
-        session_factory = resolve_session_factory(request.app.state)
 
         async def submit_upload(filename: str, file_bytes: bytes):
-            async with get_session(session_factory=session_factory) as session:
+            async with session_scope() as session:
                 return await create_upload_job(
                     filename=filename,
                     file_bytes=file_bytes,

src/transcription/worker.py

@@ -14,7 +14,7 @@ from uuid import UUID
 from sqlalchemy.ext.asyncio import async_sessionmaker
 from sqlmodel.ext.asyncio.session import AsyncSession
 
-from transcription.db import get_session
+from transcription.db import session_scope
 from transcription.errors import AppError
 from transcription.errors import classify_unexpected_error
 
@@ -178,7 +178,7 @@ async def process_next_queued_job(
         )
 
     if session is None:
-        async with get_session(session_factory=session_factory) as local_session:
+        async with session_scope(session_factory=session_factory) as local_session:
             return await process_next_queued_job_workflow(services=services, session=local_session)
 
     return await process_next_queued_job_workflow(services=services, session=session)

tests/conftest.py

@@ -13,11 +13,12 @@ from sqlmodel.pool import StaticPool
 
 from transcription.config import Settings
 from transcription.config import get_settings
+from transcription.db.engine import get_database_url
+from transcription.db.engine import get_engine
 from transcription.db.operations import create_all
-from transcription.db.runtime import dispose_database_runtime
-from transcription.db.runtime import get_engine
-from transcription.db.runtime import get_session
-from transcription.db.runtime import get_session_factory
+from transcription.db.session import dispose_session_factory
+from transcription.db.session import get_session_factory
+from transcription.db.session import session_scope
 from transcription.services.documents import DocumentService
 from transcription.services.jobs import JobService
 
@@ -39,23 +40,26 @@ def session():
 async def default_settings():
     """Provide default settings for tests."""
     settings = get_settings(database_url="sqlite:///:memory:")
-    await create_all(engine=get_engine(settings=settings))
+    db_url = get_database_url(settings)
+    await create_all(engine=get_engine(database_url=db_url))
     return settings
 
 
 @pytest_asyncio.fixture
 async def async_session(default_settings: Settings):
     """Provide a clean asynchronous database session for async tests."""
-    async with get_session(settings=default_settings) as async_session:
+    db_url = get_database_url(default_settings)
+    async with session_scope(database_url=db_url) as async_session:
         yield async_session
 
-    await dispose_database_runtime()
+    await dispose_session_factory(db_url)
 
 
 @pytest.fixture
 def default_session_factory(default_settings: Settings):
     """Provide a base fixture for tests that require database access."""
-    session_factory = get_session_factory(settings=default_settings)
+    db_url = get_database_url(default_settings)
+    session_factory = get_session_factory(database_url=db_url)
     return session_factory
 
 

tests/integration/test_pipeline_flow.py

@@ -5,8 +5,8 @@ from pathlib import Path
 import pytest
 
 from transcription.config import Settings
-from transcription.models import Job
-from transcription.models import JobStatus
+from transcription.db.models import Job
+from transcription.db.models import JobStatus
 from transcription.providers.base import TranscriptionResult
 from transcription.services.store import create_upload_job
 from transcription.worker import process_next_queued_job

tests/services/test_job_service.py

@@ -2,10 +2,10 @@ from uuid import uuid4
 
 import pytest
 
-from transcription.models import Document
-from transcription.models import Job
-from transcription.models import JobStatus
-from transcription.models import Source
+from transcription.db.models import Document
+from transcription.db.models import Job
+from transcription.db.models import JobStatus
+from transcription.db.models import Source
 from transcription.services.documents import DocumentService
 from transcription.services.jobs import JobService
 

tests/services/test_transcription_service.py

@@ -4,10 +4,10 @@ from uuid import uuid4
 
 import pytest
 
-from transcription.models import Document
-from transcription.models import Job
-from transcription.models import JobStatus
-from transcription.models import Source
+from transcription.db.models import Document
+from transcription.db.models import Job
+from transcription.db.models import JobStatus
+from transcription.db.models import Source
 from transcription.services.documents import DocumentService
 from transcription.services.jobs import JobService
 from transcription.services.transcription import TranscriptionService

tests/services/test_workflows_reliability.py

@@ -6,10 +6,10 @@ from uuid import uuid4
 import pytest
 
 from transcription.config import Settings
-from transcription.models import Document
-from transcription.models import Job
-from transcription.models import JobStatus
-from transcription.models import Source
+from transcription.db.models import Document
+from transcription.db.models import Job
+from transcription.db.models import JobStatus
+from transcription.db.models import Source
 from transcription.services import ServiceBundle
 from transcription.services.workflows import process_queued_job
 

tests/test_db.py

@@ -4,17 +4,18 @@ import pytest
 from sqlalchemy import inspect
 
 from transcription.config import Settings
+from transcription.config import SqliteSettings
 from transcription.db import create_all
 from transcription.db import dispose_database_runtime
-from transcription.db import get_session
 from transcription.db import initialize_database_runtime
+from transcription.db import session_scope
 
 
 @pytest.mark.asyncio
 async def test_create_all_creates_expected_tables(tmp_path):
     settings = Settings(
         openrouter_api_key="test-key",
-        database_url=f"sqlite:///{tmp_path / 'schema.db'}",
+        database=SqliteSettings(path=str(tmp_path / "schema.db")),
         environment="test",
     )
     runtime = initialize_database_runtime(settings=settings)
@@ -36,13 +37,13 @@ async def test_create_all_creates_expected_tables(tmp_path):
 async def test_get_session_yields_async_session(tmp_path):
     settings = Settings(
         openrouter_api_key="test-key",
-        database_url=f"sqlite:///{tmp_path / 'session.db'}",
+        database=SqliteSettings(path=str(tmp_path / "session.db")),
         environment="test",
     )
     initialize_database_runtime(settings=settings)
 
     try:
-        async with get_session(settings=settings) as session:
+        async with session_scope(settings=settings) as session:
             assert session is not None
     finally:
         await dispose_database_runtime()

tests/test_models.py

@@ -5,11 +5,11 @@ from uuid import UUID
 import pytest
 from sqlalchemy.exc import IntegrityError
 
-from transcription.models import Document
-from transcription.models import Job
-from transcription.models import JobStatus
-from transcription.models import Revision
-from transcription.models import Source
+from transcription.db.models import Document
+from transcription.db.models import Job
+from transcription.db.models import JobStatus
+from transcription.db.models import Revision
+from transcription.db.models import Source
 
 
 def _make_document(**overrides) -> Document:

tests/ui/conftest.py

@@ -4,6 +4,7 @@ from __future__ import annotations
 
 import asyncio
 from collections.abc import Callable
+from collections.abc import Generator
 from pathlib import Path
 from uuid import UUID
 
@@ -14,32 +15,31 @@ from sqlmodel import delete
 
 from transcription.app import create_app
 from transcription.config import Settings
-from transcription.config import _settings
+from transcription.config import SqliteSettings
 from transcription.db import create_all
-from transcription.db import get_session
 from transcription.db import initialize_database_runtime
-from transcription.models import Document
-from transcription.models import Job
-from transcription.models import JobStatus
-from transcription.models import Revision
-from transcription.models import Source
+from transcription.db import session_scope
+from transcription.db.models import Document
+from transcription.db.models import Job
+from transcription.db.models import JobStatus
+from transcription.db.models import Revision
+from transcription.db.models import Source
 
 RevisionSeed = str
 
 
 @pytest.fixture(scope="session")
-def app_client(tmp_path_factory: pytest.TempPathFactory) -> tuple[FastAPI, TestClient]:
+def app_client(tmp_path_factory: pytest.TempPathFactory) -> Generator[tuple[FastAPI, TestClient]]:
     """Provide a real application and test client backed by in-memory SQLite."""
     tmp_path = tmp_path_factory.mktemp("ui")
     settings = Settings(
         openrouter_api_key="test-key",
-        database_url="sqlite:///:memory:",
+        database=SqliteSettings(path=":memory:"),
         environment="test",
         bootstrap_schema_on_startup=True,
         upload_dir=tmp_path / "uploads",
         prompt_dir=tmp_path / "prompts",
     )
-    _settings.set(settings)
 
     app = create_app()
     app.state.runtime = initialize_database_runtime(settings=settings)
@@ -54,7 +54,7 @@ def clear_ui_database(app_client: tuple[FastAPI, TestClient]) -> None:
     app, _ = app_client
 
     async def _clear() -> None:
-        async with get_session(session_factory=app.state.runtime.session_factory) as session:
+        async with session_scope() as session:
             await session.exec(delete(Revision))
             await session.exec(delete(Source))
             await session.exec(delete(Job))
@@ -80,7 +80,7 @@ def seed_job(app_client: tuple[FastAPI, TestClient]) -> Callable[..., UUID]:
         source_file: Path | None = None,
     ) -> UUID:
         async def _insert() -> UUID:
-            async with get_session(session_factory=app.state.runtime.session_factory) as session:
+            async with session_scope() as session:
                 stored_path = app.state.settings.upload_dir / filename
                 stored_path.parent.mkdir(parents=True, exist_ok=True)
                 source_path = source_file or fixtures_dir / "small_png.png"

tests/ui/test_jobs_page.py

@@ -5,7 +5,7 @@ from uuid import uuid4
 
 import pytest
 
-from transcription.models import JobStatus
+from transcription.db.models import JobStatus
 
 
 @pytest.mark.integration