Test UI diagrams

docs/Intent.md

@@ -1,4 +1,4 @@
-# Historical Document Transcription
+# Historical Document Transcription Design Intent
 I have several thousand pages of family history told through letters, postcards, books, and other documents that I want to transcribe to text. 
 
 ---
@@ -20,4 +20,5 @@ I have several thousand pages of family history told through letters, postcards,
 
 ## Methodology
 
-See [transcription_methodology.md](transcription_methodology.md) for details on the transcription methodology.
+1. Follow current best practices per "A Guide to Documentary Editing" by Mary-Jo Kline. (See [Transcription Methodology](transcription_methodology.md))
+

docs/architecture.md

@@ -1,130 +0,0 @@
-# Architecture (V1 Baseline)
-
-This document describes the current architecture of the personal historical-document transcription system and serves as the V1 technical baseline.
-
-## Architecture Objectives
-
-- preserve source material as transcribed text
-- keep operational complexity low for personal-scale deployment
-- support asynchronous processing without external queue infrastructure
-- maintain clear module boundaries for incremental extension
-
-## Runtime Topology
-
-V1 runtime is a modular monolith:
-
-- one FastAPI + NiceGUI application process
-- one in-process async worker loop
-- relational persistence via SQLModel (SQLite baseline)
-
-```mermaid
-flowchart LR
-    U[Browser User] --> A[FastAPI + NiceGUI App]
-    A --> W[In-process Worker]
-    A --> DB[(SQLite via SQLModel)]
-    W --> P[OpenRouter Provider]
-    W --> DB
-```
-
-## Lifecycle Ownership
-
-Application lifespan owns runtime setup/teardown:
-
-- configure logging
-- initialize and dispose DB runtime resources
-- optional schema bootstrap by environment policy
-- recover stale processing jobs
-- start/stop worker consumer lifespan
-
-## Layered Module Structure
-
-### Interface Layer
-
-- `src/transcription/ui/**` (NiceGUI pages/components)
-- `src/transcription/api/**` (FastAPI routes and error handlers)
-
-### Application/Workflow Layer
-
-- `src/transcription/services/workflows.py`
-- `src/transcription/worker.py`
-
-Responsibilities:
-
-- orchestration and status transitions
-- retry/timeout behavior
-- provider call coordination
-
-### Service Layer
-
-- `src/transcription/services/*.py`
-
-Responsibilities:
-
-- CRUD and transactional boundaries
-- domain-aligned persistence operations
-
-### Infrastructure Layer
-
-- `src/transcription/db/**` (runtime/session/bootstrap)
-- `src/transcription/providers/**` (OpenRouter adapter)
-
-## Processing Workflow
-
-1. User uploads a source file from the UI.
-2. App persists `Document`, `Job(queued)`, and `Source`.
-3. Worker claims next queued job and marks `processing`.
-4. Worker calls provider with prompt + source bytes.
-5. On success, app writes immutable `Job.text` and marks `transcribed`.
-6. On failure, app writes `Job.error_detail` and marks `failed`.
-7. UI exposes job detail, original transcription, and optional revision.
-
-## Domain Ownership Invariants
-
-- `Job.text` is immutable original provider output.
-- `Revision` is optional, user-authored, and linked to `Source`.
-- `Revision` does not overwrite original job transcription.
-- Status lifecycle is fixed to: `queued -> processing -> transcribed|failed`.
-
-## Data Model Summary
-
-- `Document` has many `Source` and many `Job`.
-- `Source` belongs to one `Document` and one `Job`.
-- `Source` has optional `Revision` (`0..1`) enforced by unique `revision.source_id`.
-
-## Simplicity Guardrails (V1)
-
-- no external queue/broker required
-- no search engine required
-- no distributed worker fleet required
-- keep provider integration behind adapter boundary
-
-## Extension Path
-
-### V1 (current)
-
-- SQLite baseline
-- OpenRouter provider
-- in-process worker
-- optional single revision workflow
-
-### V2 (planned)
-
-- PostgreSQL as relational baseline
-- optional MongoDB adjunct store for scoped use cases
-- migration-first schema evolution
-
-See [ver2/ver2.md](ver2/ver2.md) for roadmap details.
-
-## Test Strategy
-
-- unit tests for model/service behaviors
-- integration tests for upload/workflow reliability
-- UI integration tests for page/render contracts
-- external provider tests opt-in via marker/config
-
-## Related References
-
-- [index.md](index.md)
-- [requirements.md](requirements.md)
-- [schema.md](schema.md)
-- [error_handling.md](error_handling.md)

docs/architecture_v2.md

@@ -0,0 +1,136 @@
+# System Architecture (Version 2)
+
+This document describes the V2 production architecture of the personal historical-document transcription system.
+
+## Architecture Objectives
+
+* Preserve source material as immutable transcribed text alongside page-level spatial AI metadata.
+* Support batching multi-image and folder uploads cleanly into sequential pages (`page_number`).
+* Leverage asynchronous worker pools (`asyncio`) for parallel single-image API execution bounded by rate limiters (`asyncio.Semaphore`).
+* Migrate persistence to PostgreSQL using native `UUID`, `TIMESTAMPTZ`, and `JSONB` document storage.
+* Standardize all data validation, API parsing, and database models on **Pydantic V2**.
+* Support rich historical attribution (multi-author and multi-recipient relationships).
+
+## Runtime Topology
+
+The V2 runtime operates as an asynchronous Python application:
+
+* FastAPI + NiceGUI web application process.
+* In-process `asyncio` background task orchestrator for parallel API execution.
+* Relational persistence via PostgreSQL (using `asyncpg` or `psycopg3`).
+* Pydantic V2 validation layer wrapping API payloads and PostgreSQL `JSONB` schemas.
+
+```mermaid
+flowchart LR
+U[Browser User] --> A[FastAPI + NiceGUI App]
+A --> W[Asyncio Worker Engine]
+A --> DB[(PostgreSQL Database)]
+W --> P[Vision Provider APIs\nOpenAI / Claude]
+W --> DB
+```
+
+## Lifecycle Ownership
+
+Application lifespan owns runtime setup/teardown:
+
+* Initialize environment logging and Pydantic configuration.
+* Manage asynchronous PostgreSQL connection pools (`asyncpg` / `psycopg3`).
+* Execute database migrations and index initialization.
+* Recover stale processing jobs on startup.
+* Manage graceful shutdown of active `asyncio` worker pools.
+
+## Layered Module Structure
+
+### Interface Layer
+
+* `src/transcription/ui/**` (NiceGUI pages, multi-page renderers, person cards)
+* `src/transcription/api/**` (FastAPI routes and JSON error handlers)
+
+### Application & Async Worker Layer
+
+* `src/transcription/services/workflows.py`
+* `src/transcription/worker.py`
+
+Responsibilities:
+
+* Batch orchestration and status transitions (`queued` -> `processing` -> `completed` | `partial_success` | `failed`).
+* Parallel single-image API execution using `asyncio.gather` bounded by `asyncio.Semaphore`.
+* Pydantic schema parsing (`PageAIMetadata`) and validation prior to database storage.
+
+### Domain & Service Layer
+
+* `src/transcription/models/*.py` (Pydantic V2 schemas and entity definitions)
+* `src/transcription/services/*.py` (Transactional operations for `Document`, `Person`, `Source`, `Job`, and `JobSource`)
+
+### Infrastructure Layer
+
+* `src/transcription/db/**` (PostgreSQL connection pooling and raw parameterized SQL execution)
+* `src/transcription/providers/**` (OpenAI & Anthropic Vision SDK adapters)
+
+## Processing Workflow
+
+1. User uploads a folder or batch of images for a `Document`.
+2. System creates `Document`, `Job(status='queued')`, and ordered `Source` pages (`page_number = 1..N`).
+3. Worker claims job, sets `Job.status = 'processing'`, and spawns parallel `asyncio` tasks bounded by semaphore.
+4. Each task calls Vision API for a **single** `Source` image.
+5. On task completion:
+* Writes a `JobSource` record containing `status='transcribed'`, `raw_transcription`, `ai_metadata` (bounding boxes/confidence), and `raw_api_response`.
+* Caches active text to `Source.raw_transcription`.
+
+
+6. On page failure:
+* Writes `JobSource` record with `status='failed'` and `error_detail`.
+
+
+7. Once all page tasks resolve:
+* Marks `Job.status` as `completed` (100% success), `partial_success` (at least 1 success, 1 failure), or `failed` (all failed).
+
+
+
+## Domain Ownership & Invariants
+
+* **Immutable AI Outputs:** `source.raw_transcription` and `job_source.raw_transcription` store original, point-in-time machine output and are immutable.
+* **Inlined Revisions:** Human corrections occur on `source.revised_text`. UI renders `COALESCE(revised_text, raw_transcription)`.
+* **Sequential Integrity:** Multi-page documents are strictly ordered by `source.page_number ASC`.
+* **Page Execution Isolation:** A failure on one page image does not invalidate successful transcriptions on sister pages in the same batch job.
+
+## Data Model Summary
+
+* `Document` has many `Source` pages, many `Job` runs, and many `Person` records via `DocumentPerson` junction (`author` or `recipient`).
+* `Source` belongs to one `Document` and can be processed across many `JobSource` executions.
+* `Job` has many `JobSource` execution records.
+
+## Test Strategy
+
+* Unit tests for Pydantic V2 schemas, custom validators, and JSONB serialization.
+* Integration tests for async PostgreSQL connection handling and parameterized queries.
+* Async workflow tests using mock AI providers to verify `partial_success` and retry logic.
+* UI integration tests for multi-page rendering and person management.
+
+---
+
+## Technology References
+
+- [FastAPI documentation](https://fastapi.tiangolo.com/)
+- [NiceGUI documentation](https://nicegui.io/documentation)
+- [PostgreSQL documentation](https://www.postgresql.org/docs/)
+- [Python asyncio](https://docs.python.org/3/library/asyncio.html#module-asyncio)
+- [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)
+- [Pydantic AI](https://pydantic.dev/docs/ai/overview/)
+ 
+
+## Related Local References
+
+- [System Overview](index_v1.md)
+- [System Design Intent](intent.md)
+- [Transcription Methodology](transcription_methodology.md)
+- System Architecture (this document)
+- [System Requirements](requirements_v1.md)
+- [Data model](schema_v1.md)
+- [Error Handling Policy](error_handling_v1.md)
+- [Implementation Plan](implementation_plan_v1.md)
+
+
+
+
+

docs/archive/README.md

@@ -1,23 +0,0 @@
-# V2 Archive
-
-This folder preserves pre-V1-alignment versions of core documentation that included planned target-state architecture material.
-
-Archived snapshots:
-
-- `index.pre-v1-alignment.md`
-- `requirements.pre-v1-alignment.md`
-- `architecture.pre-v1-alignment.md`
-
-Purpose:
-
-- keep a durable reference for planned architecture language
-- reduce risk of losing useful V2 direction while V1 docs stay implementation-aligned
-
-Notes:
-
-- These files are historical snapshots, not the active V1 source of truth.
-- Active V1 docs remain at:
-  - `docs/index.md`
-  - `docs/requirements.md`
-  - `docs/architecture.md`
-- V2 planning should continue in `docs/ver2/ver2.md` and related V2 artifacts.

docs/ddl_v2.sql

@@ -1,4 +1,4 @@
-## PostgreSQL DDL Specification
+## PostgreSQL DDL Specification (Version 2)
 
 ```sql
 -- Enable pgcrypto for UUID generation if on PostgreSQL < 13

docs/error_handling_v2.md

@@ -0,0 +1,88 @@
+# Error Handling Policy (Version 2)
+
+This document defines the canonical error-handling policy for the V2 document transcription system.
+
+## Error Handling Objectives
+
+* Make failures visible in clear, actionable language at both the document and individual page levels.
+* Support **isolated failure handling** in multi-image batches so single page errors do not crash an entire batch job.
+* Preserve diagnostic detail (Pydantic validation errors, raw provider responses) in PostgreSQL `JSONB` for fast troubleshooting.
+* Ensure consistent error envelope structure across API, UI, and async worker boundaries.
+
+## Scope And Authority
+
+Governs error behavior across NiceGUI pages, FastAPI routes, service orchestration, `asyncio` background tasks, PostgreSQL interactions, and AI provider adapters.
+
+## Error Taxonomy
+
+| Category | Definition | Retriable |
+| --- | --- | --- |
+| `validation_error` | Pydantic payload or parameter schema validation failure | no |
+| `user_input_error` | Unacceptable user file (unsupported image type, corrupt file) | no |
+| `not_found_error` | Requested resource (`Document`, `Source`, `Person`, `Job`) missing | no |
+| `conflict_error` | Operation violates state constraints (e.g., duplicate `document_person` role) | no |
+| `external_provider_error` | AI Provider API failure (rate limit, vision execution error) | yes |
+| `infrastructure_transient_error` | Temporary DB connection reset or HTTP timeout | yes |
+| `infrastructure_persistent_error` | Database down, missing API credentials, misconfiguration | no |
+| `internal_unexpected_error` | Uncaught Python exception or logic defect | no |
+
+## Async Batch & Page-Level Error Behavior
+
+In multi-image `asyncio` batch processing:
+
+1. **Page Isolation:** Exceptions caught during individual page calls are caught within the `asyncio` task wrapper.
+2. **Page Record Logging:** Page failure detail is written directly to `job_source.error_detail` and `job_source.status = 'failed'`.
+3. **Batch Aggregate State:**
+* If **all** page tasks succeed -> `job.status = 'completed'`.
+* If **some** page tasks fail -> `job.status = 'partial_success'`.
+* If **all** page tasks fail -> `job.status = 'failed'`.
+
+
+4. **Retry Strategy:** The UI exposes a "Retry Failed Pages" option for `partial_success` jobs, which spawns a new targeted `Job` containing *only* the `Source` IDs marked as `failed`.
+
+## API Error Response Contract
+
+API error responses return a structured JSON envelope:
+```json
+{
+"error_id": "err_uuid_12345",
+"category": "validation_error",
+"message": "The uploaded payload failed schema validation.",
+"suggestion": "Check file format and metadata fields, then try again.",
+"details": {
+"pydantic_errors": [...]
+},
+"timestamp": "2026-07-31T07:55:00Z"
+}
+```
+
+HTTP Status Mappings:
+
+* `validation_error`, `user_input_error` -> `400`
+* `not_found_error` -> `404`
+* `conflict_error` -> `409`
+* `external_provider_error` -> `502` / `503`
+* `infrastructure_transient_error` -> `503`
+* `infrastructure_persistent_error`, `internal_unexpected_error` -> `500`
+
+---
+
+## Technology References
+
+- [FastAPI documentation](https://fastapi.tiangolo.com/)
+- [NiceGUI documentation](https://nicegui.io/documentation)
+- [PostgreSQL documentation](https://www.postgresql.org/docs/)
+- [Python asyncio](https://docs.python.org/3/library/asyncio.html#module-asyncio)
+- [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)
+- [Pydantic AI](https://pydantic.dev/docs/ai/overview/)
+
+## Related Local References
+
+- [System Overview](index_v2.md)
+- [System Design Intent](intent.md)
+- [Transcription Methodology](transcription_methodology.md)
+- [System Architecture](architecture_v2.md)
+- [System Requirements](requirements_v2.md)
+- [Data model](schema_v2.md)
+- Error Handling Policy (this document)
+- [Implementation Plan](implementation_plan_v2.md)

docs/implementation_plan_v2.md

@@ -0,0 +1,248 @@
+# Implementation Plan (Version 2)
+
+This plan defines the path from the V1 baseline to **Version 2 complete**, aligned to the updated multi-image and multi-person relational domain model:
+
+* `Document` acts as a logical parent container for physical artifacts, supporting multi-author and multi-recipient relationships via `DocumentPerson`.
+* `Source` represents an individual image page within a document, maintaining sequential order (`page_number`), cached active machine output (`raw_transcription`), and inline single user revisions (`revised_text`).
+* `Job` acts as an overarching batch orchestrator for multi-page async processing tasks.
+* `JobSource` records individual point-in-time API executions per image page, storing Pydantic-validated `ai_metadata` and raw REST envelopes (`raw_api_response`).
+* **Pydantic V2** acts as the single source of truth for runtime validation, API payload parsing, and PostgreSQL JSONB serialization.
+
+The objective is to complete the V2 scope with production readiness while keeping non-V2 enhancements out of active delivery.
+
+---
+
+## V2 Completion Definition
+
+V2 is complete when all of the following are true:
+
+1. **Functional complete**
+* Multi-image and whole-folder uploads assign sequential page numbers to `Source` records under a single `Document`.
+* Batch jobs process pages concurrently using an `asyncio` worker pool with semaphore rate limiting.
+* Partial job failures resolve cleanly to `partial_success`, allowing single-page retries without re-running successful pages.
+* Multi-author and multi-recipient tagging is supported on `Document`.
+
+
+2. **Data-model complete**
+* SQLite is fully replaced with PostgreSQL (using `asyncpg` or `psycopg3`).
+* Pydantic V2 models validate all API payloads, database row mappings, and `JSONB` structures.
+
+
+3. **Operational complete**
+* Concurrency controls, worker pool metrics, and database connections operate safely under batch load.
+
+
+4. **Documentation complete**
+* `schema_v2.md`, `DDL_v2.sql`, Pydantic model contracts are updated and consistent.
+
+
+
+---
+
+## Phase 1 — Data Contract Stabilization & Pydantic Baseline
+
+**Goal:** Lock the PostgreSQL schema, DDL, and Pydantic V2 models before refactoring service logic.
+
+### Tasks
+
+1. Finalize DDL for PostgreSQL native types (`UUID`, `TIMESTAMPTZ`, `JSONB`) and junction tables (`document_person`, `job_source`).
+2. Build core Pydantic V2 schemas (`Person`, `Document`, `Source`, `Job`, `JobSource`, `PageAIMetadata`).
+3. Confirm and document data invariants:
+* `source.raw_transcription` and `job_source.raw_transcription` are immutable machine outputs.
+* `source.revised_text` holds user edits. UI renders `COALESCE(revised_text, raw_transcription)`.
+* Page sequence is strictly ordered by `source.page_number ASC`.
+
+
+4. Freeze V2 job status values (`queued`, `processing`, `completed`, `partial_success`, `failed`) and page execution status values (`pending`, `transcribed`, `failed`).
+
+### Deliverables
+
+* Canonical `docs/schema_v2.md` and `docs/DDL_v2.sql`.
+* Centralized Pydantic validation suite in `models/schemas_v2.py`.
+
+### Exit Criteria
+
+* All database tables, relationships, and JSONB structures have corresponding Pydantic V2 models passing unit validation tests.
+
+---
+
+## Phase 2 — Persistence Layer Transition (SQLite to PostgreSQL)
+
+**Goal:** Replace the SQLite storage layer with an asynchronous PostgreSQL driver (`asyncpg` or `psycopg3`).
+
+### Tasks
+
+1. Configure PostgreSQL database connection pooling and environment configuration.
+2. Refactor `services/store.py` / repository layers to execute parameterized async SQL queries (`$1`, `$2`).
+3. Implement JSONB serialization and deserialization helpers using Pydantic's `.model_dump_json()` and `.model_validate()`.
+4. Implement database bootstrap routines for PostgreSQL table creation and index initialization.
+
+### Deliverables
+
+* PostgreSQL-native database connection and query service modules.
+* Integration test suite confirming connection pooling and JSONB CRUD operations.
+
+### Exit Criteria
+
+* All database reads/writes run asynchronously against PostgreSQL with zero remaining SQLite driver dependencies.
+
+---
+
+## Phase 3 — Service Layer & `asyncio` Engine Refactor
+
+**Goal:** Implement batch orchestration and parallel single-image API execution.
+
+### Tasks
+
+1. Refactor upload service to process folder/multi-image input:
+* Group files into a single `Document`.
+* Create ordered `Source` rows (`page_number = 1..N`).
+
+
+2. Refactor `services/workflows.py` with `asyncio` worker pools:
+* Use `asyncio.Semaphore` to enforce API provider rate limits.
+* Issue parallel single-image requests to Vision APIs (OpenAI/Claude).
+* Parse API responses directly into Pydantic models (`PageAIMetadata`).
+
+
+3. Update execution tracking:
+* Create a `JobSource` row per page call to record `raw_transcription`, `ai_metadata`, and `raw_api_response`.
+* Update active `source.raw_transcription` upon task completion.
+* Calculate aggregate batch status (`completed`, `partial_success`, `failed`) on the parent `Job`.
+
+
+4. Refactor `services/person.py` and `services/documents.py` to handle multi-person roles via `document_person`.
+
+### Deliverables
+
+* Asynchronous batch execution engine in `services/workflows.py`.
+* Service routines for multi-person tagging and page-level retries.
+
+### Exit Criteria
+
+* Executing a folder upload of 10+ images processes concurrently, populates page-level `JobSource` entries, and handles partial worker errors without crashing the batch.
+
+---
+
+## Phase 4 — UI & API Contract Alignment
+
+**Goal:** Update API endpoints and frontend/UI views to render multi-page documents and person roles.
+
+### Tasks
+
+1. Update document and job API endpoints to accept batch file arrays and multi-person ID payloads.
+2. Update UI document views:
+* Render multi-page document transcriptions sequentially by `page_number`.
+* Display author and recipient chips/cards linked from `document_person`.
+
+
+3. Update job detail UI to show page-level execution statuses (`transcribed` vs. `failed`) and provide a "Retry Failed Pages" action for `partial_success` jobs.
+4. Align inline page editing controls to update `source.revised_text` and `source.date_revised`.
+
+### Deliverables
+
+* Refactored API routes and UI components supporting multi-page rendering and person management.
+
+### Exit Criteria
+
+* UI successfully displays multi-page document text, allows per-page human revisions, and shows author/recipient metadata.
+
+---
+
+## Phase 5 — Test Suite Realignment & Concurrency Testing
+
+**Goal:** Ensure end-to-end system stability under concurrent async execution and load.
+
+### Tasks
+
+1. Write unit tests for Pydantic models, custom validators, and JSONB conversions.
+2. Write integration tests for async database operations:
+* CRUD for `Document`, `Person`, `DocumentPerson`, `Source`, `Job`, and `JobSource`.
+
+
+3. Write mock-backed async workflow tests:
+* Verify `asyncio.Semaphore` bounds concurrent tasks properly.
+* Validate state transition logic for `completed`, `partial_success`, and `failed` jobs.
+* Confirm retry routines process only targeted `JobSource` records marked as `failed`.
+
+
+4. Re-enable CI quality gates (linting, type checking with Pyright/mypy, pytest).
+
+### Deliverables
+
+* Passing asynchronous test suite covering core workflows, edge cases, and failure recoveries.
+
+### Exit Criteria
+
+* CI pipeline is green with comprehensive coverage across database operations, Pydantic models, and worker queues.
+
+---
+
+## Phase 6 — Reliability, Operations, and Release Readiness
+
+**Goal:** Prepare V2 for production deployment and operator management.
+
+### Tasks
+
+1. Verify structured logging includes `job_id`, `document_id`, `source_id`, and `person_id`.
+2. Tune PostgreSQL connection pool limits and `asyncio` concurrency thresholds for production infrastructure.
+3. Update operational documentation:
+* Review and update `docs/schema_v2.md` as needed.
+* Create `docs/runbook_v2.md` detailing PostgreSQL maintenance, JSONB index management, and worker queue monitoring.
+* Create `docs/release_checklist_v2.md` for launch sign-off.
+
+
+
+### Deliverables
+
+* Updated project documentation and operational runbooks.
+* V2 release sign-off checklist.
+
+### Exit Criteria
+
+* All documentation reflects V2 architecture; launch checklist is fully verified.
+
+---
+
+## Requirement Traceability Focus
+
+Maintain evidence against these V2 requirement groups:
+
+* **Batch & Multi-Image Pipeline:** Folder ingestion, page ordering, async worker execution.
+* **Database & Persistence:** PostgreSQL, native UUIDs, JSONB execution storage, `asyncpg` pooling.
+* **Validation & Schemas:** Pydantic V2 models for DB rows, API requests, and AI vision responses.
+* **Attribution & Metadata:** Multi-author and multi-recipient tagging, biographical entity management.
+* **Error Recovery:** Partial success states, page-level status flags, isolated retry execution.
+
+---
+
+## Scope Discipline Rule (V2 Focus)
+
+* Only tasks required for V2 scope (PostgreSQL, Pydantic V2, folder/async processing, multi-person roles) enter this plan.
+* V3 candidate features (such as side-by-side multi-provider model output comparison) remain strictly in the future backlog.
+* Any schema adjustments during implementation require immediate updates to `DDL_v2.sql`, Pydantic models, and `schema_v2.md`.
+
+---
+
+## Technology References
+
+- [FastAPI documentation](https://fastapi.tiangolo.com/)
+- [NiceGUI documentation](https://nicegui.io/documentation)
+- [PostgreSQL documentation](https://www.postgresql.org/docs/)
+- [Python asyncio](https://docs.python.org/3/library/asyncio.html#module-asyncio)
+- [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)
+- [Pydantic AI](https://pydantic.dev/docs/ai/overview/)
+
+## Related Local References
+
+- [System Overview](index_v2.md)
+- [System Design Intent](intent.md)
+- [Transcription Methodology](transcription_methodology.md)
+- [System Architecture](architecture_v2.md)
+- [System Requirements](requirements_v2.md)
+- [Data model](schema_v2.md)
+- [Error Handling Policy](error_handling_v2.md)
+- Implementation Plan (this document)
+ 
+
+

docs/index.md

@@ -1,57 +0,0 @@
-## Document Transcription System (V1)
-
-This project is a personal-scale application for transcribing and preserving historical family documents.
-
-## Start Here
-
-Read [architecture.md](architecture.md) first.
-
-The architecture page is the primary technical reference for:
-
-- runtime topology and infrastructure assumptions
-- module boundaries and dependency flow
-- processing lifecycle and data ownership
-- test strategy and extension path
-
-## What The Application Does
-
-At a high level, users upload images/PDFs, jobs are processed asynchronously, and users review original transcriptions plus optional revisions.
-
-Core V1 capabilities:
-
-- upload supported source files (`.jpg`, `.jpeg`, `.png`, `.tif`, `.tiff`, `.pdf`)
-- asynchronous job processing with visible status (`queued`, `processing`, `transcribed`, `failed`)
-- immutable original transcription stored on `Job.text`
-- optional single user-authored revision per source (`0..1`)
-- prompt artifacts stored as Markdown files in `prompts/`
-
-## Current Operating Model (V1 Baseline)
-
-- application service: FastAPI + NiceGUI
-- persistence baseline: SQLModel with SQLite
-- worker: in-process async background loop
-- deployment baseline: lightweight Docker Compose app runtime
-
-> Planned persistence evolution (PostgreSQL and optional MongoDB) belongs to V2 planning and is tracked separately.
-
-## Documentation Map
-
-- Architecture and technical design: [architecture.md](architecture.md)
-- V1 runtime and requirement baseline: [requirements.md](requirements.md)
-- Data model and constraints: [schema.md](schema.md)
-- Error handling policy and operational guidance: [error_handling.md](error_handling.md)
-- V1 requirement evidence matrix: [traceability_v1.md](traceability_v1.md)
-- Operations runbook: [runbook.md](runbook.md)
-- V1 migration and rollback guidance: [migration_v1.md](migration_v1.md)
-- V1 release checklist: [release_checklist_v1.md](release_checklist_v1.md)
-- Domain context and transcription policy: [intent.md](intent.md)
-- Transcription methodology: [transcription_methodology.md](transcription_methodology.md)
-- V1 execution plan: [ver1/ver1.md](ver1/ver1.md)
-- V2 roadmap: [ver2/ver2.md](ver2/ver2.md)
-
-## Glossary
-
-- Prompt artifact: a Markdown file containing one transcription prompt.
-- Original transcription: immutable provider output stored on `Job.text`.
-- Revision: optional user-authored text linked to a `Source`.
-- System of record: the authoritative persistent store for canonical application data.

docs/index_v2.md

@@ -0,0 +1,47 @@
+# Document Transcription System Overview (Version 2)
+
+This project is a personal-scale application for transcribing, indexing, and preserving historical family documents, letters, postcards, and journals.
+
+## Start Here
+
+Read [architecture_v2.md](architecture_v2.md) first for technical overview and system design.
+
+## Core V2 Capabilities
+
+* **Folder & Multi-Image Ingestion:** Upload whole folders or image batches that map sequentially (`page_number`) under a single `Document`.
+* **Parallel Async AI Vision Engine:** Concurrently process single-page image transcriptions using Python `asyncio` bounded by rate limiters.
+* **Robust PostgreSQL Storage:** Relational storage for entities with native `UUID`, `TIMESTAMPTZ`, and `JSONB` for deep AI spatial metadata and raw envelopes.
+* **Pydantic V2 Validation:** End-to-end type safety, DB row mapping, and JSONB payload validation.
+* **Historical Person Management:** Track authors and recipients across documents with rich biographical entities (`Person`).
+* **Page-Level Execution Auditing & Revisions:** Store immutable point-in-time machine output per run while enabling inline human corrections (`revised_text`).
+* **Partial Failure Recovery:** Bounded batch execution that isolates single-page API errors (`partial_success`) for simple retries.
+
+## Technical Stack
+
+* **Application Web Framework:** FastAPI + NiceGUI
+* **Persistence Engine:** PostgreSQL 13+
+* **Data Validation & Schemas:** Pydantic V2
+* **Concurrency & Workers:** Python `asyncio` worker pool with `asyncio.Semaphore`
+* **Vision Providers:** OpenAI (GPT-4o) and Anthropic (Claude 3.5 Sonnet) via native SDKs
+
+---
+
+## Technology References
+
+- [FastAPI documentation](https://fastapi.tiangolo.com/)
+- [NiceGUI documentation](https://nicegui.io/documentation)
+- [PostgreSQL documentation](https://www.postgresql.org/docs/)
+- [Python asyncio](https://docs.python.org/3/library/asyncio.html#module-asyncio)
+- [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)
+- [Pydantic AI](https://pydantic.dev/docs/ai/overview/)
+ 
+## Documentation Index
+
+- System Overview (this document)
+- [System Design Intent](intent.md)
+- [Transcription Methodology](transcription_methodology.md)
+- [System Architecture](architecture_v2.md)
+- [System Requirements](requirements_v2.md)
+- [Data model](schema_v2.md)
+- [Error Handling Policy](error_handling_v2.md)
+- [Implementation Plan](implementation_plan_v2.md)

docs/requirements.md

@@ -1,86 +0,0 @@
-## Document Transcription System Requirements (V1 Baseline)
-
-This page captures the **Version 1 baseline requirements** for the currently implemented system. It is the source of truth for V1 acceptance and test traceability.
-
-Forward-looking architecture changes (for example PostgreSQL/Mongo adoption) are intentionally out of this document and should be tracked in a V2 planning/backlog artifact.
-
-## Scope
-
-- System of interest: a single Python application service (NiceGUI + FastAPI) with SQLModel persistence.
-- Runtime/persistence baseline: local-first execution using SQLite (default `sqlite:///./transcription.db`), with Docker Compose support.
-- Primary concern: end-to-end transcription lifecycle from upload through terminal state plus optional single revision editing.
-
-## Requirements Model (Concise Text Form)
-
-### Requirements
-
-| ID | Category | Requirement | Risk | Verify Method |
-| --- | --- | --- | --- | --- |
-| REQ-0 | System | Provide end-to-end document transcription with persistent, inspectable lifecycle state. | medium | demonstration |
-| REQ-1 | Functional | Allow users to upload supported image/PDF files as sources from the web UI. | low | test |
-| REQ-2 | Functional | Process uploads asynchronously and return either original transcription output or explicit failure. | high | test |
-| REQ-3 | Functional | Persist and expose job states: `queued`, `processing`, `transcribed`, `failed`. | high | inspection |
-| REQ-4 | Functional | Persist original provider output (`Job.text`) and failure detail (`Job.error_detail`) for each job. | medium | test |
-| REQ-5 | Interface | Expose API/UI views for status inspection and transcription reading. | medium | demonstration |
-| REQ-6 | Performance | Trigger background processing on upload to preserve UI responsiveness. | medium | analysis |
-| REQ-7 | Design Constraint | Keep lifespan-owned runtime resources (engine/session factory/worker resources) initialized and disposed at application boundaries. | medium | inspection |
-| REQ-8 | Design Constraint | Initialize configuration and logging once at startup through centralized mechanisms. | low | inspection |
-| REQ-9 | Design Constraint | Support containerized app runtime via Docker Compose using the same V1 persistence model. | medium | demonstration |
-| REQ-10 | Design Constraint | Keep schema bootstrap explicit and opt-in for production safety. | high | inspection |
-| REQ-11 | Design Constraint | Route persistence changes through service/workflow orchestration boundaries. | medium | inspection |
-| REQ-12 | Design Constraint | Store transcription prompts as individual Markdown artifacts for iterative refinement. | medium | inspection |
-| REQ-13 | Functional | Allow users to create/update one optional revision derived from the original job transcription and view/delete it from the job detail flow. | low | test |
-
-### Requirement Relationships
-
-- Contains: REQ-0 contains REQ-1 through REQ-13.
-- Derives: REQ-2 -> REQ-3, REQ-3 -> REQ-4.
-- Traces: REQ-5 -> REQ-3.
-- Refines: REQ-6 -> REQ-2.
-
-### Architecture Elements
-
-| Element | Type | Doc Reference |
-| --- | --- | --- |
-| UI | NiceGUI pages/components | `src/transcription/ui/pages`, `src/transcription/ui/components` |
-| API | FastAPI routes and handlers | `src/transcription/api`, `src/transcription/app.py` |
-| WORKER | Async queued-job processing workflow | `src/transcription/worker.py`, `src/transcription/services/workflows.py` |
-| DBREL | SQLModel relational persistence (SQLite in V1 baseline) | `src/transcription/models.py`, `src/transcription/db` |
-| SERVICES | Service-layer persistence orchestration | `src/transcription/services` |
-| OPS | Containerized runtime baseline | `docker-compose.yml`, `Dockerfile` |
-| PROMPTS | Transcription prompt artifacts | `prompts/` |
-| TESTS | Pytest verification suite | `tests/` |
-
-### Satisfaction Mapping
-
-- UI satisfies REQ-1, REQ-5, REQ-13.
-- API satisfies REQ-5.
-- WORKER satisfies REQ-2, REQ-6.
-- DBREL satisfies REQ-3, REQ-4, REQ-10, REQ-13.
-- SERVICES satisfies REQ-4, REQ-11.
-- OPS satisfies REQ-9.
-- PROMPTS satisfies REQ-12.
-
-### Verification Mapping
-
-- TESTS verifies REQ-1, REQ-2, REQ-3, REQ-4, REQ-5, REQ-10, REQ-11, REQ-12, REQ-13.
-
-## Requirement Notes
-
-- Requirement IDs (`REQ-*`) are stable references for planning, implementation, and traceability.
-- This document is intentionally **implementation-aligned** for V1 completion and release sign-off.
-- Planned storage evolution (PostgreSQL and optional MongoDB) is a **V2 concern** and should be tracked outside this V1 baseline.
-
-## Verification Intent
-
-- Demonstration: validate end-to-end behavior through operator-visible flows.
-- Inspection: verify architecture and startup/runtime policies in code and configuration.
-- Analysis: evaluate asynchronous execution behavior and design sufficiency.
-- Test: automate behavioral checks through pytest suites and service/UI integration tests.
-
-## Glossary
-
-- Original transcription: immutable provider output stored on `Job.text`.
-- Revision: optional user-authored editable text tied to a `Source` (`0..1` in V1).
-- Prompt artifact: a Markdown file containing instructions used for transcription.
-- System of record: the authoritative relational store for canonical V1 data.

docs/requirements_v2.md

@@ -0,0 +1,42 @@
+# Document Transcription System Requirements (Version 2)
+
+This document captures the **Version 2 baseline requirements** for the production implementation.
+
+## Requirements Model
+
+| ID | Category | Requirement | Verify Method |
+| --- | --- | --- | --- |
+| REQ-0 | System | Provide end-to-end multi-page document transcription with persistent, inspectable async job states. | demonstration |
+| REQ-1 | Functional | Allow users to upload folders or multi-image batches as sequential `Source` pages under a `Document`. | test |
+| REQ-2 | Functional | Process multi-page jobs asynchronously using an `asyncio` worker pool bounded by rate limits. | test |
+| REQ-3 | Functional | Persist page-level execution outputs (`raw_transcription`, `ai_metadata`, `raw_api_response`) on `JobSource`. | test |
+| REQ-4 | Functional | Support job states (`queued`, `processing`, `completed`, `partial_success`, `failed`) and page states (`pending`, `transcribed`, `failed`). | inspection |
+| REQ-5 | Functional | Allow users to manage historical `Person` records and link multiple authors/recipients to a `Document` via `DocumentPerson`. | test |
+| REQ-6 | Functional | Maintain immutable original machine output on `Source.raw_transcription` while permitting inline human edits on `Source.revised_text`. | test |
+| REQ-7 | Data Constraint | Store all persistent domain data in PostgreSQL using native `UUID`, `TIMESTAMPTZ`, and `JSONB` columns. | inspection |
+| REQ-8 | Data Constraint | Validate all API requests, database rows, and JSONB structures using Pydantic V2 schemas. | test |
+| REQ-9 | Interface | Render multi-page transcriptions sequentially by `page_number` in the web UI with author/recipient metadata. | demonstration |
+| REQ-10 | Operations | Allow operators to retry only failed pages for jobs in a `partial_success` state. | test |
+
+## Element Satisfaction Mapping
+
+* **UI (NiceGUI):** Satisfies REQ-1, REQ-5, REQ-6, REQ-9, REQ-10.
+* **API (FastAPI):** Satisfies REQ-1, REQ-4, REQ-5, REQ-8.
+* **WORKER (asyncio):** Satisfies REQ-2, REQ-3, REQ-4, REQ-10.
+* **PERSISTENCE (PostgreSQL):** Satisfies REQ-3, REQ-6, REQ-7.
+* **MODELS (Pydantic V2):** Satisfies REQ-8.
+
+---
+
+## Related Local References
+
+- [System Overview](index_v2.md)
+- [System Design Intent](intent.md)
+- [Transcription Methodology](transcription_methodology.md)
+- [System Architecture](architecture_v2.md)
+- System Requirements (this document)
+- [Data model](schema_v2.md)
+- [Error Handling Policy](error_handling_v2.md)
+- [Implementation Plan](implementation_plan_v2.md)
+
+

docs/schema_v2.md

@@ -1,4 +1,4 @@
-# Database Schema (V2 Architecture)
+# Database Schema (Version 2)
 
 This document describes the PostgreSQL relational schema for the transcription platform. It incorporates multi-image batch orchestration via `asyncio`, page-level execution tracking, many-to-many author/recipient attribution, and JSONB document storage for AI vision outputs.
 
@@ -120,3 +120,17 @@ erDiagram
 
 * Multi-Person Roles: Documents support zero, one, or many authors and recipients linked via document_person.
 * Role Uniqueness: (document_id, person_id, role) must be unique to prevent duplicate role tagging.
+
+---
+
+## Related Local References
+
+- [System Overview](index_v2.md)
+- [System Design Intent](intent.md)
+- [Transcription Methodology](transcription_methodology.md)
+- [System Architecture](architecture_v2.md)
+- [System Requirements](requirements_v2.md)
+- Data model (this document)
+- [Error Handling Policy](error_handling_v2.md)
+- [Implementation Plan](implementation_plan_v2.md)
+

docs/test_block-beta.md

@@ -0,0 +1,92 @@
+```mermaid
+block-beta
+  columns 3
+
+  %% UI Component Column
+  block:UI["UI COMPONENTS / WIREFRAME"]:1
+    columns 1
+    
+    block:HeaderUI["Header & Nav"]:1
+      columns 1
+      h_title["[Text] Document Name & Type"]
+      h_date["[Text] Date & Origin Location"]
+    end
+
+    block:EditorUI["Page Transcription Editor"]:1
+      columns 1
+      ed_img["[Image Viewer] Source Image"]
+      ed_page["[Badge] Page Number"]
+      ed_raw["[Read-Only] AI Raw Output"]
+      ed_rev["[Textarea] Human Revised Text"]
+    end
+
+    block:PeopleUI["Attribution Sidebar"]:1
+      columns 1
+      p_author["[List] Authors (Full Name)"]
+      p_recip["[List] Recipients (Full Name)"]
+      p_bio["[Card] Person Biography & Dates"]
+    end
+
+    block:JobUI["AI Processing Drawer"]:1
+      columns 1
+      j_status["[Badge] Job Status"]
+      j_model["[Text] Provider & Model"]
+      j_tokens["[JSON View] AI Token Usage"]
+    end
+  end
+
+  %% Directional Mapping / Connectors
+  block:FLOW["MAPPING / FLOW"]:1
+    columns 1
+    f1["Reads / Updates -->"]
+    f2["Renders Active Page -->"]
+    f3["Joins via Role -->"]
+    f4["Executes & Logs -->"]
+  end
+
+  %% Postgres Schema Column
+  block:DB["POSTGRES SQL SCHEMA"]:1
+    columns 1
+
+    block:DocTbl["Table: document"]:1
+      columns 1
+      d_id["id : UUID (PK)"]
+      d_name["name : TEXT"]
+      d_type["document_type : TEXT"]
+      d_date["document_date : DATE"]
+    end
+
+    block:SrcTbl["Table: source"]:1
+      columns 1
+      s_id["id : UUID (PK)"]
+      s_page["page_number : INT"]
+      s_path["file_path : TEXT"]
+      s_raw["raw_transcription : TEXT"]
+      s_rev["revised_text : TEXT"]
+    end
+
+    block:PersonTbl["Table: person & document_person"]:1
+      columns 1
+      p_id["id : UUID (PK)"]
+      p_name["full_name : TEXT"]
+      p_role["role : 'author' | 'recipient'"]
+    end
+
+    block:JobTbl["Table: job & job_source"]:1
+      columns 1
+      j_id["id : UUID (PK)"]
+      j_stat["status : VARCHAR"]
+      j_prov["provider / model : TEXT"]
+      j_meta["ai_metadata : JSONB"]
+    end
+  end
+
+  %% Connections
+  HeaderUI --> DocTbl
+  ed_img --> s_path
+  ed_page --> s_page
+  ed_raw --> s_raw
+  ed_rev --> s_rev
+  PeopleUI --> PersonTbl
+  JobUI --> JobTbl
+```

docs/test_flowchart.md

@@ -0,0 +1,53 @@
+```mermaid
+flowchart LR
+    subgraph UI["UI Components / Wireframe"]
+        direction TB
+        subgraph HeaderUI["Header & Nav"]
+            h_title["[Text] Document Name & Type"]
+            h_date["[Text] Date & Origin Location"]
+        end
+        subgraph EditorUI["Page Transcription Editor"]
+            ed_img["[Image Viewer] Source Image"]
+            ed_page["[Badge] Page Number"]
+            ed_raw["[Read-Only] AI Raw Output"]
+            ed_rev["[Textarea] Human Revised Text"]
+        end
+        subgraph PeopleUI["Attribution Sidebar"]
+            p_author["[List] Authors / Recipients"]
+        end
+        subgraph JobUI["AI Processing Drawer"]
+            j_status["[Badge] Job Status"]
+        end
+    end
+
+    subgraph DB["Postgres SQL Schema"]
+        direction TB
+        subgraph DocTbl["Table: document"]
+            d_name["name : TEXT"]
+            d_type["document_type : TEXT"]
+        end
+        subgraph SrcTbl["Table: source"]
+            s_path["file_path : TEXT"]
+            s_page["page_number : INT"]
+            s_raw["raw_transcription : TEXT"]
+            s_rev["revised_text : TEXT"]
+        end
+        subgraph PersonTbl["Table: person & document_person"]
+            p_name["full_name : TEXT"]
+            p_role["role : author | recipient"]
+        end
+        subgraph JobTbl["Table: job & job_source"]
+            j_stat["status : VARCHAR"]
+            j_meta["ai_metadata : JSONB"]
+        end
+    end
+
+    %% Mappings
+    HeaderUI --> DocTbl
+    ed_img --> s_path
+    ed_page --> s_page
+    ed_raw --> s_raw
+    ed_rev --> s_rev
+    PeopleUI --> PersonTbl
+    JobUI --> JobTbl
+```

docs/ver1/architecture_v1.md

@@ -1,4 +1,4 @@
- # Architecture
+ # System Architecture (Version 1)
 
 This document describes the production architecture of the personal historical-document transcription system. The system is intentionally optimized for single-user operation, low operational overhead, and clean internal boundaries that support future growth without rewrites.
 
@@ -265,6 +265,8 @@ Control:
 
 - first-class human review and immutable revision history
 
+---
+
 ## Technology References
 
 - [FastAPI documentation](https://fastapi.tiangolo.com/)
@@ -275,7 +277,14 @@ Control:
 
 ## Related Local References
 
-- [System overview](index.md)
+- [System Overview](index_v1.md)
+- [System Design Intent](intent.md)
+- [Transcription Methodology](transcription_methodology.md)
+- System Architecture (this document)
+- [System Requirements](requirements_v1.md)
+- [Data model](schema_v1.md)
+- [Error Handling Policy](error_handling_v1.md)
+- [Implementation Plan](implementation_plan_v1.md)
 
 ## Glossary
 

docs/ver1/error_handling_v1.md

@@ -1,4 +1,4 @@
-# Error Handling
+# Error Handling Policy
 
 This document defines the canonical error-handling policy for the document transcription system. It is the single source of truth for how errors are classified, surfaced to users, logged for diagnosis, and handled across UI, API, service, worker, and provider boundaries.
 
@@ -266,12 +266,18 @@ Change requirements:
 - preserve taxonomy stability; if changed, document migration impact
 - record noteworthy policy changes in project release notes or changelog
 
-## Related Pages
+---
 
-- [System overview](index.md)
-- [Architecture](architecture.md)
-- [Requirements](requirements.md)
-- [Intent](intent.md)
+## Related Local References
+
+- [System Overview](index_v1.md)
+- [System Design Intent](intent.md)
+- [Transcription Methodology](transcription_methodology.md)
+- [System Architecture](architecture_v1.md)
+- [System Requirements](requirements_v1.md)
+- [Data model](schema_v1.md)
+- Error Handling Policy (this document)
+- [Implementation Plan](implementation_plan_v1.md)
 
 ## Glossary
 

docs/ver1/implementation_plan_v1.md

@@ -44,7 +44,7 @@ V1 is complete when all of the following are true:
 4. Freeze V1 status lifecycle to current implementation (`queued`, `processing`, `transcribed`, `failed`).
 
 ### Deliverables
-- Updated `docs/schema.md` and `docs/requirements.md` traceability alignment.
+- Updated `schema_v1.md` and `requirements.md` traceability alignment.
 - Explicit V1 data invariants section in architecture docs.
 
 ### Exit Criteria
@@ -150,9 +150,8 @@ V1 is complete when all of the following are true:
 
 ### Deliverables
 - V1 release checklist and acceptance evidence.
-- `docs/runbook.md` for incident response and operator workflows.
-- `docs/migration_v1.md` for V1 migration/backfill/rollback guidance.
-- `docs/release_checklist_v1.md` for release sign-off.
+- `runbook_v1.md` for incident response and operator workflows.
+- `release_checklist_v1.md` for release sign-off.
 
 ### Exit Criteria
 - Stakeholder sign-off and launch readiness achieved.
@@ -188,3 +187,17 @@ A lightweight traceability table should be maintained with:
 - Only work required to satisfy V1 requirements enters this plan.
 - Nice-to-have enhancements are captured in a separate backlog document.
 - Schema or contract changes after Phase 1 require explicit approval and traceability impact review.
+
+---
+
+## Related Local References
+
+- [System Overview](index_v1.md)
+- [System Design Intent](intent.md)
+- [Transcription Methodology](transcription_methodology.md)
+- [System Architecture](architecture_v1.md)
+- [System Requirements](requirements_v1.md)
+- [Data model](schema_v1.md)
+- [Error Handling Policy](error_handling_v1.md)
+- Implementation Plan (this document)
+

docs/ver1/index_v1.md

@@ -1,10 +1,10 @@
-## Document Transcription System
+## Document Transcription System Overview
 
 This project is a production application for transcribing and preserving historical family documents. It is intentionally designed for personal-scale use, with a simplicity-first architecture that is easy to operate and easy to extend.
 
 ## Start Here
 
-Read [architecture.md](architecture.md) first.
+Read [architecture_v1.md](architecture_v1.md) first.
 
 The architecture page is the primary technical reference and defines:
 
@@ -17,7 +17,7 @@ The architecture page is the primary technical reference and defines:
 
 At a high level, users upload images or PDFs as content sources for handwritten, typed, or typeset documents, run asynchronous transcription jobs, review optional revisions, and search across accepted text.
 
-Core capabilities:
+### Core capabilities:
 
 - document grouping with one or more content sources and metadata capture
 - asynchronous transcription with visible job status
@@ -38,16 +38,18 @@ The system runs with minimal operational overhead:
 
 This operating model keeps deployment and maintenance simple while preserving clean boundaries for future scale.
 
+---
+
 ## Documentation Map
 
-- Architecture and technical design: [architecture.md](architecture.md)
-- Runtime and deployment requirements: [requirements.md](requirements.md)
-- Error handling policy and operational guidance: [error_handling.md](error_handling.md)
-- Domain context and transcription policy: [intent.md](intent.md)
-- Transcription Methodology: [transcription_methodology.md](transcription_methodology.md)
-- Data model: [schema.md](schema.md)
-
-
+- System Overview (this document)
+- [System Design Intent](intent.md)
+- [Transcription Methodology](transcription_methodology.md)
+- [System Architecture](architecture_v1.md)
+- [System Requirements](requirements_v1.md)
+- [Data model](schema_v1.md)
+- [Error Handling Policy](error_handling_v1.md)
+- [Implementation Plan](implementation_plan_v1.md)
 
 ## Glossary
 

docs/ver1/migration_v1.md

@@ -1,92 +0,0 @@
-# V1 Data Migration and Recovery Guidance
-
-This document defines migration/backfill and rollback guidance for the V1 SQLite baseline.
-
-## Purpose
-
-- provide safe procedures for local schema evolution and recovery
-- reduce data-loss risk during version upgrades
-- establish repeatable pre-change and post-change checks
-
-## Current Baseline
-
-- canonical relational store: SQLite
-- default DB path: `./transcription.db`
-- schema bootstrap may apply compatibility updates for dev/test scenarios
-
-## Pre-Change Checklist
-
-Before changing runtime version or schema behavior:
-
-1. Stop the app process.
-2. Create a timestamped DB backup copy.
-3. Capture current app commit/version.
-4. Export a quick status inventory:
-   - job counts by status
-   - total documents/sources/revisions
-5. Ensure sufficient disk space.
-
-## Backup Procedure (SQLite)
-
-Minimum procedure:
-
-1. Stop app.
-2. Copy DB file to a safe location with timestamp.
-3. Store backup path in release notes or change log.
-
-## Upgrade Procedure (V1)
-
-1. Perform pre-change checklist.
-2. Deploy updated app version.
-3. Start app and observe startup logs.
-4. Verify schema bootstrap completes (if enabled).
-5. Run smoke flow:
-   - upload valid file
-   - observe terminal status
-   - open job detail
-
-## Backfill Guidance
-
-V1 backfill is limited and conservative:
-
-- for records missing newly introduced non-null defaults, use explicit one-time SQL updates only after backup
-- avoid destructive rewrites of `Job.text` or `Revision.text`
-- never backfill by overwriting original immutable transcription output
-
-## Rollback Procedure
-
-If upgrade fails or causes data inconsistency:
-
-1. Stop app.
-2. Restore prior DB backup file.
-3. Revert app version to last known-good commit.
-4. Restart app.
-5. Run smoke flow and confirm stability.
-
-## Recovery Scenarios
-
-### Stale processing jobs after crash/restart
-
-- restart app and allow stale-job recovery to re-queue timed-out `processing` jobs
-- monitor for terminal progression
-
-### Schema mismatch symptoms
-
-- errors during startup or writes indicating missing columns/indexes
-- rollback to last good DB + app version
-- reattempt with documented upgrade path
-
-## Validation Evidence
-
-For each upgrade rehearsal, capture:
-
-- backup filename/path
-- pre and post job status counts
-- smoke test result
-- rollback rehearsal result (recommended)
-
-## Operational Constraints
-
-- treat DB backups as required before non-trivial upgrades
-- do not perform in-place DB edits while app is running
-- do not skip post-upgrade smoke validation

docs/ver1/release_checklist_v1.md

@@ -18,8 +18,8 @@ Use this checklist before declaring V1 operationally complete.
 
 ## C) Operational Readiness
 
-- [ ] `docs/runbook.md` reviewed and current.
-- [ ] `docs/migration_v1.md` reviewed and current.
+- [ ] `runbook_v1.md` reviewed and current.
+- [ ] `migration_v1.md` reviewed and current.
 - [ ] Backup and rollback procedures tested at least once.
 - [ ] Incident escalation packet template is known to operators.
 
@@ -28,14 +28,14 @@ Use this checklist before declaring V1 operationally complete.
 - [ ] Lint/type checks pass.
 - [ ] `pytest -m "not external" -q` passes.
 - [ ] Targeted external/provider checks executed (if credentials available).
-- [ ] Release evidence recorded in `docs/release_evidence_v1.md`.
+- [ ] Release evidence recorded in `release_evidence_v1.md`.
 
 ## E) Traceability and Documentation
 
-- [ ] `docs/requirements.md` aligns with implemented V1 behavior.
-- [ ] `docs/architecture.md`, `docs/schema.md`, and `docs/error_handling.md` are consistent.
-- [ ] `docs/traceability_v1.md` is updated with current implementation and test evidence.
-- [ ] `docs/ver1/ver1.md` phase status updated with evidence references.
+- [ ] `requirements_v1.md` aligns with implemented V1 behavior.
+- [ ] `architecture_v1.md`, `schema_v1.md`, and `error_handling_v1.md` are consistent.
+- [ ] `traceability_v1.md` is updated with current implementation and test evidence.
+- [ ] `implementation_plan_v1.md` phase status updated with evidence references.
 - [ ] REQ traceability evidence links recorded (tests/runbook/checks).
 
 ## Release Sign-Off

docs/ver1/requirements_v1.md

@@ -1,6 +1,6 @@
 ## Document Transcription System Requirements
 
-This page captures a SysML v1.6-style requirements baseline for the production system described in [index.md](index.md). The model is represented as concise tables and traceability lists that preserve SysML-style IDs and relationship semantics.
+This page captures a SysML v1.6-style requirements baseline for the production system described in [index_v1.md](index_v1.md). The model is represented as concise tables and traceability lists that preserve SysML-style IDs and relationship semantics.
 
 ## Scope
 
@@ -77,6 +77,19 @@ This page captures a SysML v1.6-style requirements baseline for the production s
 - Analysis: evaluate asynchronous execution behavior and design sufficiency.
 - Test: automate behavioral checks through pytest suites and service-level tests.
 
+---
+
+## Related Local References
+
+- [System Overview](index_v1.md)
+- [System Design Intent](intent.md)
+- [Transcription Methodology](transcription_methodology.md)
+- [System Architecture](architecture_v1.md)
+- System Requirements (this document)
+- [Data model](schema_v1.md)
+- [Error Handling Policy](error_handling_v1.md)
+- [Implementation Plan](implementation_plan_v1.md)
+
 ## Glossary
 
 - Document-oriented persistence: A storage approach that uses flexible document structures for variable data shapes.

docs/ver1/schema_v1.md

@@ -79,6 +79,17 @@ erDiagram
 
 ---
 
+## Related Local References
+
+- [System Overview](index_v1.md)
+- [System Design Intent](intent.md)
+- [Transcription Methodology](transcription_methodology.md)
+- [System Architecture](architecture_v1.md)
+- [System Requirements](requirements_v1.md)
+- Data model (this document)
+- [Error Handling Policy](error_handling_v1.md)
+- [Implementation Plan](implementation_plan_v1.md)
+
 ## Glossary
 
 - **Document**: logical grouping for one or more transcribed sources.

docs/ver1/traceability_v1.md

@@ -21,7 +21,7 @@ Status values:
 | REQ-6 | done | Background processing trigger/worker notifier and non-blocking workflow in `src/transcription/ui/components/upload.py`, `src/transcription/worker.py` | `tests/test_app.py`, `tests/services/test_workflows_reliability.py` |
 | REQ-7 | done | Lifespan-owned runtime resources in `src/transcription/app.py`, `src/transcription/db/runtime.py` | `tests/test_app.py`, `tests/test_db.py` |
 | REQ-8 | done | Centralized settings/logging initialization in `src/transcription/config.py`, `src/transcription/app.py` | `tests/test_config.py`, `tests/test_app.py` |
-| REQ-9 | done | Containerized runtime baseline in `docker-compose.yml`, `Dockerfile` | `docs/release_checklist_v1.md` (Ops checklist), manual demonstration step |
+| REQ-9 | done | Containerized runtime baseline in `docker-compose.yml`, `Dockerfile` | `release_checklist_v1.md` (Ops checklist), manual demonstration step |
 | REQ-10 | done | Explicit schema bootstrap policy + runtime controls in `src/transcription/config.py`, `src/transcription/app.py`, `src/transcription/db/operations.py` | `tests/test_db.py`, `tests/test_config.py` |
 | REQ-11 | done | Service/workflow persistence boundaries in `src/transcription/services/*.py`, `src/transcription/services/workflows.py` | `tests/services/test_job_service.py`, `tests/services/test_transcription_service.py` |
 | REQ-12 | done | Prompt artifacts in `prompts/` and loading/validation in `src/transcription/services/transcription.py` | `tests/test_prompts.py` |
@@ -29,9 +29,9 @@ Status values:
 
 ## Operational Evidence (Step 3 Artifacts)
 
-- Runbook: `docs/runbook.md`
-- Migration/backfill/rollback guidance: `docs/migration_v1.md`
-- Release readiness checklist: `docs/release_checklist_v1.md`
+- Runbook: `runbook_v1.md`
+- Migration/backfill/rollback guidance: `migration_v1.md`
+- Release readiness checklist: `release_checklist_v1.md`
 
 ## Verification Cadence
 

docs/ver2/V2 Context for DB Updates.md

@@ -1,35 +0,0 @@
-# AI Coding Assistant Project Briefing & Context
-
-## Project Mission
-This application is a family history archival and transcription platform. Its primary goal is to accept scanned document images (letters, postcards, logbooks, diaries), execute OCR and structured transcription via AI vision models (OpenAI GPT-4o, Anthropic Claude 3.5 Sonnet), and manage historical metadata (authors, recipients, dates, and locations).
-
----
-
-## Technical Stack & Architecture
-* **Database:** PostgreSQL 13+ with native `UUID` (`gen_random_uuid()`) and `JSONB` columns.
-* **Backend Runtime / Concurrency:** Python utilizing `asyncio` for concurrent HTTP API calls to AI providers, with strict rate-limiting via `asyncio.Semaphore`.
-* **Validation & Types:** TypeScript with **Zod** schema definitions. Incoming AI responses must be parsed and validated with Zod schemas *before* database insertion.
-* **ORM / Database Access:** Raw parameterized SQL queries or lightweight query builders (e.g., Kysely/Prisma) respecting PostgreSQL native types.
-
----
-
-## Core System Directives for AI Code Generation
-
-### 1. Data Immutability vs. Human Corrections
-* `job_source.raw_transcription` and `source.raw_transcription` represent original, point-in-time machine outputs and are **immutable**.
-* Human corrections occur on `source.revised_text`.
-* When fetching text for the UI, always display `COALESCE(source.revised_text, source.raw_transcription)`.
-
-### 2. Async Execution & Batching Rules
-* A `job` represents an overarching execution run for a folder/group of images belonging to a single `document`.
-* Images are submitted to AI APIs **one at a time in rapid succession** using `asyncio` worker pools.
-* Each single-image API call populates a row in `job_source` with its own `status`, `raw_transcription`, `ai_metadata`, and `raw_api_response`.
-* If 9 of 10 pages succeed and 1 fails, `job_source.status` for the failed image becomes `'failed'`, while `job.status` becomes `'partial_success'`. Do not mark the entire batch as failed if partial results exist.
-
-### 3. Entity Relationships
-* **Authors/Recipients:** A `document` can have multiple authors and recipients. Do NOT put direct `author_id` foreign keys on `document`. Query authors/recipients via `document_person` where `role = 'author'` or `role = 'recipient'`.
-* **Page Ordering:** Multi-page documents must always be queried using `ORDER BY page_number ASC`.
-
-### 4. Database Mutations
-* Always use parameterized SQL queries (`$1`, `$2`) to prevent SQL injection.
-* Store datetimes using UTC ISO 8601 strings or native PostgreSQL `TIMESTAMPTZ`.

docs/ver2/V2 TypeScript Zod Schemas.md

@@ -1,169 +0,0 @@
-##  TypeScript Zod schemas
-
-Here are the TypeScript Zod schemas matching your V2 PostgreSQL database definition.
-
-These schemas cover:
-1. Database Entities: Pure runtime validators representing rows fetched directly from PostgreSQL.
-2. AI Payload Extensions: The structured document output stored inside job.ai_metadata.
-3. Insert/Create Schemas: Utility types derived with .omit() for creating new records where auto-generated columns (id, created_at, updated_at, etc.) are handled by PostgreSQL defaults.
-
----
-
-```
-import { z } from "zod";
-
-// ==========================================
-// 1. ATOMIC & REUSABLE SCHEMAS
-// ==========================================
-
-export const UUIDSchema = z.string().uuid();
-export const ISODateTimeSchema = z.coerce.date();
-
-export const BoundingBoxSchema = z.object({
-  ymin: z.number().min(0).max(1000),
-  xmin: z.number().min(0).max(1000),
-  ymax: z.number().min(0).max(1000),
-  xmax: z.number().min(0).max(1000),
-});
-
-export const BlockTypeSchema = z.enum([
-  "heading",
-  "paragraph",
-  "table",
-  "margin_note",
-  "signature",
-  "footnote",
-  "header",
-]);
-
-// ==========================================
-// 2. PAGE-LEVEL AI METADATA SCHEMA (job_source.ai_metadata)
-// ==========================================
-
-export const TranscribedBlockSchema = z.object({
-  text: z.string(),
-  confidence: z.number().min(0).max(1),
-  blockType: BlockTypeSchema,
-  boundingBox: BoundingBoxSchema.optional(),
-});
-
-export const PageAIMetadataSchema = z.object({
-  detectedLanguage: z.string().optional(),
-  overallConfidence: z.number().min(0).max(1),
-  blocks: z.array(TranscribedBlockSchema),
-  inputTokens: z.number().optional(),
-  outputTokens: z.number().optional(),
-  extractedEntities: z.record(z.string(), z.unknown()).optional(),
-});
-
-export type PageAIMetadata = z.infer<typeof PageAIMetadataSchema>;
-
-// ==========================================
-// 3. TABLE ENTITY SCHEMAS
-// ==========================================
-
-// --- PERSON TABLE ---
-export const PersonSchema = z.object({
-  id: UUIDSchema,
-  fullName: z.string().min(1),
-  displayName: z.string().nullable().optional(),
-  maidenName: z.string().nullable().optional(),
-  birthDate: z.string().nullable().optional(),
-  birthDateRaw: z.string().nullable().optional(),
-  birthPlace: z.string().nullable().optional(),
-  deathDate: z.string().nullable().optional(),
-  deathDateRaw: z.string().nullable().optional(),
-  deathPlace: z.string().nullable().optional(),
-  biography: z.string().nullable().optional(),
-  portraitPath: z.string().nullable().optional(),
-  metadata: z.record(z.string(), z.unknown()).default({}),
-  createdAt: ISODateTimeSchema,
-  updatedAt: ISODateTimeSchema,
-});
-
-// --- DOCUMENT TABLE ---
-export const DocumentSchema = z.object({
-  id: UUIDSchema,
-  name: z.string().min(1),
-  documentType: z.string().nullable().optional(),
-  documentDate: z.string().nullable().optional(),
-  documentDateRaw: z.string().nullable().optional(),
-  locationCreated: z.string().nullable().optional(),
-  notes: z.string().nullable().optional(),
-  archiveIdentifier: z.string().nullable().optional(),
-  createdAt: ISODateTimeSchema,
-  updatedAt: ISODateTimeSchema,
-});
-
-// --- DOCUMENT_PERSON JUNCTION ---
-export const PersonRoleSchema = z.enum(["author", "recipient"]);
-
-export const DocumentPersonSchema = z.object({
-  id: UUIDSchema,
-  documentId: UUIDSchema,
-  personId: UUIDSchema,
-  role: PersonRoleSchema,
-  createdAt: ISODateTimeSchema,
-});
-
-// --- JOB TABLE ---
-export const JobStatusSchema = z.enum([
-  "queued",
-  "processing",
-  "completed",
-  "partial_success",
-  "failed",
-]);
-
-export const JobSchema = z.object({
-  id: UUIDSchema,
-  documentId: UUIDSchema,
-  status: JobStatusSchema.default("queued"),
-  retryCount: z.number().int().nonnegative().default(0),
-  provider: z.string(),
-  model: z.string(),
-  promptName: z.string().nullable().optional(),
-  dateCreated: ISODateTimeSchema,
-  dateUpdated: ISODateTimeSchema,
-});
-
-// --- SOURCE TABLE ---
-export const SourceSchema = z.object({
-  id: UUIDSchema,
-  documentId: UUIDSchema,
-  pageNumber: z.number().int().positive().default(1),
-  uploadName: z.string(),
-  filename: z.string(),
-  filePath: z.string(),
-  rawTranscription: z.string().nullable().optional(),
-  revisedText: z.string().nullable().optional(),
-  dateUploaded: ISODateTimeSchema,
-  dateRevised: ISODateTimeSchema.nullable().optional(),
-});
-
-// --- JOB_SOURCE JUNCTION (Page Execution Output) ---
-export const JobSourceStatusSchema = z.enum([
-  "pending",
-  "transcribed",
-  "failed",
-]);
-
-export const JobSourceSchema = z.object({
-  id: UUIDSchema,
-  jobId: UUIDSchema,
-  sourceId: UUIDSchema,
-  status: JobSourceStatusSchema.default("pending"),
-  rawTranscription: z.string().nullable().optional(),
-  aiMetadata: PageAIMetadataSchema.nullable().optional(),
-  rawApiResponse: z.record(z.string(), z.unknown()).nullable().optional(),
-  errorDetail: z.string().nullable().optional(),
-  executedAt: ISODateTimeSchema,
-});
-
-export type Person = z.infer<typeof PersonSchema>;
-export type Document = z.infer<typeof DocumentSchema>;
-export type DocumentPerson = z.infer<typeof DocumentPersonSchema>;
-export type Job = z.infer<typeof JobSchema>;
-export type Source = z.infer<typeof SourceSchema>;
-export type JobSource = z.infer<typeof JobSourceSchema>;
-```

docs/ver2/ver2.md

@@ -1,150 +0,0 @@
-# Version 2 Plan
-
-Desired Enhancements:
-1. Data store
-    * Upgrade db to PostgresSQL
-    * Begin capturing JSONB data (which will allow future migration to MongoDB if desired)
-    * Relocate db and uploaded images to a location outside of the project folder that can be backed up. (This needs to be done for all projects.) (c:/github/data/transcription?)
-2. Add ability to upload multiple images (or a folder of images)
-    * How many is too many?
-    * If there is a practical max image count, can I break a block of images up into smaller batches automatically?
-3. UI
-    * Introduce the concept of "documents" to the UI. 
-        * Before an image can be uploaded a "document" needs to be created/defined.
-        * As part of the upload process, document images need to be associated with a document.
-    * Multiple image upload
-    * Refine the job detail/log screen
-        * Is document id + original filename the best name for uploaded images?
-        * How to present multiple images within one job?
-        * Add document name, original filename to job detail.
-
-
-
-
-
-
-
-
-## Purpose
-
-Version 2 extends the V1 baseline by introducing a production-oriented persistence architecture while preserving current user workflows.
-
-Primary target changes:
-
-- Migrate relational persistence from SQLite to PostgreSQL
-- Introduce optional MongoDB for document-oriented adjunct data (non-canonical)
-
-V1 behavior remains the functional baseline unless explicitly superseded by approved V2 requirements.
-
----
-
-## V2 Goals
-
-1. **Relational migration complete**
-   - PostgreSQL becomes the default system of record for `Document`, `Source`, `Job`, and `Revision`.
-1. **Operational maturity**
-   - Repeatable migrations, rollback paths, and environment-specific deployment procedures are documented and tested.
-1. **Optional document store integration**
-   - MongoDB is introduced only for clearly scoped use cases that do not replace canonical relational ownership.
-1. **No regression of V1 workflows**
-   - Upload, queue/worker processing, status inspection, original transcription, and optional single revision remain stable.
-
----
-
-## Non-Goals (V2)
-
-- Replacing SQLModel domain ownership with MongoDB
-- Introducing breaking UI behavior for existing V1 flows
-- Expanding revision cardinality beyond current `0..1` without explicit requirements update
-
----
-
-## Proposed Scope
-
-### A) PostgreSQL migration (required)
-
-- Add PostgreSQL runtime profile for local/dev/prod
-- Introduce migration toolchain and migration history
-- Convert bootstrap strategy from compatibility patching to explicit migrations
-- Validate model constraints and indexes against PostgreSQL
-- Add operational checks (connectivity, pool, transaction behavior)
-
-### B) MongoDB integration (optional, gated)
-
-- Define approved use cases (for example: denormalized read models, audit/event projections, or search-oriented materializations)
-- Keep canonical write path in relational store
-- Add feature flag/config gate to enable or disable Mongo features
-- Document consistency model and failure behavior
-
----
-
-## Milestones
-
-## M1 — Requirements and architecture baseline
-
-- Create V2 requirements delta from V1 baseline
-- Define relational/document ownership boundaries
-- Approve migration strategy and cutover approach
-
-**Exit criteria:** signed architecture decision and updated traceability map.
-
-## M2 — PostgreSQL foundation
-
-- Add PostgreSQL environment wiring and secrets strategy
-- Add migration framework and initial schema migration
-- Add CI path using PostgreSQL service container
-
-**Exit criteria:** test suite green on PostgreSQL in CI.
-
-## M3 — Data migration and cutover rehearsal
-
-- Build SQLite -> PostgreSQL migration utility/playbook
-- Rehearse migration on representative datasets
-- Validate rollback/recovery procedures
-
-**Exit criteria:** successful dry-run migration with measured rollback test.
-
-## M4 — MongoDB optional integration
-
-- Implement scoped Mongo use case(s)
-- Add fallback behavior when Mongo unavailable
-- Add tests and operational runbook updates
-
-**Exit criteria:** feature-gated Mongo behavior validated with no V1 flow regressions.
-
-## M5 — Release readiness
-
-- Final regression suite (functional + reliability)
-- Performance and failure-mode checks
-- Production release checklist and sign-off
-
-**Exit criteria:** V2 release approval.
-
----
-
-## Risks and Mitigations
-
-- **Schema drift risk** -> enforce migration-first policy and CI migration checks.
-- **Dual-store consistency risk** -> keep relational source of truth and explicit projection contracts.
-- **Operational complexity** -> staged rollout, runbooks, and feature flags.
-- **Regression risk in worker lifecycle** -> keep dedicated reliability tests around terminal-state guarantees.
-
----
-
-## Traceability and Evidence
-
-Maintain a V2 table with:
-
-- requirement/change ID
-- status (`not started` / `in progress` / `done`)
-- implementation PR
-- validation evidence (test names, migration rehearsal logs, runbook references)
-
----
-
-## Suggested first implementation tasks
-
-1. Create `docs/ver2/adr/` and draft ADR for persistence ownership boundaries.
-2. Add PostgreSQL compose profile and env contract.
-3. Introduce migration tooling and generate initial migration from current schema.
-4. Add CI job for PostgreSQL-backed `pytest -m "not external"`.

src/transcription/__main__.py

@@ -1,20 +0,0 @@
-import uvicorn
-
-from .config import LOGGING_CONFIG
-from .config import get_settings
-
-
-def main() -> None:
-    settings = get_settings()
-    uvicorn.run(
-        "transcription.app:create_app",
-        factory=True,
-        host=settings.host,
-        port=settings.port,
-        log_level=LOGGING_CONFIG.get("root", {}).get("level", "info").lower(),
-        reload=settings.reload,
-    )
-
-
-if __name__ == "__main__":
-    main()

src/transcription/app.py

@@ -16,14 +16,11 @@ from fastapi.staticfiles import StaticFiles
 
 from .api.errors import register_error_handlers
 from .api.health import router as health_router
-from .config import Settings
 from .config import configure_logging
 from .config import get_settings
 from .db import create_all
+from .db import dispose_database_runtime
 from .db import initialize_database_runtime
-from .db.engine import get_database_url
-from .db.engine import resolve_engine
-from .db.session import dispose_session_factory
 from .services import ServiceBundle
 from .services.jobs import JobService
 from .ui import register_pages
@@ -42,7 +39,7 @@ async def _lifespan(app: FastAPI):
     app.state.runtime = initialize_database_runtime(settings=settings)
 
     if settings.should_bootstrap_schema:
-        await create_all(engine=resolve_engine(settings=settings))
+        await create_all(engine=app.state.runtime.engine)
 
     settings.upload_dir.mkdir(parents=True, exist_ok=True)
     settings.prompt_dir.mkdir(parents=True, exist_ok=True)
@@ -50,10 +47,7 @@ async def _lifespan(app: FastAPI):
     await _recover_stale_processing_jobs(app)
 
     async with AsyncExitStack() as stack:
-        stack.push_async_callback(
-            dispose_session_factory,
-            database_url=get_database_url(settings),
-        )
+        stack.push_async_callback(dispose_database_runtime)
         stop_event, worker_notifier = await stack.enter_async_context(
             worker_consumer_lifespan(
                 session_factory=app.state.runtime.session_factory,
@@ -79,14 +73,14 @@ async def _recover_stale_processing_jobs(app: FastAPI) -> None:
         logger.warning("Recovered %s stale processing job(s) at startup", recovered)
 
 
-def create_app(settings: Settings | None = None) -> FastAPI:
+def create_app() -> FastAPI:
     """Create and configure the FastAPI application."""
     app = FastAPI(title="Transcription", lifespan=_lifespan)
-    active_settings = settings or get_settings()
-    app.state.settings = active_settings
+    settings = get_settings()
+    app.state.settings = settings
     app.mount(
         "/uploads",
-        StaticFiles(directory=active_settings.upload_dir, check_dir=False),
+        StaticFiles(directory=settings.upload_dir, check_dir=False),
         name="uploads",
     )
 
@@ -98,10 +92,6 @@ def create_app(settings: Settings | None = None) -> FastAPI:
     async def ui_redirect() -> RedirectResponse:
         return RedirectResponse(url="/ui/upload", status_code=status.HTTP_307_TEMPORARY_REDIRECT)
 
-    @app.get("/healthz")
-    def health() -> dict[str, str]:
-        return {"status": "ok"}
-
     register_error_handlers(app)
     register_pages(app)
     app.include_router(health_router)

src/transcription/app_state.py

@@ -7,7 +7,7 @@ from sqlalchemy.ext.asyncio import async_sessionmaker
 from sqlmodel.ext.asyncio.session import AsyncSession
 
 from transcription.db.runtime import DatabaseRuntime
-from transcription.db.session import get_session_factory
+from transcription.db.runtime import get_session_factory
 from transcription.worker import WorkerNotifier
 from transcription.worker import resolve_worker_notifier
 

src/transcription/config.py

@@ -6,16 +6,12 @@ are resolved by the provider adapters, not here.
 """
 
 import logging.config
+from contextvars import ContextVar
 from enum import StrEnum
-from functools import cache
 from pathlib import Path
-from typing import Annotated
-from typing import Any
 from typing import Literal
 
-from pydantic import BaseModel
 from pydantic import Field
-from pydantic import SecretStr
 from pydantic_settings import BaseSettings
 from pydantic_settings import SettingsConfigDict
 
@@ -26,42 +22,13 @@ class Provider(StrEnum):
     OPENROUTER = "openrouter"
 
 
-class SqliteSettings(BaseModel):
-    driver: Literal["sqlite"] = "sqlite"
-    path: str = "app.db"
-
-
-class PostgresSettings(BaseModel):
-    driver: Literal["postgres"] = "postgres"
-    host: str
-    port: int = 5432
-    database: str
-    user: str
-    password: SecretStr
-
-
-DatabaseSettings = Annotated[
-    SqliteSettings | PostgresSettings,
-    Field(discriminator="driver"),
-]
-
-
 class Settings(BaseSettings):
     model_config = SettingsConfigDict(
         env_file=".env",
         env_file_encoding="utf-8",
         extra="ignore",
-        cli_parse_args=True,
-        cli_implicit_flags=True,
-        cli_kebab_case=True,
     )
 
-    # --- NiceGUI Server ---
-    host: str = "0.0.0.0"
-    port: int = 8000
-    log_level: Literal["critical", "error", "warning", "info", "debug", "trace"] = "info"
-    reload: bool = False
-
     # --- AI provider ---
     provider: Provider = Provider.OPENROUTER
     openrouter_api_key: str
@@ -73,9 +40,8 @@ class Settings(BaseSettings):
     environment: Literal["development", "test", "production"] = "development"
 
     # --- persistence ---
-    database: DatabaseSettings = Field(default_factory=SqliteSettings)
     database_url: str = "sqlite:///./transcription.db"
-    bootstrap_schema_on_startup: bool = False
+    bootstrap_schema_on_startup: bool | None = None
     sqlite_check_same_thread: bool = False
 
     # --- filesystem paths ---
@@ -98,12 +64,18 @@ class Settings(BaseSettings):
         return self.environment in {"development", "test"}
 
 
-@cache
+_settings: ContextVar[Settings | None] = ContextVar("settings", default=None)
+
+
 def get_settings(**kwargs) -> Settings:
-    return Settings(**kwargs)
+    settings = _settings.get()
+    if settings is None:
+        settings = Settings(**kwargs)  # pyright: ignore[reportCallIssue]
+        _settings.set(settings)
+    return settings
 
 
-LOGGING_CONFIG: dict[str, Any] = {
+LOGGING_CONFIG: dict[str, object] = {
     "version": 1,
     "disable_existing_loggers": False,
     "formatters": {
@@ -133,10 +105,7 @@ LOGGING_CONFIG: dict[str, Any] = {
 }
 
 
-def configure_logging(settings: Settings | None = None) -> None:
+def configure_logging() -> None:
     """Configure root logging once at startup."""
-    cfg = LOGGING_CONFIG.copy()
-    active_settings = settings or get_settings()
-    cfg["loggers"]["transcription"]["level"] = active_settings.log_level.upper()
-    logging.config.dictConfig(cfg)
+    logging.config.dictConfig(LOGGING_CONFIG)
     logger.debug("Logging configured")

src/transcription/db/__init__.py

@@ -1,13 +1,6 @@
 from .operations import create_all
 from .runtime import dispose_database_runtime
+from .runtime import get_session
 from .runtime import initialize_database_runtime
-from .session import session_scope
-from .session import transaction_scope
 
-__all__ = [
-    "create_all",
-    "dispose_database_runtime",
-    "initialize_database_runtime",
-    "session_scope",
-    "transaction_scope",
-]
+__all__ = ["create_all", "dispose_database_runtime", "get_session", "initialize_database_runtime"]

src/transcription/db/engine.py

@@ -1,60 +0,0 @@
-from functools import cache
-from typing import Any
-
-from sqlalchemy import URL
-from sqlalchemy import StaticPool
-from sqlalchemy.ext.asyncio import AsyncEngine
-from sqlalchemy.ext.asyncio import create_async_engine
-
-from ..config import PostgresSettings
-from ..config import Settings
-from ..config import SqliteSettings
-from ..config import get_settings
-
-
-def get_database_url(settings: Settings) -> str:
-    match settings.database:
-        case SqliteSettings(path=path):
-            url = URL.create(
-                drivername="sqlite+aiosqlite",
-                database=path,
-            )
-        case PostgresSettings() as database:
-            url = URL.create(
-                drivername="postgresql+asyncpg",
-                host=database.host,
-                port=database.port,
-                database=database.database,
-                username=database.user,
-                password=database.password.get_secret_value(),
-            )
-    return url.render_as_string(hide_password=False)
-
-
-def resolve_engine(settings: Settings | None = None) -> AsyncEngine:
-    active_settings = settings or get_settings()
-    return get_engine(get_database_url(active_settings))
-
-
-@cache
-def get_engine(database_url: str) -> AsyncEngine:
-    kwargs: dict[str, Any] = {"echo": False, "pool_pre_ping": True}
-    if database_url.startswith("sqlite"):
-        kwargs["connect_args"] = {"check_same_thread": False}
-        if ":memory:" in database_url:
-            kwargs["poolclass"] = StaticPool
-
-    return create_async_engine(database_url, **kwargs)
-
-
-async def dispose_engine(database_url: str) -> None:
-    engine = get_engine(database_url)
-    try:
-        await engine.dispose()
-    finally:
-        get_engine.cache_clear()
-
-
-async def refresh_engine(database_url: str) -> AsyncEngine:
-    await dispose_engine(database_url)
-    return get_engine(database_url)

src/transcription/db/operations.py

@@ -10,25 +10,13 @@ from sqlmodel import SQLModel
 from sqlmodel import select
 from sqlmodel.ext.asyncio.session import AsyncSession
 
-from .engine import resolve_engine
-from .models import Job
-from .models import JobStatus
+from ..models import Job
+from ..models import JobStatus
+from .runtime import get_engine
 
 logger = logging.getLogger(__name__)
 
 
-async def create_all(*, engine: AsyncEngine | None = None) -> None:
-    """Create all tables on the selected engine."""
-    # Import models so SQLModel metadata is fully registered before bootstrap.
-    from transcription.db import models as _models  # noqa: F401
-
-    active_engine = engine or resolve_engine()
-    async with active_engine.begin() as connection:
-        await connection.run_sync(SQLModel.metadata.create_all)
-        await connection.run_sync(_ensure_sqlite_compat_columns)
-    logger.debug("Database schema bootstrap complete for database_url=%s", active_engine.url)
-
-
 async def get_next_queued_job(*, session: AsyncSession) -> Job | None:
     """Get the next queued job, if any."""
     result = await session.exec(
@@ -40,6 +28,18 @@ async def get_next_queued_job(*, session: AsyncSession) -> Job | None:
     return result.first()
 
 
+async def create_all(*, engine: AsyncEngine | None = None) -> None:
+    """Create all tables on the selected engine."""
+    # Import models so SQLModel metadata is fully registered before bootstrap.
+    from transcription import models as _models  # noqa: F401
+
+    active_engine = engine or get_engine()
+    async with active_engine.begin() as connection:
+        await connection.run_sync(SQLModel.metadata.create_all)
+        await connection.run_sync(_ensure_sqlite_compat_columns)
+    logger.debug("Database schema bootstrap complete for database_url=%s", active_engine.url)
+
+
 def _ensure_sqlite_compat_columns(connection: Connection) -> None:
     """Apply lightweight dev/test SQLite compatibility column patches.
 
@@ -68,8 +68,12 @@ def _ensure_sqlite_compat_columns(connection: Connection) -> None:
                     break
             if not has_unique_source:
                 connection.execute(
-                    text("CREATE UNIQUE INDEX IF NOT EXISTS ux_revision_source_id ON revision(source_id)")
+                    text(
+                        "CREATE UNIQUE INDEX IF NOT EXISTS "
+                        "ux_revision_source_id ON revision(source_id)"
+                    )
                 )
                 logger.warning(
-                    "Applied SQLite compatibility schema patch table=revision unique_index=ux_revision_source_id"
+                    "Applied SQLite compatibility schema patch "
+                    "table=revision unique_index=ux_revision_source_id"
                 )

src/transcription/db/runtime.py

@@ -1,16 +1,18 @@
 import logging
+from collections.abc import AsyncGenerator
+from contextlib import asynccontextmanager
 from contextvars import ContextVar
 from dataclasses import dataclass
+from functools import partial
 
 from sqlalchemy.ext.asyncio import AsyncEngine
 from sqlalchemy.ext.asyncio import async_sessionmaker
+from sqlalchemy.ext.asyncio import create_async_engine
 from sqlmodel.ext.asyncio.session import AsyncSession
+from sqlmodel.pool import StaticPool
 
 from ..config import Settings
 from ..config import get_settings
-from .engine import get_database_url
-from .engine import get_engine
-from .session import get_session_factory
 
 logger = logging.getLogger(__name__)
 
@@ -35,6 +37,33 @@ async def dispose_database_runtime() -> None:
     _runtime.set(None)
 
 
+def _to_async_database_url(database_url: str) -> str:
+    """Normalize configured database URL to an async SQLAlchemy driver URL."""
+    if database_url.startswith("sqlite://") and not database_url.startswith("sqlite+aiosqlite://"):
+        return database_url.replace("sqlite://", "sqlite+aiosqlite://", 1)
+    if database_url.startswith("postgresql://") and not database_url.startswith("postgresql+asyncpg://"):
+        return database_url.replace("postgresql://", "postgresql+asyncpg://", 1)
+    return database_url
+
+
+def _build_engine(settings: Settings) -> AsyncEngine:
+    database_url = _to_async_database_url(settings.database_url)
+    engine_factory = partial(
+        create_async_engine,
+        url=database_url,
+        echo=False,
+        pool_pre_ping=True,
+    )
+
+    if database_url.startswith("sqlite"):
+        sqlite_connect_settings = {"check_same_thread": settings.sqlite_check_same_thread}
+        engine_factory = partial(engine_factory, connect_args=sqlite_connect_settings)
+        if ":memory:" in database_url:
+            engine_factory = partial(engine_factory, poolclass=StaticPool)
+
+    return engine_factory()
+
+
 def initialize_database_runtime(*, settings: Settings | None = None) -> DatabaseRuntime:
     """Initialize lifespan-owned async DB resources once per process."""
     runtime = _runtime.get()
@@ -42,10 +71,33 @@ def initialize_database_runtime(*, settings: Settings | None = None) -> Database
         return runtime
 
     active_settings = settings or get_settings()
-    database_url = get_database_url(active_settings)
-    engine = get_engine(database_url)
-    session_factory = get_session_factory(database_url)
+    engine = _build_engine(active_settings)
+    session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
     runtime = DatabaseRuntime(engine=engine, session_factory=session_factory)
     _runtime.set(runtime)
     logger.debug("Initialized async database runtime for database_url=%s", engine.url)
     return runtime
+
+
+def get_engine(settings: Settings | None = None) -> AsyncEngine:
+    """Return the current async SQLAlchemy engine."""
+    runtime = _runtime.get() or initialize_database_runtime(settings=settings)
+    return runtime.engine
+
+
+def get_session_factory(settings: Settings | None = None) -> async_sessionmaker[AsyncSession]:
+    """Return the shared async session factory."""
+    runtime = _runtime.get() or initialize_database_runtime(settings=settings)
+    return runtime.session_factory
+
+
+@asynccontextmanager
+async def get_session(
+    *,
+    settings: Settings | None = None,
+    session_factory: async_sessionmaker[AsyncSession] | None = None,
+) -> AsyncGenerator[AsyncSession]:
+    """Yield a database session and ensure cleanup."""
+    active_session_factory = session_factory or get_session_factory(settings)
+    async with active_session_factory() as session:
+        yield session

src/transcription/db/session.py

@@ -1,79 +0,0 @@
-from collections.abc import AsyncGenerator
-from contextlib import asynccontextmanager
-from functools import cache
-from typing import Annotated
-
-from fastapi import Depends
-from sqlalchemy.ext.asyncio import AsyncSessionTransaction
-from sqlalchemy.ext.asyncio import async_sessionmaker
-from sqlmodel.ext.asyncio.session import AsyncSession
-
-from ..config import get_settings
-from .engine import dispose_engine
-from .engine import get_database_url
-from .engine import get_engine
-
-type SessionFactory = async_sessionmaker[AsyncSession]
-
-
-@cache
-def get_session_factory(database_url: str) -> SessionFactory:
-    return async_sessionmaker(
-        bind=get_engine(database_url),
-        class_=AsyncSession,
-        expire_on_commit=False,
-    )
-
-
-def resolve_session_factory(database_url: str | None = None) -> SessionFactory:
-    return get_session_factory(database_url or get_database_url(get_settings()))
-
-
-type SessionFactoryDep = Annotated[SessionFactory, Depends(resolve_session_factory)]
-
-
-async def dispose_session_factory(database_url: str) -> None:
-    get_session_factory.cache_clear()
-    await dispose_engine(database_url)
-
-
-@asynccontextmanager
-async def session_scope(
-    *,
-    database_url: str | None = None,
-    session: AsyncSession | None = None,
-) -> AsyncGenerator[AsyncSession]:
-    if session is not None:
-        yield session
-        return
-
-    session_factory = resolve_session_factory(database_url)
-    async with session_factory() as owned_session:
-        yield owned_session
-
-
-type SessionScopeDep = Annotated[AsyncSession, Depends(session_scope)]
-
-
-@asynccontextmanager
-async def transaction_scope(
-    *,
-    database_url: str | None = None,
-    session: AsyncSessionTransaction | None = None,
-) -> AsyncGenerator[AsyncSessionTransaction]:
-    match session:
-        case AsyncSession() as async_session:
-            if not async_session.in_transaction():
-                raise RuntimeError("A supplied session must have an active transaction")
-            yield async_session
-            return
-        case AsyncSessionTransaction() as async_transaction:
-            yield async_transaction
-            return
-
-    session_factory = resolve_session_factory(database_url)
-    async with session_factory().begin() as owned_session:
-        yield owned_session
-
-
-type TransactionScopeDep = Annotated[AsyncSessionTransaction, Depends(transaction_scope)]

src/transcription/services/base.py

@@ -8,8 +8,7 @@ from sqlmodel.ext.asyncio.session import AsyncSession
 
 from ..config import Settings
 from ..config import get_settings
-from ..db.session import resolve_session_factory
-from ..db.session import session_scope
+from ..db.runtime import get_session_factory
 
 
 class ServiceBase(ABC):
@@ -25,14 +24,19 @@ class ServiceBase(ABC):
         queue: asyncio.Queue | None = None,
     ):
         self.settings = get_settings()
-        self.session_factory = session_factory or resolve_session_factory()
+        self.session_factory = session_factory or get_session_factory()
         self.queue = queue or asyncio.Queue()
 
     @asynccontextmanager
     async def _session_scope(self, session: AsyncSession | None = None):
         """Provide a transactional scope around a series of operations."""
-        async with session_scope(session=session) as active_session:
-            yield active_session
+        if session is not None:
+            # Reuse the provided session if one is passed in
+            yield session
+        else:
+            # Otherwise, create a new session for this scope
+            async with self.session_factory() as new_session:
+                yield new_session
 
     async def _finalize(
         self,

src/transcription/services/documents.py

@@ -9,9 +9,9 @@ from sqlalchemy.orm import selectinload
 from sqlmodel import select
 from sqlmodel.ext.asyncio.session import AsyncSession
 
-from ..db.models import Document
 from ..errors import AppError
 from ..errors import ErrorCategory
+from ..models import Document
 from .base import ServiceBase
 
 logger = logging.getLogger(__name__)

src/transcription/services/jobs.py

@@ -7,9 +7,9 @@ from sqlalchemy.orm import selectinload
 from sqlmodel import select
 from sqlmodel.ext.asyncio.session import AsyncSession
 
-from ..db.models import Job
-from ..db.models import JobStatus
-from ..db.models import Source
+from ..models import Job
+from ..models import JobStatus
+from ..models import Source
 from .base import ServiceBase
 
 
@@ -170,7 +170,11 @@ class JobService(ServiceBase):
         ``stale_before`` are considered stale and re-queued.
         """
         async with self._session_scope(session) as _session:
-            query = select(Job).where(Job.status == JobStatus.PROCESSING).where(Job.date_updated < stale_before)
+            query = (
+                select(Job)
+                .where(Job.status == JobStatus.PROCESSING)
+                .where(Job.date_updated < stale_before)
+            )
             stale_jobs = (await _session.exec(query)).all()
             if not stale_jobs:
                 return 0

src/transcription/services/store.py

@@ -11,9 +11,9 @@ from transcription.config import get_settings
 from transcription.errors import AppError
 from transcription.errors import ErrorCategory
 
-from ..db.models import Document
-from ..db.models import Job
-from ..db.models import Source
+from ..models import Document
+from ..models import Job
+from ..models import Source
 from .documents import UploadJobResult
 
 logger = logging.getLogger(__name__)

src/transcription/services/transcription.py

@@ -18,11 +18,11 @@ from sqlmodel.ext.asyncio.session import AsyncSession
 
 from transcription.config import Settings
 from transcription.config import get_settings
-from transcription.db.models import Job
-from transcription.db.models import Revision
-from transcription.db.models import Source
 from transcription.errors import AppError
 from transcription.errors import ErrorCategory
+from transcription.models import Job
+from transcription.models import Revision
+from transcription.models import Source
 from transcription.providers import ProviderAuthError
 from transcription.providers import ProviderError
 from transcription.providers import ProviderResponseError

src/transcription/services/workflows.py

@@ -5,13 +5,13 @@ from sqlmodel.ext.asyncio.session import AsyncSession
 
 from ..config import Settings
 from ..config import get_settings
-from ..db.models import Job
-from ..db.models import JobStatus
-from ..db.models import Source
 from ..errors import AppError
 from ..errors import ErrorCategory
 from ..errors import classify_unexpected_error
 from ..errors import format_error_detail
+from ..models import Job
+from ..models import JobStatus
+from ..models import Source
 from ..providers import TranscriptionResult
 from . import ServiceBundle
 from .transcription import DEFAULT_PROMPT_FILE

src/transcription/ui/components/document_panzoom.py

@@ -10,7 +10,7 @@ from uuid import uuid4
 from nicegui import ui
 
 from transcription.config import get_settings
-from transcription.db.models import Source
+from transcription.models import Source
 
 PANGOZOOM_CDN_URL = "https://unpkg.com/@panzoom/[email protected]/dist/panzoom.min.js"
 UPLOADS_URL_PREFIX = "/uploads"

src/transcription/ui/components/job_detail.py

@@ -6,9 +6,9 @@ import logging
 
 from nicegui import ui
 
-from transcription.db.models import Job
-from transcription.db.models import Revision
-from transcription.db.models import Source
+from transcription.models import Job
+from transcription.models import Revision
+from transcription.models import Source
 from transcription.ui.components.document_panzoom import render_document_panzoom
 from transcription.ui.components.transcript import render_original_transcription_card
 from transcription.ui.components.transcript import render_revision_row

src/transcription/ui/components/transcript.py

@@ -9,8 +9,8 @@ from typing import Any
 
 from nicegui import ui
 
-from transcription.db.models import Job
-from transcription.db.models import Revision
+from transcription.models import Job
+from transcription.models import Revision
 
 type RevisionAction = Callable[[Revision], Awaitable[None] | None]
 

src/transcription/ui/pages/jobs_page.py

@@ -4,18 +4,19 @@ from __future__ import annotations
 
 from uuid import UUID
 
+from fastapi import Request
 from nicegui import ui
 
-from transcription.db.models import Job
-from transcription.db.models import JobStatus
-from transcription.db.models import Source
+from transcription.app_state import resolve_session_factory
+from transcription.models import Job
+from transcription.models import JobStatus
+from transcription.models import Source
 from transcription.services.jobs import JobService
 from transcription.services.transcription import TranscriptionService
 from transcription.ui.components.app_shell import render_navigation_header
 from transcription.ui.components.error_presenter import show_error
 from transcription.ui.components.table.jobs import render_jobs_table
 
-from ...db.session import SessionFactoryDep
 from ..components.document_panzoom import render_document_panzoom
 from ..components.table.jobs import JobTableRow
 from ..components.transcript import render_original_transcription_card
@@ -26,7 +27,8 @@ def register_page() -> None:  # noqa: PLR0915
     """Register jobs list and detail routes."""
 
     @ui.page("/jobs")
-    async def jobs_page(session_factory: SessionFactoryDep) -> None:
+    async def jobs_page(request: Request) -> None:
+        session_factory = resolve_session_factory(request.app.state)
         jobs_service = JobService(session_factory=session_factory)
         render_navigation_header(current_path="/jobs")
 
@@ -49,7 +51,8 @@ def register_page() -> None:  # noqa: PLR0915
         await render_table()
 
     @ui.page("/jobs/{job_id}")
-    async def job_detail_page(job_id: str, session_factory: SessionFactoryDep) -> None:  # noqa: PLR0915
+    async def job_detail_page(job_id: str, request: Request) -> None:  # noqa: PLR0915
+        session_factory = resolve_session_factory(request.app.state)
         jobs_service = JobService(session_factory=session_factory)
         transcription_service = TranscriptionService(session_factory=session_factory)
         render_navigation_header(current_path="/jobs")

src/transcription/ui/pages/upload_page.py

@@ -5,7 +5,8 @@ from __future__ import annotations
 from fastapi import Request
 from nicegui import ui
 
-from transcription.db import session_scope
+from transcription.app_state import resolve_session_factory
+from transcription.db import get_session
 from transcription.services.store import create_upload_job
 from transcription.ui.components.app_shell import render_navigation_header
 from transcription.ui.components.upload import render_upload_widget
@@ -18,9 +19,10 @@ def register_page() -> None:
     @ui.page("/upload", title="Upload Document")
     def upload_page(request: Request) -> None:
         render_navigation_header(current_path="/upload")
+        session_factory = resolve_session_factory(request.app.state)
 
         async def submit_upload(filename: str, file_bytes: bytes):
-            async with session_scope() as session:
+            async with get_session(session_factory=session_factory) as session:
                 return await create_upload_job(
                     filename=filename,
                     file_bytes=file_bytes,

src/transcription/worker.py

@@ -14,7 +14,7 @@ from uuid import UUID
 from sqlalchemy.ext.asyncio import async_sessionmaker
 from sqlmodel.ext.asyncio.session import AsyncSession
 
-from transcription.db import session_scope
+from transcription.db import get_session
 from transcription.errors import AppError
 from transcription.errors import classify_unexpected_error
 
@@ -178,7 +178,7 @@ async def process_next_queued_job(
         )
 
     if session is None:
-        async with session_scope(session_factory=session_factory) as local_session:
+        async with get_session(session_factory=session_factory) as local_session:
             return await process_next_queued_job_workflow(services=services, session=local_session)
 
     return await process_next_queued_job_workflow(services=services, session=session)

tests/conftest.py

@@ -13,12 +13,11 @@ from sqlmodel.pool import StaticPool
 
 from transcription.config import Settings
 from transcription.config import get_settings
-from transcription.db.engine import get_database_url
-from transcription.db.engine import get_engine
 from transcription.db.operations import create_all
-from transcription.db.session import dispose_session_factory
-from transcription.db.session import get_session_factory
-from transcription.db.session import session_scope
+from transcription.db.runtime import dispose_database_runtime
+from transcription.db.runtime import get_engine
+from transcription.db.runtime import get_session
+from transcription.db.runtime import get_session_factory
 from transcription.services.documents import DocumentService
 from transcription.services.jobs import JobService
 
@@ -40,26 +39,23 @@ def session():
 async def default_settings():
     """Provide default settings for tests."""
     settings = get_settings(database_url="sqlite:///:memory:")
-    db_url = get_database_url(settings)
-    await create_all(engine=get_engine(database_url=db_url))
+    await create_all(engine=get_engine(settings=settings))
     return settings
 
 
 @pytest_asyncio.fixture
 async def async_session(default_settings: Settings):
     """Provide a clean asynchronous database session for async tests."""
-    db_url = get_database_url(default_settings)
-    async with session_scope(database_url=db_url) as async_session:
+    async with get_session(settings=default_settings) as async_session:
         yield async_session
 
-    await dispose_session_factory(db_url)
+    await dispose_database_runtime()
 
 
 @pytest.fixture
 def default_session_factory(default_settings: Settings):
     """Provide a base fixture for tests that require database access."""
-    db_url = get_database_url(default_settings)
-    session_factory = get_session_factory(database_url=db_url)
+    session_factory = get_session_factory(settings=default_settings)
     return session_factory
 
 

tests/integration/test_pipeline_flow.py

@@ -5,8 +5,8 @@ from pathlib import Path
 import pytest
 
 from transcription.config import Settings
-from transcription.db.models import Job
-from transcription.db.models import JobStatus
+from transcription.models import Job
+from transcription.models import JobStatus
 from transcription.providers.base import TranscriptionResult
 from transcription.services.store import create_upload_job
 from transcription.worker import process_next_queued_job

tests/services/test_job_service.py

@@ -2,10 +2,10 @@ from uuid import uuid4
 
 import pytest
 
-from transcription.db.models import Document
-from transcription.db.models import Job
-from transcription.db.models import JobStatus
-from transcription.db.models import Source
+from transcription.models import Document
+from transcription.models import Job
+from transcription.models import JobStatus
+from transcription.models import Source
 from transcription.services.documents import DocumentService
 from transcription.services.jobs import JobService
 

tests/services/test_transcription_service.py

@@ -4,10 +4,10 @@ from uuid import uuid4
 
 import pytest
 
-from transcription.db.models import Document
-from transcription.db.models import Job
-from transcription.db.models import JobStatus
-from transcription.db.models import Source
+from transcription.models import Document
+from transcription.models import Job
+from transcription.models import JobStatus
+from transcription.models import Source
 from transcription.services.documents import DocumentService
 from transcription.services.jobs import JobService
 from transcription.services.transcription import TranscriptionService

tests/services/test_workflows_reliability.py

@@ -6,10 +6,10 @@ from uuid import uuid4
 import pytest
 
 from transcription.config import Settings
-from transcription.db.models import Document
-from transcription.db.models import Job
-from transcription.db.models import JobStatus
-from transcription.db.models import Source
+from transcription.models import Document
+from transcription.models import Job
+from transcription.models import JobStatus
+from transcription.models import Source
 from transcription.services import ServiceBundle
 from transcription.services.workflows import process_queued_job
 

tests/test_db.py

@@ -4,18 +4,17 @@ import pytest
 from sqlalchemy import inspect
 
 from transcription.config import Settings
-from transcription.config import SqliteSettings
 from transcription.db import create_all
 from transcription.db import dispose_database_runtime
+from transcription.db import get_session
 from transcription.db import initialize_database_runtime
-from transcription.db import session_scope
 
 
 @pytest.mark.asyncio
 async def test_create_all_creates_expected_tables(tmp_path):
     settings = Settings(
         openrouter_api_key="test-key",
-        database=SqliteSettings(path=str(tmp_path / "schema.db")),
+        database_url=f"sqlite:///{tmp_path / 'schema.db'}",
         environment="test",
     )
     runtime = initialize_database_runtime(settings=settings)
@@ -37,13 +36,13 @@ async def test_create_all_creates_expected_tables(tmp_path):
 async def test_get_session_yields_async_session(tmp_path):
     settings = Settings(
         openrouter_api_key="test-key",
-        database=SqliteSettings(path=str(tmp_path / "session.db")),
+        database_url=f"sqlite:///{tmp_path / 'session.db'}",
         environment="test",
     )
     initialize_database_runtime(settings=settings)
 
     try:
-        async with session_scope(settings=settings) as session:
+        async with get_session(settings=settings) as session:
             assert session is not None
     finally:
         await dispose_database_runtime()

tests/test_models.py

@@ -5,11 +5,11 @@ from uuid import UUID
 import pytest
 from sqlalchemy.exc import IntegrityError
 
-from transcription.db.models import Document
-from transcription.db.models import Job
-from transcription.db.models import JobStatus
-from transcription.db.models import Revision
-from transcription.db.models import Source
+from transcription.models import Document
+from transcription.models import Job
+from transcription.models import JobStatus
+from transcription.models import Revision
+from transcription.models import Source
 
 
 def _make_document(**overrides) -> Document:

tests/ui/conftest.py

@@ -4,7 +4,6 @@ from __future__ import annotations
 
 import asyncio
 from collections.abc import Callable
-from collections.abc import Generator
 from pathlib import Path
 from uuid import UUID
 
@@ -15,31 +14,32 @@ from sqlmodel import delete
 
 from transcription.app import create_app
 from transcription.config import Settings
-from transcription.config import SqliteSettings
+from transcription.config import _settings
 from transcription.db import create_all
+from transcription.db import get_session
 from transcription.db import initialize_database_runtime
-from transcription.db import session_scope
-from transcription.db.models import Document
-from transcription.db.models import Job
-from transcription.db.models import JobStatus
-from transcription.db.models import Revision
-from transcription.db.models import Source
+from transcription.models import Document
+from transcription.models import Job
+from transcription.models import JobStatus
+from transcription.models import Revision
+from transcription.models import Source
 
 RevisionSeed = str
 
 
 @pytest.fixture(scope="session")
-def app_client(tmp_path_factory: pytest.TempPathFactory) -> Generator[tuple[FastAPI, TestClient]]:
+def app_client(tmp_path_factory: pytest.TempPathFactory) -> tuple[FastAPI, TestClient]:
     """Provide a real application and test client backed by in-memory SQLite."""
     tmp_path = tmp_path_factory.mktemp("ui")
     settings = Settings(
         openrouter_api_key="test-key",
-        database=SqliteSettings(path=":memory:"),
+        database_url="sqlite:///:memory:",
         environment="test",
         bootstrap_schema_on_startup=True,
         upload_dir=tmp_path / "uploads",
         prompt_dir=tmp_path / "prompts",
     )
+    _settings.set(settings)
 
     app = create_app()
     app.state.runtime = initialize_database_runtime(settings=settings)
@@ -54,7 +54,7 @@ def clear_ui_database(app_client: tuple[FastAPI, TestClient]) -> None:
     app, _ = app_client
 
     async def _clear() -> None:
-        async with session_scope() as session:
+        async with get_session(session_factory=app.state.runtime.session_factory) as session:
             await session.exec(delete(Revision))
             await session.exec(delete(Source))
             await session.exec(delete(Job))
@@ -80,7 +80,7 @@ def seed_job(app_client: tuple[FastAPI, TestClient]) -> Callable[..., UUID]:
         source_file: Path | None = None,
     ) -> UUID:
         async def _insert() -> UUID:
-            async with session_scope() as session:
+            async with get_session(session_factory=app.state.runtime.session_factory) as session:
                 stored_path = app.state.settings.upload_dir / filename
                 stored_path.parent.mkdir(parents=True, exist_ok=True)
                 source_path = source_file or fixtures_dir / "small_png.png"

tests/ui/test_jobs_page.py

@@ -5,7 +5,7 @@ from uuid import uuid4
 
 import pytest
 
-from transcription.db.models import JobStatus
+from transcription.models import JobStatus
 
 
 @pytest.mark.integration