Remove references to "v4" throughout the code and documentation

2026-08-20 16:35:20 -05:00
parent eaeb0bc806
commit bf2f3ac09c
34 changed files with 140 additions and 550 deletions
@@ -0,0 +1,169 @@
+# System Architecture (Version 4)
+
+This document defines the current Version 4 architecture baseline.
+
+## Architecture Objectives
+
+- Preserve durable archival records for Documents, Sources, People, and processing runs.
+- Execute page transcription asynchronously with bounded worker behavior.
+- Preserve append-only machine-attempt evidence with request/response provenance.
+- Keep UI, API, service, persistence, and provider boundaries explicit and testable.
+
+## Technical Stack
+
+- **Runtime:** Python 3.12+
+- **Web application:** FastAPI + NiceGUI
+- **Persistence:** SQLModel / SQLAlchemy (SQLite-first, PostgreSQL-compatible model design)
+- **Validation and settings:** Pydantic V2 + pydantic-settings
+- **Concurrency:** asyncio worker loop
+- **Provider integration:** OpenRouter adapter behind provider interface
+- **Quality and tests:** Ruff, ty, pytest, pytest-asyncio
+
+## Runtime Topology
+
+```mermaid
+flowchart LR
+U[Browser User] --> A[FastAPI + NiceGUI App]
+A --> W[Asyncio Worker]
+A --> DB[(SQLite/PostgreSQL Model)]
+W --> P[Provider Adapter]
+W --> DB
+```
+
+## Layered Boundaries
+
+### Interface Layer
+
+- `src/transcription/ui/**`
+- `src/transcription/api/**`
+
+Responsibilities:
+
+- Route registration, page orchestration, presentation adapters.
+- Structured user messaging through shared error presenter.
+- No direct persistence access from pages/components.
+
+### Service and Orchestration Layer
+
+- `src/transcription/services/documents.py`
+- `src/transcription/services/people.py`
+- `src/transcription/services/jobs.py`
+- `src/transcription/services/sources.py`
+- `src/transcription/services/evidence.py`
+- `src/transcription/services/store.py`
+- `src/transcription/services/workflows.py`
+
+Responsibilities:
+
+- Aggregate ownership and invariants.
+- Transaction-aware write helpers.
+- Cross-service workflows in orchestration modules (`store.py`, `workflows.py`).
+
+### Persistence Layer
+
+- `src/transcription/db/**`
+
+Responsibilities:
+
+- SQLModel definitions, async session/engine runtime, registry bootstrap.
+- Loader helpers that enforce explicit eager loading with `lazy="raise"` relationships.
+
+### Provider Layer
+
+- `src/transcription/providers/**`
+
+Responsibilities:
+
+- Provider API encapsulation.
+- Request manifest and transport evidence capture.
+- Normalized transcription result contract.
+
+## Core Domain Model
+
+- `Document` owns archival metadata and links to `Source`, `Job`, and `DocumentPerson`.
+- `Source` is a document page/file record with selected machine projection and human revision.
+- `Job` is an aggregate processing run with status and frozen prompt/runtime settings.
+- `JobSource` is queue/membership state for one `(job, source)` pair.
+- `ExecutionAttempt` is append-only evidence for each provider call.
+- `DocumentType` and `PersonRole` are UUID-backed registries with optional protected `semantic_key`.
+
+## Processing and Evidence Workflow
+
+1. User creates/updates Document metadata and linked People atomically through workflow orchestration.
+2. User creates a Job by uploading one or more Source files or by retranscribing an existing Source.
+3. Source files are validated and stored; orientation normalization may be applied at ingest, and stored bytes become the canonical processing bytes.
+4. Worker claims queued Job, transitions to `processing`, and processes pending pages in deterministic order.
+5. Each provider call writes one immutable `ExecutionAttempt` with:
+   - request manifest + hash
+   - transport evidence (when response exists)
+   - SDK snapshot and normalized metadata
+   - outcome, timing, and error details when applicable
+6. `JobSource` status is updated as queue/projection state; `Source.raw_transcription` is set on first successful attempt and can be explicitly re-pointed by candidate promotion.
+7. Job terminal status resolves to `transcribed`, `partial_success`, or `failed`.
+
+## Status Semantics
+
+- **Job statuses:** `queued`, `processing`, `transcribed`, `partial_success`, `failed`
+  - Operational success path resolves to `transcribed`.
+- **JobSource statuses:** `pending`, `transcribed`, `failed`, `cancelled`
+
+## Security and Path Handling Boundaries
+
+- Print media delivery uses record-validated API route:
+  - `src/transcription/api/print_api.py`
+- General UI media links resolve through:
+  - `src/transcription/ui/components/media_urls.py`
+- Local filesystem paths must never be accepted from user input as trusted media routes.
+
+## Concurrency and Reliability Principles
+
+- Worker loop reuses service bundle/provider resources for pooled calls.
+- Provider-call timeout is explicit and bounded.
+- Non-retriable worker-loop faults are surfaced and stop loop spin.
+- Per-page outcomes are durably persisted before processing next page.
+
+## Design Decisions and Rationale
+
+### Why `transcribed` is the success terminal state
+
+- The worker and job orchestration resolve successful completion to `JobStatus.TRANSCRIBED`, with mixed and failure outcomes represented by `partial_success` and `failed`.
+- This keeps terminal status vocabulary aligned with what the pipeline actually produces: transcribed page content and evidence, not a generic completion marker.
+
+### Why evidence history is append-only while page text is a projection
+
+- `ExecutionAttempt` stores immutable per-call evidence and preserves full attempt history across retries.
+- `Source.raw_transcription` is intentionally a mutable projection so UI and exports can show a selected current machine text without mutating historical evidence.
+- This split keeps auditability and UX both first-class: history is durable, presentation is editable.
+
+### Why orchestration modules own cross-service workflows
+
+- Service modules do not import each other; aggregate ownership remains local to each service.
+- Multi-aggregate writes are coordinated in orchestration modules (`store.py`, `workflows.py`) so transaction boundaries are explicit and testable.
+- This avoids circular dependencies and keeps cross-cutting workflow logic centralized.
+
+### Why explicit eager loading is required
+
+- ORM relationships are configured with `lazy="raise"` in key paths, so code must request needed relationships up front.
+- This prevents hidden query behavior in UI/service code and makes read shape deterministic and reviewable.
+
+### Why canonical source bytes may be ingest-normalized
+
+- Ingest normalization can correct orientation before persistence so provider calls, evidence hashes, and rendered processing source are consistent.
+- The canonical stored bytes, digest, and size become the durable processing identity for that source.
+
+### Why media access uses controlled routes/helpers
+
+- Print/export media uses record-validated API endpoints to avoid direct filesystem path exposure.
+- General UI media URLs are generated through shared resolver helpers to keep path handling consistent and centralized.
+
+## Scope Boundary
+
+Current architecture rules live in `docs/*`.
+
+## Related References
+
+- [System Requirements](requirements.md)
+- [Data Model](schema.md)
+- [Error Handling Policy](error_handling.md)
+- [Error Handling invariant](../invariant/error_handling.md)
+- [AI evidence invariant](../invariant/ai_evidence_and_provenance.md)