transcription/docs/ver4/architecture_v4.md

System Architecture (Version 4)

This document defines the current V4 architecture baseline.

Architecture Objectives

• Preserve durable archival records for Documents, Sources, People, and processing runs.
• Execute page transcription asynchronously with bounded worker behavior.
• Preserve append-only machine-attempt evidence with request/response provenance.
• Keep UI, API, service, persistence, and provider boundaries explicit and testable.

Technical Stack

• Runtime: Python 3.12+
• Web application: FastAPI + NiceGUI
• Persistence: SQLModel / SQLAlchemy (SQLite-first, PostgreSQL-compatible model design)
• Validation and settings: Pydantic V2 + pydantic-settings
• Concurrency: asyncio worker loop
• Provider integration: OpenRouter adapter behind provider interface
• Quality and tests: Ruff, ty, pytest, pytest-asyncio

Runtime Topology

flowchart LR
U[Browser User] --> A[FastAPI + NiceGUI App]
A --> W[Asyncio Worker]
A --> DB[(SQLite/PostgreSQL Model)]
W --> P[Provider Adapter]
W --> DB

Layered Boundaries

Interface Layer

• src/transcription/ui/**
• src/transcription/api/**

Responsibilities:

• Route registration, page orchestration, presentation adapters.
• Structured user messaging through shared error presenter.
• No direct persistence access from pages/components.

Service and Orchestration Layer

• src/transcription/services/documents.py
• src/transcription/services/people.py
• src/transcription/services/jobs.py
• src/transcription/services/sources.py
• src/transcription/services/evidence.py
• src/transcription/services/store.py
• src/transcription/services/workflows.py

Responsibilities:

• Aggregate ownership and invariants.
• Transaction-aware write helpers.
• Cross-service workflows in orchestration modules (store.py, workflows.py).

Persistence Layer

• src/transcription/db/**

Responsibilities:

• SQLModel definitions, async session/engine runtime, registry bootstrap.
• Loader helpers that enforce explicit eager loading with lazy="raise" relationships.

Provider Layer

• src/transcription/providers/**

Responsibilities:

• Provider API encapsulation.
• Request manifest and transport evidence capture.
• Normalized transcription result contract.

Core Domain Model

• Document owns archival metadata and links to Source, Job, and DocumentPerson.
• Source is a document page/file record with selected machine projection and human revision.
• Job is an aggregate processing run with status and frozen prompt/runtime settings.
• JobSource is queue/membership state for one (job, source) pair.
• ExecutionAttempt is append-only evidence for each provider call.
• DocumentType and PersonRole are UUID-backed registries with optional protected semantic_key.

Processing and Evidence Workflow

1. User creates/updates Document metadata and linked People atomically through workflow orchestration.
2. User creates a Job by uploading one or more Source files or by retranscribing an existing Source.
3. Source files are validated and stored; orientation normalization may be applied at ingest, and stored bytes become the canonical processing bytes.
4. Worker claims queued Job, transitions to processing, and processes pending pages in deterministic order.
5. Each provider call writes one immutable ExecutionAttempt with:
   • request manifest + hash
   • transport evidence (when response exists)
   • SDK snapshot and normalized metadata
   • outcome, timing, and error details when applicable
6. JobSource status is updated as queue/projection state; Source.raw_transcription is set on first successful attempt and can be explicitly re-pointed by candidate promotion.
7. Job terminal status resolves to transcribed, partial_success, or failed.

Status Semantics

• Job statuses: queued, processing, transcribed, completed, partial_success, failed
  • Operational success path currently resolves to transcribed.
  • completed remains a recognized legacy-compatible status value.
• JobSource statuses: pending, transcribed, failed, cancelled

Security and Path Handling Boundaries

• Print media delivery uses record-validated API route:
  • src/transcription/api/v4_print.py
• General UI media links resolve through:
  • src/transcription/ui/components/media_urls.py
• Local filesystem paths must never be accepted from user input as trusted media routes.

Concurrency and Reliability Principles

• Worker loop reuses service bundle/provider resources for pooled calls.
• Provider-call timeout is explicit and bounded.
• Non-retriable worker-loop faults are surfaced and stop loop spin.
• Per-page outcomes are durably persisted before processing next page.

Related References

• System Requirements
• Data Model
• Error Handling Policy
• V4 Revision History
• Error Handling invariant
• AI evidence invariant