# System Architecture (Version 4)

This document defines the current V4 architecture baseline.

## Architecture Objectives

- Preserve durable archival records for Documents, Sources, People, and processing runs.
- Execute page transcription asynchronously with bounded worker behavior.
- Preserve append-only machine-attempt evidence with request/response provenance.
- Keep UI, API, service, persistence, and provider boundaries explicit and testable.

## Technical Stack

- **Runtime:** Python 3.12+
- **Web application:** FastAPI + NiceGUI
- **Persistence:** SQLModel / SQLAlchemy (SQLite-first, PostgreSQL-compatible model design)
- **Validation and settings:** Pydantic V2 + pydantic-settings
- **Concurrency:** asyncio worker loop
- **Provider integration:** OpenRouter adapter behind provider interface
- **Quality and tests:** Ruff, ty, pytest, pytest-asyncio

## Runtime Topology

```mermaid
flowchart LR
U[Browser User] --> A[FastAPI + NiceGUI App]
A --> W[Asyncio Worker]
A --> DB[(SQLite/PostgreSQL Model)]
W --> P[Provider Adapter]
W --> DB
```

## Layered Boundaries

### Interface Layer

- `src/transcription/ui/**`
- `src/transcription/api/**`

Responsibilities:

- Route registration, page orchestration, presentation adapters.
- Structured user messaging through shared error presenter.
- No direct persistence access from pages/components.

### Service and Orchestration Layer

- `src/transcription/services/documents.py`
- `src/transcription/services/people.py`
- `src/transcription/services/jobs.py`
- `src/transcription/services/sources.py`
- `src/transcription/services/evidence.py`
- `src/transcription/services/store.py`
- `src/transcription/services/workflows.py`

Responsibilities:

- Aggregate ownership and invariants.
- Transaction-aware write helpers.
- Cross-service workflows in orchestration modules (`store.py`, `workflows.py`).

### Persistence Layer

- `src/transcription/db/**`

Responsibilities:

- SQLModel definitions, async session/engine runtime, registry bootstrap.
- Loader helpers that enforce explicit eager loading with `lazy="raise"` relationships.

### Provider Layer

- `src/transcription/providers/**`

Responsibilities:

- Provider API encapsulation.
- Request manifest and transport evidence capture.
- Normalized transcription result contract.

## Core Domain Model

- `Document` owns archival metadata and links to `Source`, `Job`, and `DocumentPerson`.
- `Source` is a document page/file record with selected machine projection and human revision.
- `Job` is an aggregate processing run with status and frozen prompt/runtime settings.
- `JobSource` is queue/membership state for one `(job, source)` pair.
- `ExecutionAttempt` is append-only evidence for each provider call.
- `DocumentType` and `PersonRole` are UUID-backed registries with optional protected `semantic_key`.

## Processing and Evidence Workflow

1. User creates/updates Document metadata and linked People atomically through workflow orchestration.
2. User creates a Job by uploading one or more Source files or by retranscribing an existing Source.
3. Source files are validated and stored; orientation normalization may be applied at ingest, and stored bytes become the canonical processing bytes.
4. Worker claims queued Job, transitions to `processing`, and processes pending pages in deterministic order.
5. Each provider call writes one immutable `ExecutionAttempt` with:
   - request manifest + hash
   - transport evidence (when response exists)
   - SDK snapshot and normalized metadata
   - outcome, timing, and error details when applicable
6. `JobSource` status is updated as queue/projection state; `Source.raw_transcription` is set on first successful attempt and can be explicitly re-pointed by candidate promotion.
7. Job terminal status resolves to `transcribed`, `partial_success`, or `failed`.

## Status Semantics

- **Job statuses:** `queued`, `processing`, `transcribed`, `partial_success`, `failed`
  - Operational success path resolves to `transcribed`.
- **JobSource statuses:** `pending`, `transcribed`, `failed`, `cancelled`

## Security and Path Handling Boundaries

- Print media delivery uses record-validated API route:
  - `src/transcription/api/v4_print.py`
- General UI media links resolve through:
  - `src/transcription/ui/components/media_urls.py`
- Local filesystem paths must never be accepted from user input as trusted media routes.

## Concurrency and Reliability Principles

- Worker loop reuses service bundle/provider resources for pooled calls.
- Provider-call timeout is explicit and bounded.
- Non-retriable worker-loop faults are surfaced and stop loop spin.
- Per-page outcomes are durably persisted before processing next page.

## Design Decisions and Rationale

### Why `transcribed` is the success terminal state

- The worker and job orchestration resolve successful completion to `JobStatus.TRANSCRIBED`, with mixed and failure outcomes represented by `partial_success` and `failed`.
- This keeps terminal status vocabulary aligned with what the pipeline actually produces: transcribed page content and evidence, not a generic completion marker.

### Why evidence history is append-only while page text is a projection

- `ExecutionAttempt` stores immutable per-call evidence and preserves full attempt history across retries.
- `Source.raw_transcription` is intentionally a mutable projection so UI and exports can show a selected current machine text without mutating historical evidence.
- This split keeps auditability and UX both first-class: history is durable, presentation is editable.

### Why orchestration modules own cross-service workflows

- Service modules do not import each other; aggregate ownership remains local to each service.
- Multi-aggregate writes are coordinated in orchestration modules (`store.py`, `workflows.py`) so transaction boundaries are explicit and testable.
- This avoids circular dependencies and keeps cross-cutting workflow logic centralized.

### Why explicit eager loading is required

- ORM relationships are configured with `lazy="raise"` in key paths, so code must request needed relationships up front.
- This prevents hidden query behavior in UI/service code and makes read shape deterministic and reviewable.

### Why canonical source bytes may be ingest-normalized

- Ingest normalization can correct orientation before persistence so provider calls, evidence hashes, and rendered processing source are consistent.
- The canonical stored bytes, digest, and size become the durable processing identity for that source.

### Why media access uses controlled routes/helpers

- Print/export media uses record-validated API endpoints to avoid direct filesystem path exposure.
- General UI media URLs are generated through shared resolver helpers to keep path handling consistent and centralized.

## Scope Boundary

Current architecture rules live only in `docs/ver4/*`.

## Related References

- [System Requirements](requirements_v4.md)
- [Data Model](schema_v4.md)
- [Error Handling Policy](error_handling_v4.md)
- [Error Handling invariant](../invariant/error_handling.md)
- [AI evidence invariant](../invariant/ai_evidence_and_provenance.md)