# Data Model and Persistence Schema (Version 4)

This schema reflects the current V4 persistence contract.

## Entity Relationship Overview

```mermaid
erDiagram
    Document ||--o{ Source : has
    Document ||--o{ Job : has
    Document ||--o{ DocumentPerson : links
    Person ||--o{ DocumentPerson : links
    Job ||--o{ JobSource : includes
    Source ||--o{ JobSource : participates
    Source ||--o{ ExecutionAttempt : records

    Document {
      uuid id PK
      string title
      uuid type_id FK
      string language
      datetime doc_date
      string date_note
      string comments
      string location
      datetime created_at
      datetime updated_at
    }

    Source {
      uuid id PK
      uuid document_id FK
      string original_name
      string media_type
      string storage_path
      int file_size
      string file_hash
      string raw_transcription
      datetime created_at
      datetime updated_at
    }

    Job {
      uuid id PK
      uuid document_id FK
      enum status
      string prompt
      json model_settings_json
      datetime created_at
      datetime updated_at
    }

    JobSource {
      uuid job_id FK
      uuid source_id FK
      enum status
      uuid selected_attempt_id FK
      string error_message
      datetime created_at
      datetime updated_at
    }

    ExecutionAttempt {
      uuid id PK
      uuid source_id FK
      uuid job_id FK
      enum outcome
      string provider_name
      string provider_model
      string request_manifest_hash
      json request_manifest_json
      json transport_evidence_json
      string transcript_text
      string error_category
      string error_message
      float duration_seconds
      datetime started_at
      datetime completed_at
      datetime created_at
    }
```

## Authoritative Enumerations

### JobStatus

- `queued`
- `processing`
- `transcribed`
- `partial_success`
- `failed`

### JobSourceStatus

- `pending`
- `transcribed`
- `failed`
- `cancelled`

## Aggregate Ownership

- `Document` aggregate: `Document`, linked `Source`, linked `DocumentPerson`.
- `Job` aggregate: `Job`, `JobSource` rows, selected-attempt pointers.
- Evidence aggregate: append-only `ExecutionAttempt` rows keyed by `source_id` + `job_id`.

## Persistence Invariants

1. `ExecutionAttempt` rows are immutable after creation, except explicit support fields reserved for compatibility migrations.
2. `JobSource.status` is queue/projection state; it does not duplicate full attempt payload.
3. `Source.raw_transcription` is a projection, not the complete evidence record.
4. `Job` terminal status is derived from `JobSource` outcomes.
5. Registry semantic keys, when present, are immutable once created.

## Schema Design Rationale

### Why `ExecutionAttempt` exists alongside `JobSource`

- `JobSource` is the mutable queue/projection row for workflow control and current-facing page outcome state.
- `ExecutionAttempt` is the durable, append-only evidence timeline for each provider call.
- Keeping both avoids overloading one table with competing concerns (queue state vs immutable audit history).

### Why `Source.raw_transcription` remains on `Source`

- `Source` needs a stable, current projection for UI and print behavior.
- Projection reads are fast and direct, while deep historical inspection remains available through attempts.
- Explicit candidate promotion updates the projection pointer without rewriting history.

### Why semantic registries use UUID identity plus optional semantic keys

- UUIDs are durable relationship identifiers for public/domain links.
- Optional immutable semantic keys support protected built-ins without exposing internal meaning as external API identity.
- Labels can evolve without breaking relationships.

### Why status enums are narrow

- Restricting `JobStatus` and `JobSourceStatus` keeps lifecycle transitions explicit and testable.
- Queue/state transitions and terminal derivation logic remain deterministic across worker and UI flows.

## Media Storage Semantics

1. `Source.storage_path` references canonical stored bytes used by processing.
2. Canonical stored bytes may reflect ingest-time normalization.
3. File hash and size fields describe canonical stored bytes.

## Query and Loading Requirements

- Relationship access from service/UI layers must use explicit eager loading patterns compatible with `lazy="raise"`.
- Candidate-attempt views should select latest/selected attempts explicitly; do not rely on implicit lazy traversal.

## Evolution and Migration Policy

- Additive schema evolution is preferred for evidence-bearing records.
- Deprecated semantics should be removed only when model enums, service logic, tests, and docs are updated together.
- Historical V4.x schema discussions are archived under tag `docs-v4x-archive`; this file is the active contract.

## Cross-Reference

- [System Architecture](architecture_v4.md)
- [System Requirements](requirements_v4.md)
- [Error Handling Policy](error_handling_v4.md)