transcription/docs/ver4/schema_v4.md

Data Model and Persistence Schema (Version 4)

This schema reflects the current V4 persistence contract.

Entity Relationship Overview

erDiagram
    Document ||--o{ Source : has
    Document ||--o{ Job : has
    Document ||--o{ DocumentPerson : links
    Person ||--o{ DocumentPerson : links
    Job ||--o{ JobSource : includes
    Source ||--o{ JobSource : participates
    Source ||--o{ ExecutionAttempt : records

    Document {
      uuid id PK
      string title
      uuid type_id FK
      string language
      datetime doc_date
      string date_note
      string comments
      string location
      datetime created_at
      datetime updated_at
    }

    Source {
      uuid id PK
      uuid document_id FK
      string original_name
      string media_type
      string storage_path
      int file_size
      string file_hash
      string raw_transcription
      datetime created_at
      datetime updated_at
    }

    Job {
      uuid id PK
      uuid document_id FK
      enum status
      string prompt
      json model_settings_json
      datetime created_at
      datetime updated_at
    }

    JobSource {
      uuid job_id FK
      uuid source_id FK
      enum status
      uuid selected_attempt_id FK
      string error_message
      datetime created_at
      datetime updated_at
    }

    ExecutionAttempt {
      uuid id PK
      uuid source_id FK
      uuid job_id FK
      enum outcome
      string provider_name
      string provider_model
      string request_manifest_hash
      json request_manifest_json
      json transport_evidence_json
      string transcript_text
      string error_category
      string error_message
      float duration_seconds
      datetime started_at
      datetime completed_at
      datetime created_at
    }

Authoritative Enumerations

JobStatus

• queued
• processing
• transcribed
• partial_success
• failed

JobSourceStatus

• pending
• transcribed
• failed
• cancelled

Aggregate Ownership

• Document aggregate: Document, linked Source, linked DocumentPerson.
• Job aggregate: Job, JobSource rows, selected-attempt pointers.
• Evidence aggregate: append-only ExecutionAttempt rows keyed by source_id + job_id.

Persistence Invariants

1. ExecutionAttempt rows are immutable after creation, except explicit support fields reserved for compatibility migrations.
2. JobSource.status is queue/projection state; it does not duplicate full attempt payload.
3. Source.raw_transcription is a projection, not the complete evidence record.
4. Job terminal status is derived from JobSource outcomes.
5. Registry semantic keys, when present, are immutable once created.

Schema Design Rationale

Why ExecutionAttempt exists alongside JobSource

• JobSource is the mutable queue/projection row for workflow control and current-facing page outcome state.
• ExecutionAttempt is the durable, append-only evidence timeline for each provider call.
• Keeping both avoids overloading one table with competing concerns (queue state vs immutable audit history).

Why Source.raw_transcription remains on Source

• Source needs a stable, current projection for UI and print behavior.
• Projection reads are fast and direct, while deep historical inspection remains available through attempts.
• Explicit candidate promotion updates the projection pointer without rewriting history.

Why semantic registries use UUID identity plus optional semantic keys

• UUIDs are durable relationship identifiers for public/domain links.
• Optional immutable semantic keys support protected built-ins without exposing internal meaning as external API identity.
• Labels can evolve without breaking relationships.

Why status enums are narrow

• Restricting JobStatus and JobSourceStatus keeps lifecycle transitions explicit and testable.
• Queue/state transitions and terminal derivation logic remain deterministic across worker and UI flows.

Media Storage Semantics

1. Source.storage_path references canonical stored bytes used by processing.
2. Canonical stored bytes may reflect ingest-time normalization.
3. File hash and size fields describe canonical stored bytes.

Query and Loading Requirements

• Relationship access from service/UI layers must use explicit eager loading patterns compatible with lazy="raise".
• Candidate-attempt views should select latest/selected attempts explicitly; do not rely on implicit lazy traversal.

Evolution and Migration Policy

• Additive schema evolution is preferred for evidence-bearing records.
• Deprecated semantics should be removed only when model enums, service logic, tests, and docs are updated together.
• Historical V4.x schema discussions are archived under tag docs-v4x-archive; this file is the active contract.

Cross-Reference

• System Architecture
• System Requirements
• Error Handling Policy