Remove references to "v4" throughout the code and documentation

2026-08-20 16:35:20 -05:00
parent eaeb0bc806
commit bf2f3ac09c
34 changed files with 140 additions and 550 deletions
@@ -0,0 +1,237 @@
+# Data Model and Persistence Schema (Version 4)
+
+This document is the field-accurate Version 4 schema contract aligned to `src/transcription/db/models.py`.
+
+## Source of Truth Anchors
+
+- `src/transcription/db/models.py:60-78` (status and purpose enums)
+- `src/transcription/db/models.py:80-120` (`DocumentType`, `PersonRole`)
+- `src/transcription/db/models.py:122-207` (`Document`, `Person`, `DocumentPerson`)
+- `src/transcription/db/models.py:208-255` (`Job`)
+- `src/transcription/db/models.py:273-386` (`Source`, `JobSource`)
+- `src/transcription/db/models.py:387-445` (`ExecutionAttempt`)
+
+## Entity Relationship Overview
+
+```mermaid
+erDiagram
+    DocumentType ||--o{ Document : classifies
+    Document ||--o{ Job : has
+    Document ||--o{ Source : has
+    Document ||--o{ DocumentPerson : links
+    Person ||--o{ DocumentPerson : links
+    PersonRole ||--o{ DocumentPerson : labels
+    Job ||--o{ JobSource : includes
+    Source ||--o{ JobSource : participates
+    JobSource ||--o{ ExecutionAttempt : attempts
+```
+
+## Authoritative Enumerations
+
+### JobStatus
+
+- `queued`
+- `processing`
+- `transcribed`
+- `partial_success`
+- `failed`
+
+### JobSourceStatus
+
+- `pending`
+- `transcribed`
+- `failed`
+- `cancelled`
+
+### JobPurpose
+
+- `transcription`
+- `retranscription`
+
+## Field-Accurate Table Contracts
+
+### `DocumentType`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `semantic_key` | `str \| None` | nullable unique, indexed |
+| `label` | `str` | required |
+| `normalized_label` | `str` | unique, indexed |
+| `is_active` | `bool` | default `True` |
+| `created_at` | `datetime` | default now |
+| `updated_at` | `datetime` | default now, onupdate |
+
+### `PersonRole`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `semantic_key` | `str \| None` | nullable unique, indexed |
+| `label` | `str` | required |
+| `normalized_label` | `str` | unique, indexed |
+| `is_active` | `bool` | default `True` |
+| `created_at` | `datetime` | default now |
+| `updated_at` | `datetime` | default now, onupdate |
+
+### `Document`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `name` | `str` | required |
+| `document_type_id` | `UUID \| None` | FK -> `document_type.id`, indexed |
+| `document_date` | `date \| None` | optional |
+| `document_date_raw` | `str \| None` | optional |
+| `location_created` | `str \| None` | optional |
+| `notes` | `str \| None` | optional |
+| `archive_identifier` | `str \| None` | optional |
+| `created_at` | `datetime` | default now |
+| `updated_at` | `datetime` | default now, onupdate |
+
+### `Person`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `full_name` | `str` | required |
+| `display_name` | `str \| None` | optional |
+| `maiden_name` | `str \| None` | optional |
+| `birth_date` | `date \| None` | optional |
+| `birth_date_raw` | `str \| None` | optional |
+| `birth_place` | `str \| None` | optional |
+| `death_date` | `date \| None` | optional |
+| `death_date_raw` | `str \| None` | optional |
+| `death_place` | `str \| None` | optional |
+| `biography` | `str \| None` | optional |
+| `portrait_path` | `str \| None` | optional |
+| `family_search_id` | `str \| None` | nullable unique |
+| `metadata_` | `dict[str, JsonValue] \| None` | stored as DB column `metadata` (`JSONBCompat`) |
+| `created_at` | `datetime` | default now |
+| `updated_at` | `datetime` | default now, onupdate |
+
+### `DocumentPerson`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `document_id` | `UUID` | FK -> `document.id`, indexed |
+| `person_id` | `UUID` | FK -> `person.id`, indexed |
+| `role_id` | `UUID` | FK -> `person_role.id`, indexed |
+| `created_at` | `datetime` | default now |
+| `updated_at` | `datetime` | default now, onupdate |
+
+Constraint:
+- `UniqueConstraint(document_id, person_id)` named `uq_document_person`
+
+### `Job`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `document_id` | `UUID` | FK -> `document.id`, indexed |
+| `status` | `JobStatus` | non-null enum (stored as enum values) |
+| `retry_count` | `int` | default `0`, `ge=0` |
+| `purpose` | `JobPurpose` | non-null enum, default `transcription` |
+| `date_created` | `datetime` | default now |
+| `date_updated` | `datetime` | default now, onupdate |
+| `provider` | `str \| None` | optional |
+| `model` | `str \| None` | optional |
+| `prompt_name` | `str \| None` | optional |
+| `prompt_hash` | `str \| None` | optional |
+| `system_prompt` | `str \| None` | optional |
+| `user_prompt` | `str \| None` | optional |
+| `temperature` | `float \| None` | optional |
+| `top_p` | `float \| None` | optional |
+
+Index:
+- `Index("ix_job_status_date_created", "status", "date_created")`
+
+### `Source`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `document_id` | `UUID` | FK -> `document.id`, indexed |
+| `page_number` | `int` | default `1`, `ge=1` |
+| `upload_name` | `str` | required |
+| `filename` | `str` | required |
+| `file_path` | `str` | required |
+| `file_hash` | `str` | required |
+| `file_size_bytes` | `int` | `BigInteger`, non-null |
+| `raw_transcription` | `str \| None` | projection field |
+| `preferred_execution_attempt_id` | `UUID \| None` | nullable FK -> `execution_attempt.id`, indexed (`use_alter`) |
+| `revised_text` | `str \| None` | optional human revision |
+| `date_uploaded` | `datetime` | default now |
+| `date_revised` | `datetime \| None` | optional |
+
+### `JobSource`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `job_id` | `UUID` | FK -> `job.id`, indexed |
+| `source_id` | `UUID` | FK -> `source.id`, indexed |
+| `status` | `JobSourceStatus` | non-null enum, default `pending` |
+
+Constraint:
+- `UniqueConstraint(job_id, source_id)` named `uq_job_source_job_source`
+
+### `ExecutionAttempt`
+
+| Field | Type | Notes |
+| :--- | :--- | :--- |
+| `id` | `UUID` | PK |
+| `job_source_id` | `UUID` | FK -> `job_source.id`, indexed |
+| `job_id` | `UUID` | FK -> `job.id`, indexed |
+| `source_id` | `UUID` | FK -> `source.id`, indexed |
+| `attempt_number` | `int` | `ge=1` |
+| `status` | `JobSourceStatus` | non-null enum, value-stable with `JobSource.status` |
+| `provider` | `str` | required |
+| `model` | `str \| None` | optional |
+| `request_manifest` | `dict[str, JsonValue] \| None` | JSONBCompat |
+| `request_manifest_sha256` | `str \| None` | optional |
+| `request_manifest_schema_version` | `str \| None` | optional |
+| `response_received` | `bool` | default `False` |
+| `transport_status_code` | `int \| None` | optional |
+| `transport_body` | `bytes \| None` | LargeBinary |
+| `transport_content_type` | `str \| None` | optional |
+| `transport_content_encoding` | `str \| None` | optional |
+| `transport_safe_headers` | `dict[str, JsonValue] \| None` | JSONBCompat |
+| `router_request_id` | `str \| None` | optional |
+| `router_generation_id` | `str \| None` | optional |
+| `sdk_response_snapshot` | `dict[str, JsonValue] \| None` | JSONBCompat |
+| `normalized_metadata` | `dict[str, JsonValue] \| None` | JSONBCompat; may include app-namespaced `processing_timing` (`provider_call_duration_ms`, `processing_duration_ms`) |
+| `software_context` | `dict[str, JsonValue] \| None` | JSONBCompat |
+| `raw_transcription` | `str \| None` | optional |
+| `error_category` | `str \| None` | optional |
+| `error_detail` | `str \| None` | optional |
+| `failure_phase` | `str \| None` | optional |
+| `started_at` | `datetime` | required |
+| `finished_at` | `datetime` | required |
+| `duration_ms` | `int` | `ge=0` |
+| `created_at` | `datetime` | default now |
+
+Constraint:
+- `UniqueConstraint(job_id, source_id, attempt_number)` named `uq_execution_attempt_number`
+
+## Relationship Loading Contract
+
+- Most ORM relationships are configured with `lazy="raise"`.
+- `JobSource.execution_attempts` is intentionally `lazy="noload"` with ordered attempts.
+- Service/UI read paths must explicitly eager-load required relationships before access.
+
+## Persistence Invariants (Ground Truth)
+
+1. `ExecutionAttempt` is append-only runtime evidence.
+2. `JobSource.status` represents queue/projection execution state and is not a full evidence container.
+3. `Source.raw_transcription` is a mutable projection and not authoritative attempt history.
+4. `Job` terminal status derives from page outcomes (`JobSource` state), not from a separate summary table.
+5. `DocumentType.semantic_key` and `PersonRole.semantic_key` are nullable-unique semantic identifiers.
+
+## Cross-Reference
+
+- [System Architecture](architecture.md)
+- [System Requirements](requirements.md)
+- [Error Handling Policy](error_handling.md)
+- [AI Evidence and Provenance Invariant](../invariant/ai_evidence_and_provenance.md)