Begin planning V2

2026-07-30 19:39:12 -05:00
parent a975ca299a
commit d2b793ea69
6 changed files with 457 additions and 4 deletions
@@ -0,0 +1,122 @@
+# Database Schema (V2 Architecture)
+
+This document describes the PostgreSQL relational schema for the transcription platform. It incorporates multi-image batch orchestration via `asyncio`, page-level execution tracking, many-to-many author/recipient attribution, and JSONB document storage for AI vision outputs.
+
+All primary and foreign keys are PostgreSQL native UUIDs (`gen_random_uuid()`).
+
+## Entity Relationship Diagram
+
+```mermaid
+erDiagram
+    PERSON {
+        UUID id PK
+        TEXT full_name
+        TEXT display_name
+        TEXT maiden_name
+        DATE birth_date
+        TEXT birth_date_raw
+        TEXT birth_place
+        DATE death_date
+        TEXT death_date_raw
+        TEXT death_place
+        TEXT biography
+        TEXT portrait_path
+        JSONB metadata
+        TIMESTAMPTZ created_at
+        TIMESTAMPTZ updated_at
+    }
+
+    DOCUMENT {
+        UUID id PK
+        TEXT name
+        TEXT document_type
+        DATE document_date
+        TEXT document_date_raw
+        TEXT location_created
+        TEXT notes
+        TEXT archive_identifier
+        TIMESTAMPTZ created_at
+        TIMESTAMPTZ updated_at
+    }
+
+    DOCUMENT_PERSON {
+        UUID id PK
+        UUID document_id FK
+        UUID person_id FK
+        VARCHAR role "author | recipient"
+        TIMESTAMPTZ created_at
+    }
+
+    JOB {
+        UUID id PK
+        UUID document_id FK
+        VARCHAR status "queued | processing | completed | partial_success | failed"
+        INTEGER retry_count
+        TEXT provider
+        TEXT model
+        TEXT prompt_name
+        TIMESTAMPTZ date_created
+        TIMESTAMPTZ date_updated
+    }
+
+    SOURCE {
+        UUID id PK
+        UUID document_id FK
+        INTEGER page_number
+        TEXT upload_name
+        TEXT filename
+        TEXT file_path
+        TEXT raw_transcription
+        TEXT revised_text
+        TIMESTAMPTZ date_uploaded
+        TIMESTAMPTZ date_revised
+    }
+
+    JOB_SOURCE {
+        UUID id PK
+        UUID job_id FK
+        UUID source_id FK
+        VARCHAR status "pending | transcribed | failed"
+        TEXT raw_transcription
+        JSONB ai_metadata
+        JSONB raw_api_response
+        TEXT error_detail
+        TIMESTAMPTZ executed_at
+    }
+
+    DOCUMENT ||--o{ DOCUMENT_PERSON : "has_people"
+    PERSON ||--o{ DOCUMENT_PERSON : "participates_in"
+    DOCUMENT ||--o{ JOB : "has_jobs"
+    DOCUMENT ||--o{ SOURCE : "contains_pages"
+    JOB ||--o{ JOB_SOURCE : "executes"
+    SOURCE ||--o{ JOB_SOURCE : "processed_in"
+```
+
+## Domain Invariants & Rules
+
+### Page-Level Execution & AI Outputs
+
+* Execution Granularity: Every single image execution by an AI model produces a dedicated record in job_source.
+* Point-in-Time Auditability: job_source.raw_api_response stores the unparsed REST response envelope for that specific image page call. job_source.ai_metadata stores spatial bounding boxes, token usage, and layout details for that specific image page call.
+* Active Output Caching: Upon successful completion of an image call, source.raw_transcription is updated with the latest output string from job_source.raw_transcription for fast UI rendering.
+
+### Page Ordering & Revisions
+
+* Sequential Integrity: source.page_number dictates page ordering within a document. Reads assembling full documents must query ORDER BY source.document_id, source.page_number ASC.
+* Inlined Human Corrections: User edits occur at the page level inside source.revised_text. source.raw_transcription remains immutable. If source.revised_text is non-null, application frontends must render source.revised_text.
+
+### Async Job Lifecycle & Failure Isolation
+
+* Batch Orchestrator: A job represents an overarching execution run across one or more source images belonging to a document.
+* Isolated Failures: API requests run concurrently (e.g., using asyncio). A failure on page 3 does not invalidate successful transcriptions on page 1 or 2.
+* Job States:
+    - queued: Created, awaiting worker execution.
+    - processing: Concurrent HTTP tasks actively running.
+    - completed: 100% of linked job_source tasks succeeded (transcribed).
+    - partial_success: At least one job_source succeeded and at least one failed.
+    - failed: All linked job_source tasks failed or a job-level runtime error occurred.
+
+### Attribution & Person Roles
+
+* Multi-Person Roles: Documents support zero, one, or many authors and recipients linked via document_person.
+* Role Uniqueness: (document_id, person_id, role) must be unique to prevent duplicate role tagging.