Database Schema (Version 2)

This document describes the PostgreSQL relational schema for the transcription platform. It incorporates multi-image batch orchestration via asyncio, page-level execution tracking, many-to-many author/recipient attribution, and JSONB document storage for AI vision outputs.

All primary and foreign keys are PostgreSQL native UUIDs (gen_random_uuid()).

Entity Relationship Diagram

erDiagram
    PERSON {
        UUID id PK
        TEXT full_name
        TEXT display_name
        TEXT maiden_name
        DATE birth_date
        TEXT birth_date_raw
        TEXT birth_place
        DATE death_date
        TEXT death_date_raw
        TEXT death_place
        TEXT biography
        TEXT portrait_path
        JSONB metadata
        TIMESTAMPTZ created_at
        TIMESTAMPTZ updated_at
    }

    DOCUMENT {
        UUID id PK
        TEXT name
        TEXT document_type
        DATE document_date
        TEXT document_date_raw
        TEXT location_created
        TEXT notes
        TEXT archive_identifier
        TIMESTAMPTZ created_at
        TIMESTAMPTZ updated_at
    }

    DOCUMENT_PERSON {
        UUID id PK
        UUID document_id FK
        UUID person_id FK
        VARCHAR role "author | recipient"
        TIMESTAMPTZ created_at
    }

    JOB {
        UUID id PK
        UUID document_id FK
        VARCHAR status "queued | processing | transcribed | completed | partial_success | failed"
        INTEGER retry_count
        TEXT provider
        TEXT model
        TEXT prompt_name
        TIMESTAMPTZ date_created
        TIMESTAMPTZ date_updated
    }

    SOURCE {
        UUID id PK
        UUID document_id FK
        INTEGER page_number
        TEXT upload_name
        TEXT filename
        TEXT file_path
        TEXT raw_transcription
        TEXT revised_text
        TIMESTAMPTZ date_uploaded
        TIMESTAMPTZ date_revised
    }

    JOB_SOURCE {
        UUID id PK
        UUID job_id FK
        UUID source_id FK
        VARCHAR status "pending | transcribed | failed"
        TEXT raw_transcription
        JSONB ai_metadata
        JSONB raw_api_response
        TEXT error_detail
        TIMESTAMPTZ executed_at
    }

    DOCUMENT ||--o{ DOCUMENT_PERSON : "has_people"
    PERSON ||--o{ DOCUMENT_PERSON : "participates_in"
    DOCUMENT ||--o{ JOB : "has_jobs"
    DOCUMENT ||--o{ SOURCE : "contains_pages"
    JOB ||--o{ JOB_SOURCE : "executes"
    SOURCE ||--o{ JOB_SOURCE : "processed_in"

Domain Invariants & Rules

Page-Level Execution & AI Outputs

Execution Granularity: Every single image execution by an AI model produces a dedicated record in job_source.
Source vs Execution Status: source does not carry a status column. Per-source execution state is tracked in job_source.status (pending, transcribed, failed).
Point-in-Time Auditability: job_source.raw_api_response stores the unparsed REST response envelope for that specific image page call. job_source.ai_metadata stores spatial bounding boxes, token usage, and layout details for that specific image page call.
Active Output Caching: Upon successful completion of an image call, source.raw_transcription is updated with the latest output string from job_source.raw_transcription for fast UI rendering.

Page Ordering & Revisions

Sequential Integrity: source.page_number dictates page ordering within a document. Reads assembling full documents must query ORDER BY source.document_id, source.page_number ASC.
Inlined Human Corrections: User edits occur at the page level inside source.revised_text. source.raw_transcription remains immutable. If source.revised_text is non-null, application frontends must render source.revised_text.

Async Job Lifecycle & Failure Isolation

Batch Orchestrator: A job represents an overarching execution run across one or more source images belonging to a document.
Isolated Failures: API requests run concurrently (e.g., using asyncio). A failure on page 3 does not invalidate successful transcriptions on page 1 or 2.
Job States:
- queued: Created, awaiting worker execution.
- processing: Concurrent HTTP tasks actively running.
- completed: 100% of linked job_source tasks succeeded (transcribed).
- partial_success: At least one job_source succeeded and at least one failed.
- failed: All linked job_source tasks failed or a job-level runtime error occurred.

Attribution & Person Roles

Multi-Person Roles: Documents support zero, one, or many authors and recipients linked via document_person.
Role Uniqueness: (document_id, person_id, role) must be unique to prevent duplicate role tagging.

5.0 KiB Raw Blame History