transcription/docs/schema.md

Database schema

This document describes the structure of the database underlying the personal historical-document transcription system.

---

Schema diagram

erDiagram
    document {
        INTEGER id PK
        TEXT name
    }

    job {
        INTEGER id PK
        INTEGER document_id FK
        TEXT status
        INTEGER retry_count
        DATETIME date_created
        DATETIME date_updated
        TEXT provider
        TEXT model
        TEXT prompt_name
        TEXT text
        TEXT error_detail
    }

    source {
        INTEGER id PK
        INTEGER document_id FK
        INTEGER job_id FK
        TEXT upload_name
        TEXT filename
        TEXT file_path
        DATETIME date_uploaded
    }

    revision {
        INTEGER id PK
        INTEGER source_id FK
        INTEGER revision
        TEXT text
        DATETIME date_created
    }

    document ||--o{ source : "has 0 or more"
    document ||--o{ job : "has 0 or more"
    job ||--o{ source : "processes 0 or more"
    source ||--o{ revision : "has 0 or more"  

---

Table Relationships & Constraints

• A document can consist of 0 or more content sources. A document can have 0 or more jobs.
• A source can belong to only one job (which contains the original transcription). A source can only belong to one document. A source can have 0 or more transcription revisions.
• A job can process one or more sources. A job can belong to only one document.
• A revision can belong to only one source. A source can have 0 or more revisions.
• Job.text stores the original immutable provider transcription.
• Revision rows are optional user-authored edits and may be derived from either the original transcription or a prior revision.

---

Glossary

• Document - Documents consist of one or more content sources and their related transcriptions.
• Source - A content source that is transcribed to text. It can either be an image (.jpg, .tiff, .png) or a PDF (.pdf).
• Image - The scanned image of one page of a document.
• PDF - A PDF containing the image of one or more pages of a document.
• Job - A processing job ingests one or more sources, sends them to an AI model along with a prompt for transcription, then stores the results. The results are immutable, including the original transcription. The user can create a revision of the original transcription, but the user cannot modify the original.
• Transcription - The text contained in a content source. A job creates the original immutable transcription. A user can optionally create a revised transcription, or "revision".
• Revision - A revision is a user-created modification of an existing transcription. A user can create a revision from either the original transcription or a previous revision, but it is entirely optional. Some original transcriptions will have no revisions.