generated from john/python-template
73 lines
2.7 KiB
Markdown
73 lines
2.7 KiB
Markdown
|
|
## Database schema
|
|
This document describes the structure of the database underlying the personal historical-document transcription system.
|
|
|
|
---
|
|
|
|
## Schema diagram
|
|
|
|
```mermaid
|
|
erDiagram
|
|
document {
|
|
INTEGER id PK
|
|
TEXT name
|
|
}
|
|
|
|
job {
|
|
INTEGER id PK
|
|
INTEGER document_id FK
|
|
TEXT status
|
|
INTEGER retry_count
|
|
DATETIME date_created
|
|
DATETIME date_updated
|
|
TEXT provider
|
|
TEXT model
|
|
TEXT prompt_name
|
|
TEXT text
|
|
TEXT error_detail
|
|
}
|
|
|
|
source {
|
|
INTEGER id PK
|
|
INTEGER document_id FK
|
|
INTEGER job_id FK
|
|
TEXT upload_name
|
|
TEXT filename
|
|
TEXT file_path
|
|
DATETIME date_uploaded
|
|
}
|
|
|
|
revision {
|
|
INTEGER id PK
|
|
INTEGER source_id FK
|
|
INTEGER revision
|
|
TEXT text
|
|
DATETIME date_created
|
|
}
|
|
|
|
document ||--o{ source : "has 0 or more"
|
|
document ||--o{ job : "has 0 or more"
|
|
job ||--o{ source : "processes 0 or more"
|
|
source ||--o{ revision : "has 0 or 1"
|
|
```
|
|
|
|
---
|
|
|
|
## Table Relationships & Constraints
|
|
* A document can consist of 0 or more content sources. A document can have 0 or more jobs.
|
|
* A source can belong to only one job (which contains the original transcription). A source can only belong to one document. A source may have one optional transcription revision.
|
|
* A job can process one or more sources. A job can belong to only one document.
|
|
* A revision can belong to only one source. A source may have one optional revision.
|
|
* `Job.text` stores the original immutable provider transcription.
|
|
* Revision rows are optional user-authored edits and are derived from the original transcription. Unlike jobs, revision rows can be updated.
|
|
|
|
---
|
|
|
|
## Glossary
|
|
* **Document** - Documents consist of one or more content sources and their related transcriptions.
|
|
* **Source** - A content source that is transcribed to text. It can either be an image (.jpg, .tiff, .png) or a PDF (.pdf).
|
|
* **Image** - The scanned image of one page of a document.
|
|
* **PDF** - A PDF containing the image of one or more pages of a document.
|
|
* **Job** - A processing job ingests one or more sources, sends them to an AI model along with a prompt for transcription, then stores the results. The results are immutable, *including the original transcription*. The user can create a revision of the original transcription, but the user cannot modify the original.
|
|
* **Transcription** - The text contained in a content source. A job creates the original immutable transcription. A user can optionally create a revised transcription, or "revision".
|
|
* **Revision** - A revision is a user-created modification of an existing transcription. It is optional. Some original transcriptions will have no revisions. |