generated from john/python-template
V1 mostly complete except for some testing. Linting in the last step changed nearly every file which is why this commit is so larger.
This commit is contained in:
+49
-36
@@ -1,21 +1,23 @@
|
||||
## Database Schema (V1 Baseline)
|
||||
|
||||
## Database schema
|
||||
This document describes the structure of the database underlying the personal historical-document transcription system.
|
||||
This document describes the current relational schema for the transcription system.
|
||||
|
||||
All primary and foreign keys in the domain models are UUID-based in V1.
|
||||
|
||||
---
|
||||
|
||||
## Schema diagram
|
||||
## Schema Diagram
|
||||
|
||||
```mermaid
|
||||
```mermaid
|
||||
erDiagram
|
||||
document {
|
||||
INTEGER id PK
|
||||
DOCUMENT {
|
||||
UUID id PK
|
||||
TEXT name
|
||||
}
|
||||
|
||||
job {
|
||||
INTEGER id PK
|
||||
INTEGER document_id FK
|
||||
JOB {
|
||||
UUID id PK
|
||||
UUID document_id FK
|
||||
TEXT status
|
||||
INTEGER retry_count
|
||||
DATETIME date_created
|
||||
@@ -27,48 +29,59 @@ erDiagram
|
||||
TEXT error_detail
|
||||
}
|
||||
|
||||
source {
|
||||
INTEGER id PK
|
||||
INTEGER document_id FK
|
||||
INTEGER job_id FK
|
||||
SOURCE {
|
||||
UUID id PK
|
||||
UUID document_id FK
|
||||
UUID job_id FK
|
||||
TEXT upload_name
|
||||
TEXT filename
|
||||
TEXT file_path
|
||||
DATETIME date_uploaded
|
||||
}
|
||||
|
||||
revision {
|
||||
INTEGER id PK
|
||||
INTEGER source_id FK
|
||||
REVISION {
|
||||
UUID id PK
|
||||
UUID source_id FK UNIQUE
|
||||
INTEGER revision
|
||||
TEXT text
|
||||
DATETIME date_created
|
||||
}
|
||||
|
||||
document ||--o{ source : "has 0 or more"
|
||||
document ||--o{ job : "has 0 or more"
|
||||
job ||--o{ source : "processes 0 or more"
|
||||
source ||--o{ revision : "has 0 or 1"
|
||||
DOCUMENT ||--o{ SOURCE : has_many
|
||||
DOCUMENT ||--o{ JOB : has_many
|
||||
JOB ||--o{ SOURCE : referenced_by
|
||||
SOURCE ||--o| REVISION : has_optional_one
|
||||
```
|
||||
|
||||
---
|
||||
---
|
||||
|
||||
## Table Relationships & Constraints
|
||||
* A document can consist of 0 or more content sources. A document can have 0 or more jobs.
|
||||
* A source can belong to only one job (which contains the original transcription). A source can only belong to one document. A source may have one optional transcription revision.
|
||||
* A job can process one or more sources. A job can belong to only one document.
|
||||
* A revision can belong to only one source. A source may have one optional revision.
|
||||
* 1:1 optionality is enforced by uniqueness on `revision.source_id` (no revision history chain).
|
||||
* `Job.text` stores the original immutable provider transcription.
|
||||
* Revision rows are optional user-authored edits and are derived from the original transcription. Unlike jobs, revision rows can be updated.
|
||||
## Table Relationships and Constraints
|
||||
|
||||
- A `Document` can have zero or more `Source` records.
|
||||
- A `Document` can have zero or more `Job` records.
|
||||
- A `Source` belongs to exactly one `Document` and one `Job`.
|
||||
- A `Source` may have one optional `Revision`.
|
||||
- Optional `0..1` revision cardinality is enforced by uniqueness on `revision.source_id`.
|
||||
|
||||
### Invariants
|
||||
|
||||
- `Job.text` stores immutable original provider transcription output.
|
||||
- `Revision` rows are optional user-authored edits derived from original transcription.
|
||||
- Revisions do not overwrite original `Job.text`.
|
||||
- Job status lifecycle values are: `queued`, `processing`, `transcribed`, `failed`.
|
||||
|
||||
### Timestamp Fields
|
||||
|
||||
- `Job.date_created`
|
||||
- `Job.date_updated`
|
||||
- `Source.date_uploaded`
|
||||
- `Revision.date_created`
|
||||
|
||||
---
|
||||
|
||||
## Glossary
|
||||
* **Document** - Documents consist of one or more content sources and their related transcriptions.
|
||||
* **Source** - A content source that is transcribed to text. It can either be an image (.jpg, .tiff, .png) or a PDF (.pdf).
|
||||
* **Image** - The scanned image of one page of a document.
|
||||
* **PDF** - A PDF containing the image of one or more pages of a document.
|
||||
* **Job** - A processing job ingests one or more sources, sends them to an AI model along with a prompt for transcription, then stores the results. The results are immutable, *including the original transcription*. The user can create a revision of the original transcription, but the user cannot modify the original.
|
||||
* **Transcription** - The text contained in a content source. A job creates the original immutable transcription. A user can optionally create a revised transcription, or "revision".
|
||||
* **Revision** - A revision is a user-created modification of an existing transcription. It is optional. Some original transcriptions will have no revisions.
|
||||
|
||||
- **Document**: logical grouping for one or more transcribed sources.
|
||||
- **Source**: uploaded file content (image/PDF) linked to a job.
|
||||
- **Job**: processing record that stores lifecycle status and original output.
|
||||
- **Revision**: optional single user-authored edited text linked to a source.
|
||||
|
||||
Reference in New Issue
Block a user