transcription/docs/schema.md

Data Model and Persistence Schema (Current Baseline: V6.1)

This document is the field-accurate V6.1 schema contract aligned to src/transcription/db/models.py.

Source of Truth Anchors

• src/transcription/db/models.py (status and purpose enums, including maintenance lifecycle enums)
• src/transcription/db/models.py:80-120 (DocumentType, PersonRole)
• src/transcription/db/models.py:122-172 (Tag, Document)
• src/transcription/db/models.py (Person, GenealogyPerson, GenealogyFamily, GenealogyFamilyChild, GenealogyCitation)
• src/transcription/db/models.py (Photo, DocumentPerson, DocumentTag)
• src/transcription/db/models.py:285-347 (Job)
• src/transcription/db/models.py (MaintenanceRun)
• src/transcription/db/models.py:350-462 (Source, JobSource)
• src/transcription/db/models.py:465-522 (ExecutionAttempt)

Entity Relationship Overview

erDiagram
    DocumentType ||--o{ Document : classifies
    Document ||--o{ Job : has
    Document ||--o{ Source : has
    Document ||--o{ DocumentPerson : links
    Document ||--o{ DocumentTag : tagged
    Person ||--o{ DocumentPerson : links
    Person ||--o{ PersonTag : tagged
    Person ||--o{ Photo : owns
    GenealogyPerson ||--o{ GenealogyFamily : husband
    GenealogyPerson ||--o{ GenealogyFamily : wife
    GenealogyPerson ||--o{ GenealogyFamilyChild : child
    GenealogyFamily ||--o{ GenealogyFamilyChild : includes
    GenealogyPerson ||--o{ GenealogyCitation : cited
    GenealogyFamily ||--o{ GenealogyCitation : cited
    Document ||--o{ GenealogyCitation : evidence
    PersonRole ||--o{ DocumentPerson : labels
    Tag ||--o{ DocumentTag : labels
    Tag ||--o{ PersonTag : labels
    Job ||--o{ JobSource : includes
    Source ||--o{ JobSource : participates
    JobSource ||--o{ ExecutionAttempt : attempts
    MaintenanceRun {
        uuid id PK
    }

Authoritative Enumerations

JobStatus

• queued
• processing
• transcribed
• partial_success
• failed

JobSourceStatus

• pending
• transcribed
• failed
• cancelled

JobPurpose

• transcription
• retranscription

MaintenanceJobType

• backup
• storage_reconciliation
• gedcom_import

MaintenanceRunStatus

• queued
• processing
• succeeded
• failed

Field-Accurate Table Contracts

DocumentType

Field	Type	Notes
id	UUID	PK
semantic_key	str | None	nullable unique, indexed
label	str	required
normalized_label	str	unique, indexed
is_active	bool	default True
created_at	datetime	default now
updated_at	datetime	default now, onupdate

PersonRole

Field	Type	Notes
id	UUID	PK
semantic_key	str | None	nullable unique, indexed
label	str	required
normalized_label	str	unique, indexed
is_active	bool	default True
created_at	datetime	default now
updated_at	datetime	default now, onupdate

Tag

Field	Type	Notes
id	UUID	PK
semantic_key	str | None	nullable unique, indexed
label	str	required
normalized_label	str	unique, indexed
is_active	bool	default True
created_at	datetime	default now
updated_at	datetime	default now, onupdate

Document

Field	Type	Notes
id	UUID	PK
name	str	required
document_type_id	UUID | None	FK -> document_type.id, indexed
document_date	date | None	optional
document_date_raw	str | None	optional
location_created	str | None	optional
notes	str | None	optional
archive_identifier	str | None	optional
created_at	datetime	default now
updated_at	datetime	default now, onupdate

Person

Field	Type	Notes
id	UUID	PK
last_name	str	required
given_names	str	required
birth_date	date | None	optional
birth_date_raw	str | None	optional
birth_place	str | None	optional
death_date	date | None	optional
death_date_raw	str | None	optional
death_place	str | None	optional
biography	str | None	optional
family_search_id	str | None	nullable unique
metadata_	dict[str, JsonValue] | None	stored as DB column metadata (JSONBCompat)
created_at	datetime	default now
updated_at	datetime	default now, onupdate

GenealogyPerson

Field	Type	Notes
id	UUID	PK
fs_id	str	unique, indexed FamilySearch identifier
full_name	str	required
birth_date	date | None	optional
birth_date_raw	str | None	optional
birth_place	str | None	optional
death_date	date | None	optional
death_date_raw	str | None	optional
death_place	str | None	optional
created_at	datetime	default now
updated_at	datetime	default now, onupdate

GenealogyFamily

Field	Type	Notes
id	UUID	PK
fs_family_id	str	unique, indexed FamilySearch family identifier
husband_id	UUID | None	nullable FK -> genealogy_person.id, indexed
wife_id	UUID | None	nullable FK -> genealogy_person.id, indexed
marriage_date	date | None	optional
marriage_date_raw	str | None	optional
marriage_place	str | None	optional
created_at	datetime	default now
updated_at	datetime	default now, onupdate

GenealogyFamilyChild

Field	Type	Notes
id	UUID	PK
family_id	UUID	FK -> genealogy_family.id, indexed
child_id	UUID	FK -> genealogy_person.id, indexed
relationship_type	str | None	optional
created_at	datetime	default now

Constraint:

• UniqueConstraint(family_id, child_id) named uq_genealogy_family_child

GenealogyCitation

Field	Type	Notes
id	UUID	PK
genealogy_person_id	UUID | None	nullable FK -> genealogy_person.id, indexed
genealogy_family_id	UUID | None	nullable FK -> genealogy_family.id, indexed
fact_type	GenealogyCitationFactType	enum: birth, death, marriage, other
raw_citation_text	str	required raw GEDCOM citation text
source_kind	GenealogyCitationSourceKind	enum: familysearch_imported, transcription_evidence
document_id	UUID | None	nullable FK -> document.id, indexed
created_at	datetime	default now

Photo

Field	Type	Notes
id	UUID	PK
person_id	UUID | None	nullable FK -> person.id, indexed (NULL = homepage photo)
path	str	required upload-root-relative POSIX path (photos/...)
description	str | None	optional
is_primary	bool	default False; owner-level "featured/primary" marker
created_at	datetime	default now
updated_at	datetime	default now, onupdate

DocumentPerson

Field	Type	Notes
id	UUID	PK
document_id	UUID	FK -> document.id, indexed
person_id	UUID	FK -> person.id, indexed
role_id	UUID	FK -> person_role.id, indexed
created_at	datetime	default now
updated_at	datetime	default now, onupdate

Constraint:

• UniqueConstraint(document_id, person_id) named uq_document_person

DocumentTag

Field	Type	Notes
id	UUID	PK
document_id	UUID	FK -> document.id, indexed
tag_id	UUID	FK -> tag.id, indexed
created_at	datetime	default now
updated_at	datetime	default now, onupdate

Constraint:

• UniqueConstraint(document_id, tag_id) named uq_document_tag

PersonTag

Field	Type	Notes
id	UUID	PK
person_id	UUID	FK -> person.id, indexed
tag_id	UUID	FK -> tag.id, indexed
created_at	datetime	default now
updated_at	datetime	default now, onupdate

Constraint:

• UniqueConstraint(person_id, tag_id) named uq_person_tag

Job

Field	Type	Notes
id	UUID	PK
document_id	UUID	FK -> document.id, indexed
status	JobStatus	non-null enum (stored as enum values)
retry_count	int	default 0, ge=0
purpose	JobPurpose	non-null enum, default transcription
date_created	datetime	default now
date_updated	datetime	default now, onupdate
provider	str | None	optional
model	str | None	optional
prompt_name	str | None	optional
prompt_hash	str | None	optional
system_prompt	str | None	optional
user_prompt	str | None	optional
temperature	float | None	optional
top_p	float | None	optional

Index:

• Index("ix_job_status_date_created", "status", "date_created")

MaintenanceRun

Field	Type	Notes
id	UUID	PK
job_type	MaintenanceJobType	non-null enum
status	MaintenanceRunStatus	non-null enum, default queued
started_at	datetime | None	optional
finished_at	datetime | None	optional
triggered_by	str | None	optional
summary	str | None	optional
log_path	str | None	optional, log-root-relative POSIX path
error_detail	str | None	optional internal failure detail
created_at	datetime	default now
updated_at	datetime	default now, onupdate

Source

Field	Type	Notes
id	UUID	PK
document_id	UUID	FK -> document.id, indexed
page_number	int	default 1, ge=1
upload_name	str	required
filename	str	required
file_path	str	required upload-root-relative POSIX path (documents/...)
file_hash	str	required
file_size_bytes	int	BigInteger, non-null
raw_transcription	str | None	projection field
preferred_execution_attempt_id	UUID | None	nullable FK -> execution_attempt.id, indexed (use_alter)
revised_text	str | None	optional human revision
date_uploaded	datetime	default now
date_revised	datetime | None	optional

JobSource

Field	Type	Notes
id	UUID	PK
job_id	UUID	FK -> job.id, indexed
source_id	UUID	FK -> source.id, indexed
status	JobSourceStatus	non-null enum, default pending

Constraint:

• UniqueConstraint(job_id, source_id) named uq_job_source_job_source

Runtime reconciliation:

• Startup database operations remove retired V4.6 job_source evidence columns (raw_transcription, ai_metadata, raw_api_response, error_detail, executed_at) when present so persisted schema matches this contract.

ExecutionAttempt

Field	Type	Notes
id	UUID	PK
job_source_id	UUID	FK -> job_source.id, indexed
job_id	UUID	FK -> job.id, indexed
source_id	UUID	FK -> source.id, indexed
attempt_number	int	ge=1
status	JobSourceStatus	non-null enum, value-stable with JobSource.status
provider	str	required
model	str | None	optional
request_manifest	dict[str, JsonValue] | None	JSONBCompat
request_manifest_sha256	str | None	optional
request_manifest_schema_version	str | None	optional
response_received	bool	default False
transport_status_code	int | None	optional
transport_body	bytes | None	LargeBinary
transport_content_type	str | None	optional
transport_content_encoding	str | None	optional
transport_safe_headers	dict[str, JsonValue] | None	JSONBCompat
router_request_id	str | None	optional
router_generation_id	str | None	optional
sdk_response_snapshot	dict[str, JsonValue] | None	JSONBCompat
normalized_metadata	dict[str, JsonValue] | None	JSONBCompat; may include app-namespaced processing_timing (provider_call_duration_ms, processing_duration_ms)
software_context	dict[str, JsonValue] | None	JSONBCompat
raw_transcription	str | None	optional
error_category	str | None	optional
error_detail	str | None	optional
failure_phase	str | None	optional
started_at	datetime	required
finished_at	datetime	required
duration_ms	int	ge=0
created_at	datetime	default now

Constraint:

• UniqueConstraint(job_id, source_id, attempt_number) named uq_execution_attempt_number

Relationship Loading Contract

• Most ORM relationships are configured with lazy="raise".
• JobSource.execution_attempts is intentionally lazy="noload" with ordered attempts.
• Service/UI read paths must explicitly eager-load required relationships before access.

Persistence Invariants (Ground Truth)

1. ExecutionAttempt is append-only runtime evidence.
2. JobSource.status represents queue/projection execution state and is not a full evidence container.
3. Source.raw_transcription is a mutable projection and not authoritative attempt history.
4. Job terminal status derives from page outcomes (JobSource state), not from a separate summary table.
5. DocumentType.semantic_key and PersonRole.semantic_key are nullable-unique semantic identifiers.

Cross-Reference

• System Architecture
• System Requirements
• Error Handling Policy
• AI Evidence and Provenance Invariant