transcription/docs/implementation_plan_v2.md

implementation_plan_v2

Goal

Replace the current V1 SQLModel schema with the approved V2 schema and make sure every database operation works through the existing async SQLAlchemy/SQLModel session layer.

Use a fresh database. There will be no migrations, data conversion, legacy compatibility shims, or parallel V1/V2 code paths.

Current Project Impact

• src/transcription/db/models.py still defines the V1 Document, Source, Job, and Revision tables.
• The V2 target adds Person, DocumentPerson, and JobSource, moves revisions onto Source, and removes the direct Source.job_id relationship.
• The engine, session factory, transaction handling, and PostgreSQL async support already exist and do not need to be rewritten.
• Async CRUD currently lives in DocumentService, JobService, TranscriptionService, and the upload record helper. Their queries and eager-loading options depend on V1 relationships.
• Existing tests cover only part of the schema and CRUD surface.

Implementation

1. Update the schema

• Replace the models in src/transcription/db/models.py with the approved V2 tables, enums, relationships, foreign keys, constraints, and indexes.
• Remove Revision, Source.job_id, and the transcription fields that no longer belong on Job.
• Keep create_all() as the schema bootstrap for a fresh database.
• Delete _ensure_sqlite_compat_columns() and all schema patching from src/transcription/db/operations.py.
• Keep the Python models and docs/schema_v2.md consistent.

2. Align the async CRUD methods

• Keep the existing ServiceBase session and transaction pattern.
• Update document CRUD to load and manage its ordered Source rows and DocumentPerson links.
• Update job CRUD and queue queries to use JobSource instead of Source.job_id.
• Add the missing async CRUD operations for Person, Source, DocumentPerson, and JobSource using the existing service style. Do not add another repository abstraction.
• Replace revision CRUD with direct updates to Source.revised_text and Source.date_revised.
• Remove the temporary transcript compatibility aliases instead of redirecting them.
• Update only direct database call sites that construct or query these records; UI and worker feature changes are not part of this work.

3. Verify the schema and CRUD

• Update the schema bootstrap test to expect person, document, document_person, source, job, and job_source, with no revision table.
• Add async create, read, update, delete, list, and filtered-query tests for each entity that exposes those operations.
• Test relationship loading, page ordering, uniqueness constraints, delete behavior, status values, and JobSource JSON fields.
• Test both service-owned sessions and caller-provided sessions so flush/commit behavior remains correct.
• Run the focused database and service tests, then the full suite with uv run pytest.

4. Update the UI for the V2 schema

• Review the UI components and views that display document, job, person, and source data so they reference the V2 schema instead of V1 relationships.
• Update upload, detail, and listing screens to show the new person and source associations, revised-source fields, and the revised status values.
• Keep the UI behavior aligned with the updated service layer and ensure the existing UI tests continue to pass with the V2 data model.
• Consider the guidance in docs/ui_style_guide.md when making UI changes so the updated views remain consistent with the project’s visual and interaction conventions.

Done When

• A fresh database is created directly from the V2 SQLModel metadata.
• All async CRUD methods pass against the V2 relationships and fields.
• No code references Revision, Source.job_id, removed Job transcription fields, or compatibility aliases.
• The focused tests and full test suite pass.

Out of Scope

• Database migrations or preservation of V1 data
• Legacy compatibility code
• Database engine or session-layer rewrites
• UI redesign, batch orchestration, worker concurrency, deployment, and operational runbooks