ver1 - Step 4 implemented - Revision functionality added

README.md

@@ -42,11 +42,32 @@ PROMPT_DIR=./prompts
 uv run uvicorn transcription.app:create_app --factory --reload
 ```
 
-### 4) Open in browser
+### 4) (Optional) Run explicit migrations/checks
+
+Use the migration runner for Step 4 schema safety workflows:
+
+```bash
+uv run python -m transcription.migration_runner --list
+uv run python -m transcription.migration_runner --apply
+uv run python -m transcription.migration_runner --check
+```
+
+### 5) Open in browser
+
 
 - GUI: [http://[IP_ADDRESS]:8000/ui](http://[IP_ADDRESS]:8000/ui)
 - Health check: [http://[IP_ADDRESS]:8000/healthz](http://[IP_ADDRESS]:8000/healthz)
 
+### Schema safety settings
+
+Optional environment settings (defaults shown):
+
+```env
+MIGRATION_AUTO_APPLY_ON_STARTUP=false
+VALIDATE_SCHEMA_ON_STARTUP=true
+```
+
+
 ## How to navigate the GUI
 
 - **Upload page** (`/ui`)

docs/ver1/ver1-step4-migration-runbook.md

@@ -0,0 +1,113 @@
+# Ver1 Step 4 Migration and Rollback Runbook
+
+## Purpose
+
+Provide a concise, operator-safe procedure for schema migration execution,
+compatibility validation, and rollback/mitigation for personal-scale deployments.
+
+This runbook supports `docs/ver1/ver1-step4.md` and REQ-10 by keeping normal
+production startup non-mutating unless explicitly configured otherwise.
+
+---
+
+## Preconditions
+
+1. Application version to deploy is known and checked out.
+2. `.env` values are configured for target environment.
+3. Database backup path is prepared.
+4. Application process is stopped before migration on production-like systems.
+
+---
+
+## Commands
+
+Use explicit migration runner operations:
+
+1. List pending migrations:
+   - `uv run python -m transcription.migration_runner --list`
+2. Apply pending migrations:
+   - `uv run python -m transcription.migration_runner --apply`
+3. Validate schema compatibility:
+   - `uv run python -m transcription.migration_runner --check`
+
+Recommended execution order:
+
+1. `--list`
+2. backup database
+3. `--apply`
+4. `--check`
+5. start application
+
+---
+
+## Backup Procedure (SQLite Baseline)
+
+For SQLite deployments, copy the DB file before migration:
+
+- Example DB path default: `./transcription.db`
+- Keep timestamped backup copy in a safe location.
+
+If the file is in active use, stop the app first.
+
+---
+
+## Verification Checklist
+
+After migration apply:
+
+1. `--check` exits successfully.
+2. `schema_migration_history` includes applied revisions.
+3. Application starts successfully.
+4. Health endpoint responds: `/healthz`.
+5. Critical flows smoke-check:
+   - upload
+   - job processing
+   - revision listing/acceptance
+
+---
+
+## Rollback and Mitigation Decision Tree
+
+1. If migration fails before changes commit:
+   - fix issue
+   - re-run apply
+2. If migration partially applied or compatibility check fails:
+   - stop app
+   - restore from backup
+   - investigate and produce forward-fix migration if needed
+3. If app starts but functional invariants fail:
+   - stop app
+   - restore backup
+   - add corrective migration/backfill and rehearse before retry
+
+For this Step 4 baseline, backup restore is the primary rollback mechanism.
+
+---
+
+## Failure Classification Guidance
+
+Classify migration failures using `docs/error_handling.md` categories:
+
+- transient connection issues -> `infrastructure_transient_error`
+- permissions/misconfiguration -> `infrastructure_persistent_error`
+- unexpected migration logic defects -> `internal_unexpected_error`
+
+Record failure details with operation context and timestamp.
+
+---
+
+## Operational Notes
+
+- `migration_auto_apply_on_startup` defaults to `False`.
+- `validate_schema_on_startup` defaults to `True`.
+- Startup schema validation fails fast on incompatibility.
+
+This protects production from accidental schema drift.
+
+---
+
+## Post-Step-4 Follow-Up
+
+If migration complexity grows beyond lightweight revision scripts,
+introduce a dedicated migration framework in a future step while preserving
+this runbook structure and operator-first workflow.

docs/ver1/ver1-step4-results.md

@@ -2,111 +2,146 @@
 
 ## Summary
 
-Step 4 implementation status: **in progress**.
+Step 4 implementation status: **complete (baseline scope)**.
 
 This document records completed migration-safety work, validation evidence, and remaining follow-ups for Ver1 Step 4.
 
 Implemented in this step:
 
-1. _TBD_
+1. Added explicit migration framework module with revision history tracking.
-2. _TBD_
+2. Added schema compatibility validation and startup guardrails.
-3. _TBD_
+3. Added migration runner CLI for list/apply/check operations.
-
+4. Added migration tests and Step 4 validation evidence.
+5. Added Step 4 migration/rollback runbook.
 ---
 
 ## Implemented Changes
 
 ### 1) Schema audit and invariant lock
 
-_TBD_
+Implemented read-only compatibility checks in `src/transcription/db.py`:
+
+- `validate_schema_compatibility(...)` verifies required V1 tables:
+  - `document`
+  - `job`
+  - `transcript`
+  - `transcriptrevision`
+- verifies required `job.retry_count` column
+- returns explicit issue identifiers (non-mutating check)
 
 ### 2) Migration policy/tooling lock
 
-_TBD_
+Added explicit migration revision model in `src/transcription/migrations.py`:
+
+- `MigrationRevision` dataclass
+- ordered `MIGRATIONS` registry
+- migration history table: `schema_migration_history`
+- explicit pending-list and apply operations
 
 ### 3) Forward migration implementation
 
-_TBD_
+Implemented two baseline forward migrations:
+
+1. `0001_add_retry_count_to_job`
+2. `0002_create_transcriptrevision_table`
+
+Each migration is idempotent and recorded in migration history.
 
 ### 4) Rollback and mitigation runbook
 
-_TBD_
+Created `docs/ver1/ver1-step4-migration-runbook.md` with:
+
+- preconditions
+- list/apply/check command sequence
+- backup-first procedure
+- verification checklist
+- rollback/mitigation decision tree
+- error classification guidance aligned to `docs/error_handling.md`
 
 ### 5) Backfill implementation or explicit no-backfill decision
 
-_TBD_
+No backfill required for this baseline Step 4 scope.
 
+Rationale:
+
+- additive migration operations only
+- default values and new-table creation do not require historical row rewrites for current V1 invariants
+- residual advanced backfill scenarios deferred unless future schema evolution introduces incompatible transforms
 ---
 
 ## Test and Verification Evidence
 
 ### Added/Updated Tests
 
-1. _TBD_
+1. `tests/test_migrations.py`
-2. _TBD_
+   - pending migration discovery
-3. _TBD_
+   - migration apply + history recording
-
+   - idempotent re-apply behavior
+2. `tests/test_db.py`
+   - compatibility-check behavior on fresh schema
+   - table expectation updates for `transcriptrevision`
+3. `tests/test_config.py`
+   - migration safety setting defaults
+4. `tests/test_app.py`
+   - lifespan test compatibility with migration/validation startup hooks
 ### Validation Runs
 
 Run and record outcomes:
 
-- `uv run pytest --collect-only -q` -> _TBD_
+- `uv run pytest --collect-only -q` -> passed
-- `uv run pytest -m unit -q` -> _TBD_
+- `uv run pytest -m unit -q` -> passed
-- `uv run pytest -m "not external" -q` -> _TBD_
+- `uv run pytest -m "not external" -q` -> passed
-- `uv run pytest -q` -> _TBD_
+- `uv run pytest -q` -> passed
-
 ### Migration Rehearsal Evidence
 
-Record migration rehearsal details:
+Migration rehearsal details (test-based):
-
-- baseline data set used: _TBD_
-- forward migration result: _TBD_
-- post-migration verification result: _TBD_
-- rollback/mitigation rehearsal result: _TBD_
 
+- baseline data set used: in-memory SQLite legacy-shaped schema fixture (`job` table missing Step 4 additions)
+- forward migration result: pending revisions applied successfully (`0001`, `0002`)
+- post-migration verification result: schema checks pass and migration history recorded
+- rollback/mitigation rehearsal result: runbook defined backup-restore primary rollback class for personal-scale SQLite deployment
 ---
 
 ## Requirement Traceability (Step 4)
 
 | Step 4 Area | REQ Coverage | Status | Evidence |
 | --- | --- | --- | --- |
-| Schema lifecycle and state persistence safety | REQ-3, REQ-4, REQ-11 | _TBD_ | _TBD_ |
+| Schema lifecycle and state persistence safety | REQ-3, REQ-4, REQ-11 | met | `src/transcription/migrations.py`, `tests/test_migrations.py`, `tests/test_db.py` |
-| Lifespan/runtime ownership continuity | REQ-7 | _TBD_ | _TBD_ |
+| Lifespan/runtime ownership continuity | REQ-7 | met | `src/transcription/app.py` startup checks + existing lifespan ownership model |
-| Explicit non-mutating production startup policy | REQ-10 | _TBD_ | _TBD_ |
+| Explicit non-mutating production startup policy | REQ-10 | met | `migration_auto_apply_on_startup=False` default + explicit runner workflow + startup validation gate |
-| Prompt/data continuity constraints | REQ-12 | _TBD_ | _TBD_ |
+| Prompt/data continuity constraints | REQ-12 | met (continued) | no prompt-contract mutation in Step 4 changes |
-
 ---
 
 ## Operational Artifacts Produced
 
 - `docs/ver1/ver1-step4.md`
-- `docs/ver1/ver1-step4-migration-runbook.md` (_if created_)
+- `docs/ver1/ver1-step4-migration-runbook.md`
-- _TBD additional artifacts_
+- `src/transcription/migrations.py`
-
+- `src/transcription/migration_runner.py`
+- README migration workflow updates
 ---
 
 ## Risks, Exceptions, and Follow-Ups
 
-1. _TBD_
+1. This lightweight migration system is appropriate for current personal-scale scope but may require a dedicated framework as schema complexity grows.
-2. _TBD_
+2. Rollback remains backup-restore primary; reversible down-migration coverage is intentionally limited in this baseline.
-3. _TBD_
+3. Startup compatibility checks currently fail fast with generic runtime error text and can be further normalized under API/operator error envelopes in later hardening.
 
 Open follow-ups to carry forward:
 
-- _TBD_
+- Evaluate migration framework escalation criteria in Step 9/10 readiness updates.
-
+- Add optional richer structured migration logging fields if observability scope expands.
 ---
 
 ## Step 4 Exit Assessment
 
-- Schema validation against finalized V1 domain: **_TBD_**
+- Schema validation against finalized V1 domain: **met**
-- Forward migration path safety and repeatability: **_TBD_**
+- Forward migration path safety and repeatability: **met (baseline scope)**
-- Rollback/mitigation readiness: **_TBD_**
+- Rollback/mitigation readiness: **met (backup-restore primary path)**
-- Backfill risk closure: **_TBD_**
+- Backfill risk closure: **met (no backfill required for current deltas)**
-- Test and regression safety: **_TBD_**
+- Test and regression safety: **met**
-
-Step 4 completion status: **_TBD_**
 
+Step 4 completion status: **complete (baseline scope)**
 ---
 
 ## Handoff to Step 5

src/transcription/app.py

@@ -16,7 +16,9 @@ from transcription.db import (
     dispose_database_runtime,
     initialize_database_runtime,
     should_bootstrap_schema,
+    validate_schema_compatibility,
 )
+from transcription.migrations import apply_pending_migrations
 from transcription.ui import register_pages
 from transcription.worker import run_worker_loop
 
@@ -58,6 +60,15 @@ async def _lifespan(app: FastAPI):
     if should_bootstrap_schema(settings):
         create_all(engine=app.state.db_runtime.engine)
 
+    if settings.migration_auto_apply_on_startup:
+        apply_pending_migrations(engine=app.state.db_runtime.engine)
+
+    if settings.validate_schema_on_startup:
+        compatibility_issues = validate_schema_compatibility(engine=app.state.db_runtime.engine)
+        if compatibility_issues:
+            issues_text = ", ".join(compatibility_issues)
+            raise RuntimeError(f"Schema compatibility check failed: {issues_text}")
+
     settings.upload_dir.mkdir(parents=True, exist_ok=True)
     settings.prompt_dir.mkdir(parents=True, exist_ok=True)
 

src/transcription/config.py

@@ -39,6 +39,8 @@ class Settings(BaseSettings):
     # --- persistence ---
     database_url: str = "sqlite:///./transcription.db"
     bootstrap_schema_on_startup: bool | None = None
+    migration_auto_apply_on_startup: bool = False
+    validate_schema_on_startup: bool = True
 
     # --- filesystem paths ---
     upload_dir: Path = Path("./uploads")

src/transcription/db.py

@@ -5,17 +5,15 @@ startup/shutdown behavior is predictable and lifespan-managed.
 """
 
 import contextlib
-import logging
 from collections.abc import Generator
 from dataclasses import dataclass
 
-from sqlalchemy import inspect, text
+from sqlalchemy import inspect
 from sqlalchemy.engine import Engine
 from sqlmodel import Session, SQLModel, create_engine
 
 from transcription.config import Settings, get_settings
 
-logger = logging.getLogger(__name__)
 
 
 @dataclass(frozen=True)
@@ -80,32 +78,30 @@ def create_all(*, engine: Engine | None = None) -> None:
 
     active_engine = engine or get_database_runtime().engine
     SQLModel.metadata.create_all(active_engine)
-    _ensure_sqlite_compat_columns(active_engine)
 
 
-def _ensure_sqlite_compat_columns(engine: Engine) -> None:
+def validate_schema_compatibility(*, engine: Engine | None = None) -> list[str]:
-    """Apply lightweight dev/test SQLite compatibility column patches.
+    """Return schema compatibility issues for known V1 requirements.
 
-    This keeps local bootstrap resilient when models evolve but no full
+    This performs read-only validation and never mutates schema.
-    migration tooling is in place yet.
     """
-    if engine.url.get_backend_name() != "sqlite":
+    active_engine = engine or get_database_runtime().engine
-        return
+    inspector = inspect(active_engine)
 
-    inspector = inspect(engine)
+    issues: list[str] = []
     table_names = set(inspector.get_table_names())
-    if "job" not in table_names:
-        return
 
-    columns = {column["name"] for column in inspector.get_columns("job")}
+    required_tables = {"document", "job", "transcript", "transcriptrevision"}
-    if "retry_count" not in columns:
+    missing_tables = sorted(required_tables - table_names)
-        with engine.begin() as connection:
+    for table_name in missing_tables:
-            connection.execute(
+        issues.append(f"missing_table:{table_name}")
-                text("ALTER TABLE job ADD COLUMN retry_count INTEGER NOT NULL DEFAULT 0")
+
-            )
+    if "job" in table_names:
-        logger.warning(
+        job_columns = {column["name"] for column in inspector.get_columns("job")}
-            "Applied SQLite compatibility schema patch table=job column=retry_count default=0"
+        if "retry_count" not in job_columns:
-        )
+            issues.append("missing_column:job.retry_count")
+
+    return issues
 
 
 @contextlib.contextmanager

src/transcription/migration_runner.py

@@ -0,0 +1,74 @@
+"""CLI entrypoint for explicit schema migration and compatibility checks."""
+
+from __future__ import annotations
+
+import argparse
+import sys
+
+from transcription.config import get_settings
+from transcription.db import initialize_database_runtime, validate_schema_compatibility
+from transcription.migrations import apply_pending_migrations, list_pending_migrations
+
+
+def _build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(description="Transcription schema migration runner")
+    parser.add_argument(
+        "--apply",
+        action="store_true",
+        help="Apply all pending migrations.",
+    )
+    parser.add_argument(
+        "--list",
+        action="store_true",
+        help="List pending migrations.",
+    )
+    parser.add_argument(
+        "--check",
+        action="store_true",
+        help="Run schema compatibility check.",
+    )
+    return parser
+
+
+def main() -> int:
+    parser = _build_parser()
+    args = parser.parse_args()
+
+    if not (args.apply or args.list or args.check):
+        parser.error("Specify at least one action: --list, --apply, or --check")
+
+    runtime = initialize_database_runtime(settings=get_settings())
+    engine = runtime.engine
+
+    if args.list:
+        pending = list_pending_migrations(engine=engine)
+        if not pending:
+            print("No pending migrations.")
+        else:
+            print("Pending migrations:")
+            for migration in pending:
+                print(f"- {migration.revision_id}: {migration.description}")
+
+    if args.apply:
+        applied = apply_pending_migrations(engine=engine)
+        if not applied:
+            print("No migrations applied.")
+        else:
+            print("Applied migrations:")
+            for revision_id in applied:
+                print(f"- {revision_id}")
+
+    if args.check:
+        issues = validate_schema_compatibility(engine=engine)
+        if issues:
+            print("Schema compatibility check failed:")
+            for issue in issues:
+                print(f"- {issue}")
+            return 1
+        print("Schema compatibility check passed.")
+
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

src/transcription/migrations.py

@@ -0,0 +1,128 @@
+"""Lightweight schema migration helpers for V1 Step 4.
+
+This module provides explicit, operator-invoked migration execution for
+personal-scale deployments without introducing heavyweight migration tooling.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from datetime import datetime, timezone
+import logging
+from typing import Callable
+
+from sqlalchemy import inspect, text
+from sqlalchemy.engine import Connection, Engine
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass(frozen=True)
+class MigrationRevision:
+    """Represents one ordered schema migration revision."""
+
+    revision_id: str
+    description: str
+    apply: Callable[[Connection], None]
+
+
+def _ensure_history_table(connection: Connection) -> None:
+    """Create migration history table when missing."""
+    connection.execute(
+        text(
+            """
+            CREATE TABLE IF NOT EXISTS schema_migration_history (
+                revision_id VARCHAR(64) PRIMARY KEY,
+                description VARCHAR(255) NOT NULL,
+                applied_at VARCHAR(64) NOT NULL
+            )
+            """
+        )
+    )
+
+
+def _get_applied_revisions(connection: Connection) -> set[str]:
+    """Return applied migration revision IDs."""
+    _ensure_history_table(connection)
+    rows = connection.execute(text("SELECT revision_id FROM schema_migration_history")).fetchall()
+    return {row[0] for row in rows}
+
+
+def _record_revision(connection: Connection, revision: MigrationRevision) -> None:
+    """Persist one applied migration revision record."""
+    connection.execute(
+        text(
+            """
+            INSERT INTO schema_migration_history (revision_id, description, applied_at)
+            VALUES (:revision_id, :description, :applied_at)
+            """
+        ),
+        {
+            "revision_id": revision.revision_id,
+            "description": revision.description,
+            "applied_at": datetime.now(timezone.utc).isoformat(),
+        },
+    )
+
+
+def _apply_0001_add_retry_count(connection: Connection) -> None:
+    """Ensure job.retry_count exists for legacy databases."""
+    inspector = inspect(connection)
+    table_names = set(inspector.get_table_names())
+    if "job" not in table_names:
+        return
+
+    columns = {column["name"] for column in inspector.get_columns("job")}
+    if "retry_count" in columns:
+        return
+
+    # Compatible with SQLite and PostgreSQL for this additive integer column.
+    connection.execute(text("ALTER TABLE job ADD COLUMN retry_count INTEGER NOT NULL DEFAULT 0"))
+
+
+def _apply_0002_create_transcriptrevision(connection: Connection) -> None:
+    """Ensure transcriptrevision table exists."""
+    # Import models lazily so metadata is fully populated.
+    from transcription.models import TranscriptRevision  # noqa: F401
+    from sqlmodel import SQLModel
+
+    table = SQLModel.metadata.tables["transcriptrevision"]
+    table.create(bind=connection, checkfirst=True)
+
+
+MIGRATIONS: tuple[MigrationRevision, ...] = (
+    MigrationRevision(
+        revision_id="0001_add_retry_count_to_job",
+        description="Add retry_count column to job table with default 0",
+        apply=_apply_0001_add_retry_count,
+    ),
+    MigrationRevision(
+        revision_id="0002_create_transcriptrevision_table",
+        description="Create transcriptrevision table for immutable transcript history",
+        apply=_apply_0002_create_transcriptrevision,
+    ),
+)
+
+
+def list_pending_migrations(*, engine: Engine) -> list[MigrationRevision]:
+    """Return pending migrations ordered by revision."""
+    with engine.begin() as connection:
+        applied = _get_applied_revisions(connection)
+    return [revision for revision in MIGRATIONS if revision.revision_id not in applied]
+
+
+def apply_pending_migrations(*, engine: Engine) -> list[str]:
+    """Apply all pending migrations and return applied revision IDs."""
+    pending = list_pending_migrations(engine=engine)
+    applied_ids: list[str] = []
+
+    for revision in pending:
+        logger.info("Applying migration revision=%s", revision.revision_id)
+        with engine.begin() as connection:
+            _ensure_history_table(connection)
+            revision.apply(connection)
+            _record_revision(connection, revision)
+        applied_ids.append(revision.revision_id)
+        logger.info("Applied migration revision=%s", revision.revision_id)
+
+    return applied_ids

tests/artifacts/transcriptions/Book_Two_-_page_02.jpg.txt

@@ -32,7 +32,7 @@ enough pictures but we had to take only part of them. We think there are great
 possibilities in reproducing old pictures. We wish we had a Pickard group. Some
 Pickard descendant may wish to make a collection.
 
-We are much impressed with the future possibilities of getting a complete geneal-
+We are much impressed with the future possibilities of getting a complete geneol-[sic]
 ogy of the Pickard family. Mr. Cochran has a fine chapter on the Pickards but
 to date we have not had the pleasure of finding all of the family dates. We had
 intended to give more family data in this book but it takes time to get the

tests/artifacts/transcriptions/Omie_Writes_Home.pdf.txt

@@ -6,9 +6,7 @@ JOHN E. COCHRAN
 FAMILY ASSOCIATION
 Family Only
 Home | Sibling's Stories | 1st Cousins | Ancestor's Stories | Reunion History | Next Reunion | JECMEF
-
 OMIE WRITES HOME
-
 Ed. Note: The following letter was written by Omie Cochran in Nome, Alaska and sent to her
 sister, Ethel Shinn, in Canfield, Idaho in 1923. It has been stored away these 63 years in the
 original envelope with its 2 cent stamp. The letter has a number of references to the Shinn
@@ -17,7 +15,9 @@ Miss Saville was the nurse at the Nome Hospital that was mentioned in the articl
 the family newsletter two years ago.
 
 Nome Alaska August 26, 1923
+
 My Dear Ethel et al.
+
 I don't know when I did write or when you did
 but I am going to write now however and never
 the less. But I wish I could talk (I can yet but I
@@ -26,7 +26,7 @@ and Polly sit up and listen and that little black
 rascal of yours would fairly sparkle with
 listening. Can't I see him listening now to all the
 yarns we told last summer?
-[photo of people on ice with ship in background]
+[photo of people on ice with kayak and dog sled]
 You see, we-Miss Saville and I, took a trip north
 on the Buford and it was very interesting. We
 went north thru the Bering Strait into the Arctic and as far as the Ice Pack. There the captain
@@ -37,7 +37,6 @@ along the icebergs and the walrus were 'heisted' on board by the use of the cran
 tons of freight and the beasts were so huge that they made the pulleys just creak. They were
 over 12 feet long, as big around as three cows, had no feet but toenails on their flippers or
 flappers, no head but their body just suddenly ended with a hole for a mouth and big bristles
-
 all around it similar to a currycomb in coarseness; no ears but huge tusks of ivory. They are
 the most repulsive looking animals imaginable and tho I have always read about them I never
 expect such disagreeable looking creatures. They had a rough brown hairy skin and some of
@@ -80,7 +79,6 @@ us talk. Miss Saville talked quite a bit. Any how if you folks don't like this I
 all I had to write about and I know Buster'ud listen anyway and I'd soak ole Peter's head if he
 didn't and Polly would in my lap and I don't know much about the youngest one of yours so
 likely he would be squawling. But we did surely enjoy our trip and were gone just long enuf.
-
 I expect there were 150 passengers on board and almost or more of the crew and helpers. We
 had a stateroom down next to the kitchen and 'twas pretty fierce for odor at times.
 

tests/artifacts/transcriptions/Rod_Moser_Letter_-_p1.jpg.txt

@@ -2,30 +2,28 @@ source: Rod Moser Letter - p1.jpg
 provider: openrouter
 model: google/gemini-2.5-flash
 ---
-JOHN ISBILL
+JOHN ISBILL                                                                       R. T. MOSER
-R. T. MOSER
-
 ISBILL & MOSER
 DEALERS IN
 GENERAL MERCHANDISE
 
-Vonore, Tenn. January 27 – 1913
+Vonore, Tenn. January 27 - 1913
-Dear Uncle Aunt Adeline
+Dear Uncle [sic] Aun[t Adeline?]
 Was at home a
 few nights ago & saw a
-letter from your folks, So
+letter from you folks, so
 I decided to write you
-a few lines myself ok
+a few lines myself &
-I am contemplate a
+I am contemplating a
-trip out west next summer
+trip out west next summ[er]
-& [inserted: I] would like of adders [sic] to go
+& I want both of fillers [sic] to go
-where I [inserted: am] them.
+when I am [to] them.
 Am getting
-up in years & unmarried
+up in years & unmarried,
 so you see the object of
-my trip is to get a wife
+my trip, is to get a wife
-& if there is any old maid
+& I hear is a lot old maids
-or widows out there, I
+& widows out there. I
-want you to kiss them
+want you to see them
-at my [inserted: mind] for me at there [sic]
+at my land my [sic] at there [sic]
-as soon as I get there.
+as soon as I get there

tests/test_app.py

@@ -35,6 +35,8 @@ class TestAppLifespan:
         monkeypatch.setattr("transcription.app.should_bootstrap_schema", lambda _settings: True)
         monkeypatch.setattr("transcription.app._start_worker", lambda _app: calls.append("start_worker"))
         monkeypatch.setattr("transcription.app._stop_worker", lambda _app: calls.append("stop_worker"))
+        monkeypatch.setattr("transcription.app.apply_pending_migrations", lambda **_kwargs: calls.append("migrate"))
+        monkeypatch.setattr("transcription.app.validate_schema_compatibility", lambda **_kwargs: [])
 
         class _Dir:
             def mkdir(self, parents: bool, exist_ok: bool):
@@ -43,6 +45,8 @@ class TestAppLifespan:
         class _Settings:
             upload_dir = _Dir()
             prompt_dir = _Dir()
+            migration_auto_apply_on_startup = False
+            validate_schema_on_startup = True
 
         monkeypatch.setattr("transcription.app.get_settings", lambda: _Settings())
 
@@ -70,6 +74,8 @@ class TestAppLifespan:
         monkeypatch.setattr("transcription.app.should_bootstrap_schema", lambda _settings: True)
         monkeypatch.setattr("transcription.app._start_worker", lambda _app: calls.append("start_worker"))
         monkeypatch.setattr("transcription.app._stop_worker", lambda _app: calls.append("stop_worker"))
+        monkeypatch.setattr("transcription.app.apply_pending_migrations", lambda **_kwargs: calls.append("migrate"))
+        monkeypatch.setattr("transcription.app.validate_schema_compatibility", lambda **_kwargs: [])
 
         class _Dir:
             def mkdir(self, parents: bool, exist_ok: bool):
@@ -78,6 +84,8 @@ class TestAppLifespan:
         class _Settings:
             upload_dir = _Dir()
             prompt_dir = _Dir()
+            migration_auto_apply_on_startup = False
+            validate_schema_on_startup = True
 
         monkeypatch.setattr("transcription.app.get_settings", lambda: _Settings())
 

tests/test_config.py

@@ -63,6 +63,16 @@ class TestPathSettings:
         assert isinstance(settings.prompt_dir, Path)
 
 
+class TestMigrationSafetySettings:
+    """Verify migration safety settings defaults."""
+
+    def test_migration_safety_defaults(self):
+        """Migration auto-apply is off and startup schema validation is on by default."""
+        settings = _make_settings()
+        assert settings.migration_auto_apply_on_startup is False
+        assert settings.validate_schema_on_startup is True
+
+
 class TestWorkerReliabilitySettings:
     """Verify worker retry settings defaults."""
 

tests/test_db.py

@@ -18,10 +18,10 @@ class TestSchemaBootstrap:
     """Verify create_all produces the expected table set."""
 
     def test_create_all_creates_expected_tables(self):
-        """After create_all(), document, job, and transcript tables exist."""
+        """After create_all(), core V1 tables exist."""
         engine = _in_memory_engine()
         # Ensure models are imported so metadata is populated
-        from transcription.models import Document, Job, Transcript  # noqa: F401
+        from transcription.models import Document, Job, Transcript, TranscriptRevision  # noqa: F401
 
         import transcription.db as db_module
 
@@ -32,6 +32,19 @@ class TestSchemaBootstrap:
         assert "document" in table_names
         assert "job" in table_names
         assert "transcript" in table_names
+        assert "transcriptrevision" in table_names
+
+
+    def test_validate_schema_compatibility_returns_no_issues_for_fresh_schema(self):
+        """validate_schema_compatibility reports no issues on fresh schema."""
+        engine = _in_memory_engine()
+
+        import transcription.db as db_module
+
+        db_module.create_all(engine=engine)
+        issues = db_module.validate_schema_compatibility(engine=engine)
+
+        assert issues == []
 
 
 class TestSessionFactory:

tests/test_migrations.py

@@ -0,0 +1,106 @@
+"""Tests for transcription.migrations — explicit Step 4 migration safety behavior."""
+
+from sqlalchemy import inspect, text
+from sqlmodel import create_engine
+from sqlmodel.pool import StaticPool
+
+from transcription.migrations import apply_pending_migrations, list_pending_migrations
+
+
+def _in_memory_engine():
+    """Create isolated in-memory SQLite engine."""
+    return create_engine(
+        "sqlite://",
+        connect_args={"check_same_thread": False},
+        poolclass=StaticPool,
+    )
+
+
+class TestMigrations:
+    """Verify migration listing and application behavior."""
+
+    def test_list_pending_returns_all_before_apply(self):
+        """All known migrations are pending on a fresh legacy-shaped database."""
+        engine = _in_memory_engine()
+
+        with engine.begin() as connection:
+            connection.execute(
+                text(
+                    """
+                    CREATE TABLE job (
+                        id TEXT PRIMARY KEY,
+                        document_id TEXT NOT NULL,
+                        status TEXT NOT NULL,
+                        created_at TEXT NOT NULL,
+                        updated_at TEXT NOT NULL
+                    )
+                    """
+                )
+            )
+
+        pending = list_pending_migrations(engine=engine)
+        assert [migration.revision_id for migration in pending] == [
+            "0001_add_retry_count_to_job",
+            "0002_create_transcriptrevision_table",
+        ]
+
+    def test_apply_pending_migrations_records_history_and_schema(self):
+        """Applying pending migrations mutates schema and records revision history."""
+        engine = _in_memory_engine()
+
+        with engine.begin() as connection:
+            connection.execute(
+                text(
+                    """
+                    CREATE TABLE job (
+                        id TEXT PRIMARY KEY,
+                        document_id TEXT NOT NULL,
+                        status TEXT NOT NULL,
+                        created_at TEXT NOT NULL,
+                        updated_at TEXT NOT NULL
+                    )
+                    """
+                )
+            )
+
+        applied = apply_pending_migrations(engine=engine)
+        assert applied == [
+            "0001_add_retry_count_to_job",
+            "0002_create_transcriptrevision_table",
+        ]
+
+        inspector = inspect(engine)
+        job_columns = {column["name"] for column in inspector.get_columns("job")}
+        assert "retry_count" in job_columns
+        assert "transcriptrevision" in set(inspector.get_table_names())
+
+        with engine.begin() as connection:
+            rows = connection.execute(
+                text("SELECT revision_id FROM schema_migration_history ORDER BY revision_id")
+            ).fetchall()
+        assert [row[0] for row in rows] == applied
+
+    def test_apply_pending_migrations_is_idempotent(self):
+        """Re-running apply_pending_migrations with no pending revisions is a no-op."""
+        engine = _in_memory_engine()
+
+        with engine.begin() as connection:
+            connection.execute(
+                text(
+                    """
+                    CREATE TABLE job (
+                        id TEXT PRIMARY KEY,
+                        document_id TEXT NOT NULL,
+                        status TEXT NOT NULL,
+                        created_at TEXT NOT NULL,
+                        updated_at TEXT NOT NULL
+                    )
+                    """
+                )
+            )
+
+        first_apply = apply_pending_migrations(engine=engine)
+        second_apply = apply_pending_migrations(engine=engine)
+
+        assert len(first_apply) == 2
+        assert second_apply == []