ver1 - Step 4 implemented - Revision functionality added

tests/artifacts/transcriptions/Book_Two_-_page_02.jpg.txt

@@ -32,7 +32,7 @@ enough pictures but we had to take only part of them. We think there are great
 possibilities in reproducing old pictures. We wish we had a Pickard group. Some
 Pickard descendant may wish to make a collection.
 
-We are much impressed with the future possibilities of getting a complete geneal-
+We are much impressed with the future possibilities of getting a complete geneol-[sic]
 ogy of the Pickard family. Mr. Cochran has a fine chapter on the Pickards but
 to date we have not had the pleasure of finding all of the family dates. We had
 intended to give more family data in this book but it takes time to get the

tests/artifacts/transcriptions/Omie_Writes_Home.pdf.txt

@@ -6,9 +6,7 @@ JOHN E. COCHRAN
 FAMILY ASSOCIATION
 Family Only
 Home | Sibling's Stories | 1st Cousins | Ancestor's Stories | Reunion History | Next Reunion | JECMEF
-
 OMIE WRITES HOME
-
 Ed. Note: The following letter was written by Omie Cochran in Nome, Alaska and sent to her
 sister, Ethel Shinn, in Canfield, Idaho in 1923. It has been stored away these 63 years in the
 original envelope with its 2 cent stamp. The letter has a number of references to the Shinn
@@ -17,7 +15,9 @@ Miss Saville was the nurse at the Nome Hospital that was mentioned in the articl
 the family newsletter two years ago.
 
 Nome Alaska August 26, 1923
+
 My Dear Ethel et al.
+
 I don't know when I did write or when you did
 but I am going to write now however and never
 the less. But I wish I could talk (I can yet but I
@@ -26,7 +26,7 @@ and Polly sit up and listen and that little black
 rascal of yours would fairly sparkle with
 listening. Can't I see him listening now to all the
 yarns we told last summer?
-[photo of people on ice with ship in background]
+[photo of people on ice with kayak and dog sled]
 You see, we-Miss Saville and I, took a trip north
 on the Buford and it was very interesting. We
 went north thru the Bering Strait into the Arctic and as far as the Ice Pack. There the captain
@@ -37,7 +37,6 @@ along the icebergs and the walrus were 'heisted' on board by the use of the cran
 tons of freight and the beasts were so huge that they made the pulleys just creak. They were
 over 12 feet long, as big around as three cows, had no feet but toenails on their flippers or
 flappers, no head but their body just suddenly ended with a hole for a mouth and big bristles
-
 all around it similar to a currycomb in coarseness; no ears but huge tusks of ivory. They are
 the most repulsive looking animals imaginable and tho I have always read about them I never
 expect such disagreeable looking creatures. They had a rough brown hairy skin and some of
@@ -80,7 +79,6 @@ us talk. Miss Saville talked quite a bit. Any how if you folks don't like this I
 all I had to write about and I know Buster'ud listen anyway and I'd soak ole Peter's head if he
 didn't and Polly would in my lap and I don't know much about the youngest one of yours so
 likely he would be squawling. But we did surely enjoy our trip and were gone just long enuf.
-
 I expect there were 150 passengers on board and almost or more of the crew and helpers. We
 had a stateroom down next to the kitchen and 'twas pretty fierce for odor at times.
 

tests/artifacts/transcriptions/Rod_Moser_Letter_-_p1.jpg.txt

@@ -2,30 +2,28 @@ source: Rod Moser Letter - p1.jpg
 provider: openrouter
 model: google/gemini-2.5-flash
 ---
-JOHN ISBILL
-R. T. MOSER
-
+JOHN ISBILL                                                                       R. T. MOSER
 ISBILL & MOSER
 DEALERS IN
 GENERAL MERCHANDISE
 
-Vonore, Tenn. January 27 – 1913
-Dear Uncle Aunt Adeline
+Vonore, Tenn. January 27 - 1913
+Dear Uncle [sic] Aun[t Adeline?]
 Was at home a
 few nights ago & saw a
-letter from your folks, So
+letter from you folks, so
 I decided to write you
-a few lines myself ok
-I am contemplate a
-trip out west next summer
-& [inserted: I] would like of adders [sic] to go
-where I [inserted: am] them.
+a few lines myself &
+I am contemplating a
+trip out west next summ[er]
+& I want both of fillers [sic] to go
+when I am [to] them.
 Am getting
-up in years & unmarried
+up in years & unmarried,
 so you see the object of
-my trip is to get a wife
-& if there is any old maid
-or widows out there, I
-want you to kiss them
-at my [inserted: mind] for me at there [sic]
-as soon as I get there.
+my trip, is to get a wife
+& I hear is a lot old maids
+& widows out there. I
+want you to see them
+at my land my [sic] at there [sic]
+as soon as I get there

tests/test_app.py

@@ -35,6 +35,8 @@ class TestAppLifespan:
         monkeypatch.setattr("transcription.app.should_bootstrap_schema", lambda _settings: True)
         monkeypatch.setattr("transcription.app._start_worker", lambda _app: calls.append("start_worker"))
         monkeypatch.setattr("transcription.app._stop_worker", lambda _app: calls.append("stop_worker"))
+        monkeypatch.setattr("transcription.app.apply_pending_migrations", lambda **_kwargs: calls.append("migrate"))
+        monkeypatch.setattr("transcription.app.validate_schema_compatibility", lambda **_kwargs: [])
 
         class _Dir:
             def mkdir(self, parents: bool, exist_ok: bool):
@@ -43,6 +45,8 @@ class TestAppLifespan:
         class _Settings:
             upload_dir = _Dir()
             prompt_dir = _Dir()
+            migration_auto_apply_on_startup = False
+            validate_schema_on_startup = True
 
         monkeypatch.setattr("transcription.app.get_settings", lambda: _Settings())
 
@@ -70,6 +74,8 @@ class TestAppLifespan:
         monkeypatch.setattr("transcription.app.should_bootstrap_schema", lambda _settings: True)
         monkeypatch.setattr("transcription.app._start_worker", lambda _app: calls.append("start_worker"))
         monkeypatch.setattr("transcription.app._stop_worker", lambda _app: calls.append("stop_worker"))
+        monkeypatch.setattr("transcription.app.apply_pending_migrations", lambda **_kwargs: calls.append("migrate"))
+        monkeypatch.setattr("transcription.app.validate_schema_compatibility", lambda **_kwargs: [])
 
         class _Dir:
             def mkdir(self, parents: bool, exist_ok: bool):
@@ -78,6 +84,8 @@ class TestAppLifespan:
         class _Settings:
             upload_dir = _Dir()
             prompt_dir = _Dir()
+            migration_auto_apply_on_startup = False
+            validate_schema_on_startup = True
 
         monkeypatch.setattr("transcription.app.get_settings", lambda: _Settings())
 

tests/test_config.py

@@ -63,6 +63,16 @@ class TestPathSettings:
         assert isinstance(settings.prompt_dir, Path)
 
 
+class TestMigrationSafetySettings:
+    """Verify migration safety settings defaults."""
+
+    def test_migration_safety_defaults(self):
+        """Migration auto-apply is off and startup schema validation is on by default."""
+        settings = _make_settings()
+        assert settings.migration_auto_apply_on_startup is False
+        assert settings.validate_schema_on_startup is True
+
+
 class TestWorkerReliabilitySettings:
     """Verify worker retry settings defaults."""
 

tests/test_db.py

@@ -18,10 +18,10 @@ class TestSchemaBootstrap:
     """Verify create_all produces the expected table set."""
 
     def test_create_all_creates_expected_tables(self):
-        """After create_all(), document, job, and transcript tables exist."""
+        """After create_all(), core V1 tables exist."""
         engine = _in_memory_engine()
         # Ensure models are imported so metadata is populated
-        from transcription.models import Document, Job, Transcript  # noqa: F401
+        from transcription.models import Document, Job, Transcript, TranscriptRevision  # noqa: F401
 
         import transcription.db as db_module
 
@@ -32,6 +32,19 @@ class TestSchemaBootstrap:
         assert "document" in table_names
         assert "job" in table_names
         assert "transcript" in table_names
+        assert "transcriptrevision" in table_names
+
+
+    def test_validate_schema_compatibility_returns_no_issues_for_fresh_schema(self):
+        """validate_schema_compatibility reports no issues on fresh schema."""
+        engine = _in_memory_engine()
+
+        import transcription.db as db_module
+
+        db_module.create_all(engine=engine)
+        issues = db_module.validate_schema_compatibility(engine=engine)
+
+        assert issues == []
 
 
 class TestSessionFactory:

tests/test_migrations.py

@@ -0,0 +1,106 @@
+"""Tests for transcription.migrations — explicit Step 4 migration safety behavior."""
+
+from sqlalchemy import inspect, text
+from sqlmodel import create_engine
+from sqlmodel.pool import StaticPool
+
+from transcription.migrations import apply_pending_migrations, list_pending_migrations
+
+
+def _in_memory_engine():
+    """Create isolated in-memory SQLite engine."""
+    return create_engine(
+        "sqlite://",
+        connect_args={"check_same_thread": False},
+        poolclass=StaticPool,
+    )
+
+
+class TestMigrations:
+    """Verify migration listing and application behavior."""
+
+    def test_list_pending_returns_all_before_apply(self):
+        """All known migrations are pending on a fresh legacy-shaped database."""
+        engine = _in_memory_engine()
+
+        with engine.begin() as connection:
+            connection.execute(
+                text(
+                    """
+                    CREATE TABLE job (
+                        id TEXT PRIMARY KEY,
+                        document_id TEXT NOT NULL,
+                        status TEXT NOT NULL,
+                        created_at TEXT NOT NULL,
+                        updated_at TEXT NOT NULL
+                    )
+                    """
+                )
+            )
+
+        pending = list_pending_migrations(engine=engine)
+        assert [migration.revision_id for migration in pending] == [
+            "0001_add_retry_count_to_job",
+            "0002_create_transcriptrevision_table",
+        ]
+
+    def test_apply_pending_migrations_records_history_and_schema(self):
+        """Applying pending migrations mutates schema and records revision history."""
+        engine = _in_memory_engine()
+
+        with engine.begin() as connection:
+            connection.execute(
+                text(
+                    """
+                    CREATE TABLE job (
+                        id TEXT PRIMARY KEY,
+                        document_id TEXT NOT NULL,
+                        status TEXT NOT NULL,
+                        created_at TEXT NOT NULL,
+                        updated_at TEXT NOT NULL
+                    )
+                    """
+                )
+            )
+
+        applied = apply_pending_migrations(engine=engine)
+        assert applied == [
+            "0001_add_retry_count_to_job",
+            "0002_create_transcriptrevision_table",
+        ]
+
+        inspector = inspect(engine)
+        job_columns = {column["name"] for column in inspector.get_columns("job")}
+        assert "retry_count" in job_columns
+        assert "transcriptrevision" in set(inspector.get_table_names())
+
+        with engine.begin() as connection:
+            rows = connection.execute(
+                text("SELECT revision_id FROM schema_migration_history ORDER BY revision_id")
+            ).fetchall()
+        assert [row[0] for row in rows] == applied
+
+    def test_apply_pending_migrations_is_idempotent(self):
+        """Re-running apply_pending_migrations with no pending revisions is a no-op."""
+        engine = _in_memory_engine()
+
+        with engine.begin() as connection:
+            connection.execute(
+                text(
+                    """
+                    CREATE TABLE job (
+                        id TEXT PRIMARY KEY,
+                        document_id TEXT NOT NULL,
+                        status TEXT NOT NULL,
+                        created_at TEXT NOT NULL,
+                        updated_at TEXT NOT NULL
+                    )
+                    """
+                )
+            )
+
+        first_apply = apply_pending_migrations(engine=engine)
+        second_apply = apply_pending_migrations(engine=engine)
+
+        assert len(first_apply) == 2
+        assert second_apply == []