bbchops/transcription
1
0
Fork 0
generated from john/python-template
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ver1 - Step 4 implemented - Revision functionality added

Browse Source
This commit is contained in:
Jim Lancaster
2026-06-26 14:59:16 -05:00
parent 754a273ead
commit dd80cd60cf
16 changed files with 607 additions and 94 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+22 -1
View File
@@ -42,11 +42,32 @@ PROMPT_DIR=./prompts
uv run uvicorn transcription.app:create_app --factory --reload
```
### 4) Open in browser
### 4) (Optional) Run explicit migrations/checks
Use the migration runner for Step 4 schema safety workflows:
```bash
uv run python -m transcription.migration_runner --list
uv run python -m transcription.migration_runner --apply
uv run python -m transcription.migration_runner --check
```
### 5) Open in browser
- GUI: [http://[IP_ADDRESS]:8000/ui](http://[IP_ADDRESS]:8000/ui)
- Health check: [http://[IP_ADDRESS]:8000/healthz](http://[IP_ADDRESS]:8000/healthz)
### Schema safety settings
Optional environment settings (defaults shown):
```env
MIGRATION_AUTO_APPLY_ON_STARTUP=false
VALIDATE_SCHEMA_ON_STARTUP=true
```
## How to navigate the GUI
- **Upload page** (`/ui`)
docs/ver1/ver1-step4-migration-runbook.md
+113
View File
@@ -0,0 +1,113 @@
# Ver1 Step 4 Migration and Rollback Runbook
## Purpose
Provide a concise, operator-safe procedure for schema migration execution,
compatibility validation, and rollback/mitigation for personal-scale deployments.
This runbook supports `docs/ver1/ver1-step4.md` and REQ-10 by keeping normal
production startup non-mutating unless explicitly configured otherwise.
---
## Preconditions
1. Application version to deploy is known and checked out.
2. `.env` values are configured for target environment.
3. Database backup path is prepared.
4. Application process is stopped before migration on production-like systems.
---
## Commands
Use explicit migration runner operations:
1. List pending migrations:
- `uv run python -m transcription.migration_runner --list`
2. Apply pending migrations:
- `uv run python -m transcription.migration_runner --apply`
3. Validate schema compatibility:
- `uv run python -m transcription.migration_runner --check`
Recommended execution order:
1. `--list`
2. backup database
3. `--apply`
4. `--check`
5. start application
---
## Backup Procedure (SQLite Baseline)
For SQLite deployments, copy the DB file before migration:
- Example DB path default: `./transcription.db`
- Keep timestamped backup copy in a safe location.
If the file is in active use, stop the app first.
---
## Verification Checklist
After migration apply:
1. `--check` exits successfully.
2. `schema_migration_history` includes applied revisions.
3. Application starts successfully.
4. Health endpoint responds: `/healthz`.
5. Critical flows smoke-check:
- upload
- job processing
- revision listing/acceptance
---
## Rollback and Mitigation Decision Tree
1. If migration fails before changes commit:
- fix issue
- re-run apply
2. If migration partially applied or compatibility check fails:
- stop app
- restore from backup
- investigate and produce forward-fix migration if needed
3. If app starts but functional invariants fail:
- stop app
- restore backup
- add corrective migration/backfill and rehearse before retry
For this Step 4 baseline, backup restore is the primary rollback mechanism.
---
## Failure Classification Guidance
Classify migration failures using `docs/error_handling.md` categories:
- transient connection issues -> `infrastructure_transient_error`
- permissions/misconfiguration -> `infrastructure_persistent_error`
- unexpected migration logic defects -> `internal_unexpected_error`
Record failure details with operation context and timestamp.
---
## Operational Notes
- `migration_auto_apply_on_startup` defaults to `False`.
- `validate_schema_on_startup` defaults to `True`.
- Startup schema validation fails fast on incompatibility.
This protects production from accidental schema drift.
---
## Post-Step-4 Follow-Up
If migration complexity grows beyond lightweight revision scripts,
introduce a dedicated migration framework in a future step while preserving
this runbook structure and operator-first workflow.
docs/ver1/ver1-step4-results.md
+80 -45
View File
@@ -2,111 +2,146 @@
## Summary
Step 4 implementation status: **in progress**.
Step 4 implementation status: **complete (baseline scope)**.
This document records completed migration-safety work, validation evidence, and remaining follow-ups for Ver1 Step 4.
Implemented in this step:
1. _TBD_
2. _TBD_
3. _TBD_
1. Added explicit migration framework module with revision history tracking.
2. Added schema compatibility validation and startup guardrails.
3. Added migration runner CLI for list/apply/check operations.
4. Added migration tests and Step 4 validation evidence.
5. Added Step 4 migration/rollback runbook.
---
## Implemented Changes
### 1) Schema audit and invariant lock
_TBD_
Implemented read-only compatibility checks in `src/transcription/db.py`:
- `validate_schema_compatibility(...)` verifies required V1 tables:
- `document`
- `job`
- `transcript`
- `transcriptrevision`
- verifies required `job.retry_count` column
- returns explicit issue identifiers (non-mutating check)
### 2) Migration policy/tooling lock
_TBD_
Added explicit migration revision model in `src/transcription/migrations.py`:
- `MigrationRevision` dataclass
- ordered `MIGRATIONS` registry
- migration history table: `schema_migration_history`
- explicit pending-list and apply operations
### 3) Forward migration implementation
_TBD_
Implemented two baseline forward migrations:
1. `0001_add_retry_count_to_job`
2. `0002_create_transcriptrevision_table`
Each migration is idempotent and recorded in migration history.
### 4) Rollback and mitigation runbook
_TBD_
Created `docs/ver1/ver1-step4-migration-runbook.md` with:
- preconditions
- list/apply/check command sequence
- backup-first procedure
- verification checklist
- rollback/mitigation decision tree
- error classification guidance aligned to `docs/error_handling.md`
### 5) Backfill implementation or explicit no-backfill decision
_TBD_
No backfill required for this baseline Step 4 scope.
Rationale:
- additive migration operations only
- default values and new-table creation do not require historical row rewrites for current V1 invariants
- residual advanced backfill scenarios deferred unless future schema evolution introduces incompatible transforms
---
## Test and Verification Evidence
### Added/Updated Tests
1. _TBD_
2. _TBD_
3. _TBD_
1. `tests/test_migrations.py`
- pending migration discovery
- migration apply + history recording
- idempotent re-apply behavior
2. `tests/test_db.py`
- compatibility-check behavior on fresh schema
- table expectation updates for `transcriptrevision`
3. `tests/test_config.py`
- migration safety setting defaults
4. `tests/test_app.py`
- lifespan test compatibility with migration/validation startup hooks
### Validation Runs
Run and record outcomes:
- `uv run pytest --collect-only -q` -> _TBD_
- `uv run pytest -m unit -q` -> _TBD_
- `uv run pytest -m "not external" -q` -> _TBD_
- `uv run pytest -q` -> _TBD_
- `uv run pytest --collect-only -q` -> passed
- `uv run pytest -m unit -q` -> passed
- `uv run pytest -m "not external" -q` -> passed
- `uv run pytest -q` -> passed
### Migration Rehearsal Evidence
Record migration rehearsal details:
- baseline data set used: _TBD_
- forward migration result: _TBD_
- post-migration verification result: _TBD_
- rollback/mitigation rehearsal result: _TBD_
Migration rehearsal details (test-based):
- baseline data set used: in-memory SQLite legacy-shaped schema fixture (`job` table missing Step 4 additions)
- forward migration result: pending revisions applied successfully (`0001`, `0002`)
- post-migration verification result: schema checks pass and migration history recorded
- rollback/mitigation rehearsal result: runbook defined backup-restore primary rollback class for personal-scale SQLite deployment
---
## Requirement Traceability (Step 4)
| Step 4 Area | REQ Coverage | Status | Evidence |
| --- | --- | --- | --- |
| Schema lifecycle and state persistence safety | REQ-3, REQ-4, REQ-11 | _TBD_ | _TBD_ |
| Lifespan/runtime ownership continuity | REQ-7 | _TBD_ | _TBD_ |
| Explicit non-mutating production startup policy | REQ-10 | _TBD_ | _TBD_ |
| Prompt/data continuity constraints | REQ-12 | _TBD_ | _TBD_ |
| Schema lifecycle and state persistence safety | REQ-3, REQ-4, REQ-11 | met | `src/transcription/migrations.py`, `tests/test_migrations.py`, `tests/test_db.py` |
| Lifespan/runtime ownership continuity | REQ-7 | met | `src/transcription/app.py` startup checks + existing lifespan ownership model |
| Explicit non-mutating production startup policy | REQ-10 | met | `migration_auto_apply_on_startup=False` default + explicit runner workflow + startup validation gate |
| Prompt/data continuity constraints | REQ-12 | met (continued) | no prompt-contract mutation in Step 4 changes |
---
## Operational Artifacts Produced
- `docs/ver1/ver1-step4.md`
- `docs/ver1/ver1-step4-migration-runbook.md` (_if created_)
- _TBD additional artifacts_
- `docs/ver1/ver1-step4-migration-runbook.md`
- `src/transcription/migrations.py`
- `src/transcription/migration_runner.py`
- README migration workflow updates
---
## Risks, Exceptions, and Follow-Ups
1. _TBD_
2. _TBD_
3. _TBD_
1. This lightweight migration system is appropriate for current personal-scale scope but may require a dedicated framework as schema complexity grows.
2. Rollback remains backup-restore primary; reversible down-migration coverage is intentionally limited in this baseline.
3. Startup compatibility checks currently fail fast with generic runtime error text and can be further normalized under API/operator error envelopes in later hardening.
Open follow-ups to carry forward:
- _TBD_
- Evaluate migration framework escalation criteria in Step 9/10 readiness updates.
- Add optional richer structured migration logging fields if observability scope expands.
---
## Step 4 Exit Assessment
- Schema validation against finalized V1 domain: **_TBD_**
- Forward migration path safety and repeatability: **_TBD_**
- Rollback/mitigation readiness: **_TBD_**
- Backfill risk closure: **_TBD_**
- Test and regression safety: **_TBD_**
Step 4 completion status: **_TBD_**
- Schema validation against finalized V1 domain: **met**
- Forward migration path safety and repeatability: **met (baseline scope)**
- Rollback/mitigation readiness: **met (backup-restore primary path)**
- Backfill risk closure: **met (no backfill required for current deltas)**
- Test and regression safety: **met**
Step 4 completion status: **complete (baseline scope)**
---
## Handoff to Step 5
src/transcription/app.py
+11
View File
@@ -16,7 +16,9 @@ from transcription.db import (
dispose_database_runtime,
initialize_database_runtime,
should_bootstrap_schema,
validate_schema_compatibility,
)
from transcription.migrations import apply_pending_migrations
from transcription.ui import register_pages
from transcription.worker import run_worker_loop
@@ -58,6 +60,15 @@ async def _lifespan(app: FastAPI):
if should_bootstrap_schema(settings):
create_all(engine=app.state.db_runtime.engine)
if settings.migration_auto_apply_on_startup:
apply_pending_migrations(engine=app.state.db_runtime.engine)
if settings.validate_schema_on_startup:
compatibility_issues = validate_schema_compatibility(engine=app.state.db_runtime.engine)
if compatibility_issues:
issues_text = ", ".join(compatibility_issues)
raise RuntimeError(f"Schema compatibility check failed: {issues_text}")
settings.upload_dir.mkdir(parents=True, exist_ok=True)
settings.prompt_dir.mkdir(parents=True, exist_ok=True)
src/transcription/config.py
+2
View File
@@ -39,6 +39,8 @@ class Settings(BaseSettings):
# --- persistence ---
database_url: str = "sqlite:///./transcription.db"
bootstrap_schema_on_startup: bool | None = None
migration_auto_apply_on_startup: bool = False
validate_schema_on_startup: bool = True
# --- filesystem paths ---
upload_dir: Path = Path("./uploads")
src/transcription/db.py
+18 -22
View File
@@ -5,17 +5,15 @@ startup/shutdown behavior is predictable and lifespan-managed.
"""
import contextlib
import logging
from collections.abc import Generator
from dataclasses import dataclass
from sqlalchemy import inspect, text
from sqlalchemy import inspect
from sqlalchemy.engine import Engine
from sqlmodel import Session, SQLModel, create_engine
from transcription.config import Settings, get_settings
logger = logging.getLogger(__name__)
@dataclass(frozen=True)
@@ -80,32 +78,30 @@ def create_all(*, engine: Engine | None = None) -> None:
active_engine = engine or get_database_runtime().engine
SQLModel.metadata.create_all(active_engine)
_ensure_sqlite_compat_columns(active_engine)
def _ensure_sqlite_compat_columns(engine: Engine) -> None:
"""Apply lightweight dev/test SQLite compatibility column patches.
def validate_schema_compatibility(*, engine: Engine | None = None) -> list[str]:
"""Return schema compatibility issues for known V1 requirements.
This keeps local bootstrap resilient when models evolve but no full
migration tooling is in place yet.
This performs read-only validation and never mutates schema.
"""
if engine.url.get_backend_name() != "sqlite":
return
active_engine = engine or get_database_runtime().engine
inspector = inspect(active_engine)
inspector = inspect(engine)
issues: list[str] = []
table_names = set(inspector.get_table_names())
if "job" not in table_names:
return
columns = {column["name"] for column in inspector.get_columns("job")}
if "retry_count" not in columns:
with engine.begin() as connection:
connection.execute(
text("ALTER TABLE job ADD COLUMN retry_count INTEGER NOT NULL DEFAULT 0")
)
logger.warning(
"Applied SQLite compatibility schema patch table=job column=retry_count default=0"
)
required_tables = {"document", "job", "transcript", "transcriptrevision"}
missing_tables = sorted(required_tables - table_names)
for table_name in missing_tables:
issues.append(f"missing_table:{table_name}")
if "job" in table_names:
job_columns = {column["name"] for column in inspector.get_columns("job")}
if "retry_count" not in job_columns:
issues.append("missing_column:job.retry_count")
return issues
@contextlib.contextmanager
src/transcription/migration_runner.py
+74
View File
@@ -0,0 +1,74 @@
"""CLI entrypoint for explicit schema migration and compatibility checks."""
from __future__ import annotations
import argparse
import sys
from transcription.config import get_settings
from transcription.db import initialize_database_runtime, validate_schema_compatibility
from transcription.migrations import apply_pending_migrations, list_pending_migrations
def _build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description="Transcription schema migration runner")
parser.add_argument(
"--apply",
action="store_true",
help="Apply all pending migrations.",
)
parser.add_argument(
"--list",
action="store_true",
help="List pending migrations.",
)
parser.add_argument(
"--check",
action="store_true",
help="Run schema compatibility check.",
)
return parser
def main() -> int:
parser = _build_parser()
args = parser.parse_args()
if not (args.apply or args.list or args.check):
parser.error("Specify at least one action: --list, --apply, or --check")
runtime = initialize_database_runtime(settings=get_settings())
engine = runtime.engine
if args.list:
pending = list_pending_migrations(engine=engine)
if not pending:
print("No pending migrations.")
else:
print("Pending migrations:")
for migration in pending:
print(f"- {migration.revision_id}: {migration.description}")
if args.apply:
applied = apply_pending_migrations(engine=engine)
if not applied:
print("No migrations applied.")
else:
print("Applied migrations:")
for revision_id in applied:
print(f"- {revision_id}")
if args.check:
issues = validate_schema_compatibility(engine=engine)
if issues:
print("Schema compatibility check failed:")
for issue in issues:
print(f"- {issue}")
return 1
print("Schema compatibility check passed.")
return 0
if __name__ == "__main__":
raise SystemExit(main())
src/transcription/migrations.py
+128
View File
@@ -0,0 +1,128 @@
"""Lightweight schema migration helpers for V1 Step 4.
This module provides explicit, operator-invoked migration execution for
personal-scale deployments without introducing heavyweight migration tooling.
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import logging
from typing import Callable
from sqlalchemy import inspect, text
from sqlalchemy.engine import Connection, Engine
logger = logging.getLogger(__name__)
@dataclass(frozen=True)
class MigrationRevision:
"""Represents one ordered schema migration revision."""
revision_id: str
description: str
apply: Callable[[Connection], None]
def _ensure_history_table(connection: Connection) -> None:
"""Create migration history table when missing."""
connection.execute(
text(
"""
CREATE TABLE IF NOT EXISTS schema_migration_history (
revision_id VARCHAR(64) PRIMARY KEY,
description VARCHAR(255) NOT NULL,
applied_at VARCHAR(64) NOT NULL
)
"""
)
)
def _get_applied_revisions(connection: Connection) -> set[str]:
"""Return applied migration revision IDs."""
_ensure_history_table(connection)
rows = connection.execute(text("SELECT revision_id FROM schema_migration_history")).fetchall()
return {row[0] for row in rows}
def _record_revision(connection: Connection, revision: MigrationRevision) -> None:
"""Persist one applied migration revision record."""
connection.execute(
text(
"""
INSERT INTO schema_migration_history (revision_id, description, applied_at)
VALUES (:revision_id, :description, :applied_at)
"""
),
{
"revision_id": revision.revision_id,
"description": revision.description,
"applied_at": datetime.now(timezone.utc).isoformat(),
},
)
def _apply_0001_add_retry_count(connection: Connection) -> None:
"""Ensure job.retry_count exists for legacy databases."""
inspector = inspect(connection)
table_names = set(inspector.get_table_names())
if "job" not in table_names:
return
columns = {column["name"] for column in inspector.get_columns("job")}
if "retry_count" in columns:
return
# Compatible with SQLite and PostgreSQL for this additive integer column.
connection.execute(text("ALTER TABLE job ADD COLUMN retry_count INTEGER NOT NULL DEFAULT 0"))
def _apply_0002_create_transcriptrevision(connection: Connection) -> None:
"""Ensure transcriptrevision table exists."""
# Import models lazily so metadata is fully populated.
from transcription.models import TranscriptRevision # noqa: F401
from sqlmodel import SQLModel
table = SQLModel.metadata.tables["transcriptrevision"]
table.create(bind=connection, checkfirst=True)
MIGRATIONS: tuple[MigrationRevision, ...] = (
MigrationRevision(
revision_id="0001_add_retry_count_to_job",
description="Add retry_count column to job table with default 0",
apply=_apply_0001_add_retry_count,
),
MigrationRevision(
revision_id="0002_create_transcriptrevision_table",
description="Create transcriptrevision table for immutable transcript history",
apply=_apply_0002_create_transcriptrevision,
),
)
def list_pending_migrations(*, engine: Engine) -> list[MigrationRevision]:
"""Return pending migrations ordered by revision."""
with engine.begin() as connection:
applied = _get_applied_revisions(connection)
return [revision for revision in MIGRATIONS if revision.revision_id not in applied]
def apply_pending_migrations(*, engine: Engine) -> list[str]:
"""Apply all pending migrations and return applied revision IDs."""
pending = list_pending_migrations(engine=engine)
applied_ids: list[str] = []
for revision in pending:
logger.info("Applying migration revision=%s", revision.revision_id)
with engine.begin() as connection:
_ensure_history_table(connection)
revision.apply(connection)
_record_revision(connection, revision)
applied_ids.append(revision.revision_id)
logger.info("Applied migration revision=%s", revision.revision_id)
return applied_ids
tests/artifacts/transcriptions/Book_Two_-_page_02.jpg.txt
+1 -1
View File
@@ -32,7 +32,7 @@ enough pictures but we had to take only part of them. We think there are great
possibilities in reproducing old pictures. We wish we had a Pickard group. Some
Pickard descendant may wish to make a collection.
We are much impressed with the future possibilities of getting a complete geneal-
We are much impressed with the future possibilities of getting a complete geneol-[sic]
ogy of the Pickard family. Mr. Cochran has a fine chapter on the Pickards but
to date we have not had the pleasure of finding all of the family dates. We had
intended to give more family data in this book but it takes time to get the
tests/artifacts/transcriptions/Omie_Writes_Home.pdf.txt
+3 -5
View File
@@ -6,9 +6,7 @@ JOHN E. COCHRAN
FAMILY ASSOCIATION
Family Only
Home | Sibling's Stories | 1st Cousins | Ancestor's Stories | Reunion History | Next Reunion | JECMEF
OMIE WRITES HOME
Ed. Note: The following letter was written by Omie Cochran in Nome, Alaska and sent to her
sister, Ethel Shinn, in Canfield, Idaho in 1923. It has been stored away these 63 years in the
original envelope with its 2 cent stamp. The letter has a number of references to the Shinn
@@ -17,7 +15,9 @@ Miss Saville was the nurse at the Nome Hospital that was mentioned in the articl
the family newsletter two years ago.
Nome Alaska August 26, 1923
My Dear Ethel et al.
I don't know when I did write or when you did
but I am going to write now however and never
the less. But I wish I could talk (I can yet but I
@@ -26,7 +26,7 @@ and Polly sit up and listen and that little black
rascal of yours would fairly sparkle with
listening. Can't I see him listening now to all the
yarns we told last summer?
[photo of people on ice with ship in background]
[photo of people on ice with kayak and dog sled]
You see, we-Miss Saville and I, took a trip north
on the Buford and it was very interesting. We
went north thru the Bering Strait into the Arctic and as far as the Ice Pack. There the captain
@@ -37,7 +37,6 @@ along the icebergs and the walrus were 'heisted' on board by the use of the cran
tons of freight and the beasts were so huge that they made the pulleys just creak. They were
over 12 feet long, as big around as three cows, had no feet but toenails on their flippers or
flappers, no head but their body just suddenly ended with a hole for a mouth and big bristles
all around it similar to a currycomb in coarseness; no ears but huge tusks of ivory. They are
the most repulsive looking animals imaginable and tho I have always read about them I never
expect such disagreeable looking creatures. They had a rough brown hairy skin and some of
@@ -80,7 +79,6 @@ us talk. Miss Saville talked quite a bit. Any how if you folks don't like this I
all I had to write about and I know Buster'ud listen anyway and I'd soak ole Peter's head if he
didn't and Polly would in my lap and I don't know much about the youngest one of yours so
likely he would be squawling. But we did surely enjoy our trip and were gone just long enuf.
I expect there were 150 passengers on board and almost or more of the crew and helpers. We
had a stateroom down next to the kitchen and 'twas pretty fierce for odor at times.
tests/artifacts/transcriptions/Rod_Moser_Letter_-_p1.jpg.txt
+16 -18
View File
@@ -2,30 +2,28 @@ source: Rod Moser Letter - p1.jpg
provider: openrouter
model: google/gemini-2.5-flash
---
JOHN ISBILL
R. T. MOSER
JOHN ISBILL R. T. MOSER
ISBILL & MOSER
DEALERS IN
GENERAL MERCHANDISE
Vonore, Tenn. January 27 – 1913
Dear Uncle Aunt Adeline
Vonore, Tenn. January 27 - 1913
Dear Uncle [sic] Aun[t Adeline?]
Was at home a
few nights ago & saw a
letter from your folks, So
letter from you folks, so
I decided to write you
a few lines myself ok
I am contemplate a
trip out west next summer
& [inserted: I] would like of adders [sic] to go
where I [inserted: am] them.
a few lines myself &
I am contemplating a
trip out west next summ[er]
& I want both of fillers [sic] to go
when I am [to] them.
Am getting
up in years & unmarried
up in years & unmarried,
so you see the object of
my trip is to get a wife
& if there is any old maid
or widows out there, I
want you to kiss them
at my [inserted: mind] for me at there [sic]
as soon as I get there.
my trip, is to get a wife
& I hear is a lot old maids
& widows out there. I
want you to see them
at my land my [sic] at there [sic]
as soon as I get there
tests/test_app.py
+8
View File
@@ -35,6 +35,8 @@ class TestAppLifespan:
monkeypatch.setattr("transcription.app.should_bootstrap_schema", lambda _settings: True)
monkeypatch.setattr("transcription.app._start_worker", lambda _app: calls.append("start_worker"))
monkeypatch.setattr("transcription.app._stop_worker", lambda _app: calls.append("stop_worker"))
monkeypatch.setattr("transcription.app.apply_pending_migrations", lambda **_kwargs: calls.append("migrate"))
monkeypatch.setattr("transcription.app.validate_schema_compatibility", lambda **_kwargs: [])
class _Dir:
def mkdir(self, parents: bool, exist_ok: bool):
@@ -43,6 +45,8 @@ class TestAppLifespan:
class _Settings:
upload_dir = _Dir()
prompt_dir = _Dir()
migration_auto_apply_on_startup = False
validate_schema_on_startup = True
monkeypatch.setattr("transcription.app.get_settings", lambda: _Settings())
@@ -70,6 +74,8 @@ class TestAppLifespan:
monkeypatch.setattr("transcription.app.should_bootstrap_schema", lambda _settings: True)
monkeypatch.setattr("transcription.app._start_worker", lambda _app: calls.append("start_worker"))
monkeypatch.setattr("transcription.app._stop_worker", lambda _app: calls.append("stop_worker"))
monkeypatch.setattr("transcription.app.apply_pending_migrations", lambda **_kwargs: calls.append("migrate"))
monkeypatch.setattr("transcription.app.validate_schema_compatibility", lambda **_kwargs: [])
class _Dir:
def mkdir(self, parents: bool, exist_ok: bool):
@@ -78,6 +84,8 @@ class TestAppLifespan:
class _Settings:
upload_dir = _Dir()
prompt_dir = _Dir()
migration_auto_apply_on_startup = False
validate_schema_on_startup = True
monkeypatch.setattr("transcription.app.get_settings", lambda: _Settings())
tests/test_config.py
+10
View File
@@ -63,6 +63,16 @@ class TestPathSettings:
assert isinstance(settings.prompt_dir, Path)
class TestMigrationSafetySettings:
"""Verify migration safety settings defaults."""
def test_migration_safety_defaults(self):
"""Migration auto-apply is off and startup schema validation is on by default."""
settings = _make_settings()
assert settings.migration_auto_apply_on_startup is False
assert settings.validate_schema_on_startup is True
class TestWorkerReliabilitySettings:
"""Verify worker retry settings defaults."""
tests/test_db.py
+15 -2
View File
@@ -18,10 +18,10 @@ class TestSchemaBootstrap:
"""Verify create_all produces the expected table set."""
def test_create_all_creates_expected_tables(self):
"""After create_all(), document, job, and transcript tables exist."""
"""After create_all(), core V1 tables exist."""
engine = _in_memory_engine()
# Ensure models are imported so metadata is populated
from transcription.models import Document, Job, Transcript # noqa: F401
from transcription.models import Document, Job, Transcript, TranscriptRevision # noqa: F401
import transcription.db as db_module
@@ -32,6 +32,19 @@ class TestSchemaBootstrap:
assert "document" in table_names
assert "job" in table_names
assert "transcript" in table_names
assert "transcriptrevision" in table_names
def test_validate_schema_compatibility_returns_no_issues_for_fresh_schema(self):
"""validate_schema_compatibility reports no issues on fresh schema."""
engine = _in_memory_engine()
import transcription.db as db_module
db_module.create_all(engine=engine)
issues = db_module.validate_schema_compatibility(engine=engine)
assert issues == []
class TestSessionFactory:
tests/test_migrations.py
+106
View File
@@ -0,0 +1,106 @@
"""Tests for transcription.migrations — explicit Step 4 migration safety behavior."""
from sqlalchemy import inspect, text
from sqlmodel import create_engine
from sqlmodel.pool import StaticPool
from transcription.migrations import apply_pending_migrations, list_pending_migrations
def _in_memory_engine():
"""Create isolated in-memory SQLite engine."""
return create_engine(
"sqlite://",
connect_args={"check_same_thread": False},
poolclass=StaticPool,
)
class TestMigrations:
"""Verify migration listing and application behavior."""
def test_list_pending_returns_all_before_apply(self):
"""All known migrations are pending on a fresh legacy-shaped database."""
engine = _in_memory_engine()
with engine.begin() as connection:
connection.execute(
text(
"""
CREATE TABLE job (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL,
status TEXT NOT NULL,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
)
"""
)
)
pending = list_pending_migrations(engine=engine)
assert [migration.revision_id for migration in pending] == [
"0001_add_retry_count_to_job",
"0002_create_transcriptrevision_table",
]
def test_apply_pending_migrations_records_history_and_schema(self):
"""Applying pending migrations mutates schema and records revision history."""
engine = _in_memory_engine()
with engine.begin() as connection:
connection.execute(
text(
"""
CREATE TABLE job (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL,
status TEXT NOT NULL,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
)
"""
)
)
applied = apply_pending_migrations(engine=engine)
assert applied == [
"0001_add_retry_count_to_job",
"0002_create_transcriptrevision_table",
]
inspector = inspect(engine)
job_columns = {column["name"] for column in inspector.get_columns("job")}
assert "retry_count" in job_columns
assert "transcriptrevision" in set(inspector.get_table_names())
with engine.begin() as connection:
rows = connection.execute(
text("SELECT revision_id FROM schema_migration_history ORDER BY revision_id")
).fetchall()
assert [row[0] for row in rows] == applied
def test_apply_pending_migrations_is_idempotent(self):
"""Re-running apply_pending_migrations with no pending revisions is a no-op."""
engine = _in_memory_engine()
with engine.begin() as connection:
connection.execute(
text(
"""
CREATE TABLE job (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL,
status TEXT NOT NULL,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
)
"""
)
)
first_apply = apply_pending_migrations(engine=engine)
second_apply = apply_pending_migrations(engine=engine)
assert len(first_apply) == 2
assert second_apply == []
uploads/f4322c0e-a1df-434e-88b2-acb88a09a021_Hig's postcards to Zenna - January 7, 1925.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 274 KiB