V1 mostly complete except for some testing. Linting in the last step changed nearly every file which is why this commit is so larger.

2026-07-29 17:27:21 -05:00
parent bc21a97019
commit a3b3bab571
37 changed files with 1383 additions and 410 deletions
@@ -0,0 +1,92 @@
+# V1 Data Migration and Recovery Guidance
+
+This document defines migration/backfill and rollback guidance for the V1 SQLite baseline.
+
+## Purpose
+
+- provide safe procedures for local schema evolution and recovery
+- reduce data-loss risk during version upgrades
+- establish repeatable pre-change and post-change checks
+
+## Current Baseline
+
+- canonical relational store: SQLite
+- default DB path: `./transcription.db`
+- schema bootstrap may apply compatibility updates for dev/test scenarios
+
+## Pre-Change Checklist
+
+Before changing runtime version or schema behavior:
+
+1. Stop the app process.
+2. Create a timestamped DB backup copy.
+3. Capture current app commit/version.
+4. Export a quick status inventory:
+   - job counts by status
+   - total documents/sources/revisions
+5. Ensure sufficient disk space.
+
+## Backup Procedure (SQLite)
+
+Minimum procedure:
+
+1. Stop app.
+2. Copy DB file to a safe location with timestamp.
+3. Store backup path in release notes or change log.
+
+## Upgrade Procedure (V1)
+
+1. Perform pre-change checklist.
+2. Deploy updated app version.
+3. Start app and observe startup logs.
+4. Verify schema bootstrap completes (if enabled).
+5. Run smoke flow:
+   - upload valid file
+   - observe terminal status
+   - open job detail
+
+## Backfill Guidance
+
+V1 backfill is limited and conservative:
+
+- for records missing newly introduced non-null defaults, use explicit one-time SQL updates only after backup
+- avoid destructive rewrites of `Job.text` or `Revision.text`
+- never backfill by overwriting original immutable transcription output
+
+## Rollback Procedure
+
+If upgrade fails or causes data inconsistency:
+
+1. Stop app.
+2. Restore prior DB backup file.
+3. Revert app version to last known-good commit.
+4. Restart app.
+5. Run smoke flow and confirm stability.
+
+## Recovery Scenarios
+
+### Stale processing jobs after crash/restart
+
+- restart app and allow stale-job recovery to re-queue timed-out `processing` jobs
+- monitor for terminal progression
+
+### Schema mismatch symptoms
+
+- errors during startup or writes indicating missing columns/indexes
+- rollback to last good DB + app version
+- reattempt with documented upgrade path
+
+## Validation Evidence
+
+For each upgrade rehearsal, capture:
+
+- backup filename/path
+- pre and post job status counts
+- smoke test result
+- rollback rehearsal result (recommended)
+
+## Operational Constraints
+
+- treat DB backups as required before non-trivial upgrades
+- do not perform in-place DB edits while app is running
+- do not skip post-upgrade smoke validation
@@ -0,0 +1,45 @@
+# V1 Release Readiness Checklist
+
+Use this checklist before declaring V1 operationally complete.
+
+## A) Functional Readiness
+
+- [ ] Upload flow works for supported file types.
+- [ ] Worker transitions jobs through `queued -> processing -> transcribed|failed`.
+- [ ] Job detail displays immutable original transcription from `Job.text`.
+- [ ] Revision workflow supports create/update/view/delete for optional single revision.
+
+## B) Reliability and Error Handling
+
+- [ ] Error categories surface with actionable messages in UI/API pathways.
+- [ ] Failed jobs persist `error_detail` and terminal state.
+- [ ] Stale processing recovery verified on restart.
+- [ ] Retry/timeout behavior validated against configured limits.
+
+## C) Operational Readiness
+
+- [ ] `docs/runbook.md` reviewed and current.
+- [ ] `docs/migration_v1.md` reviewed and current.
+- [ ] Backup and rollback procedures tested at least once.
+- [ ] Incident escalation packet template is known to operators.
+
+## D) Quality Gates
+
+- [ ] Lint/type checks pass.
+- [ ] `pytest -m "not external" -q` passes.
+- [ ] Targeted external/provider checks executed (if credentials available).
+- [ ] Release evidence recorded in `docs/release_evidence_v1.md`.
+
+## E) Traceability and Documentation
+
+- [ ] `docs/requirements.md` aligns with implemented V1 behavior.
+- [ ] `docs/architecture.md`, `docs/schema.md`, and `docs/error_handling.md` are consistent.
+- [ ] `docs/traceability_v1.md` is updated with current implementation and test evidence.
+- [ ] `docs/ver1/ver1.md` phase status updated with evidence references.
+- [ ] REQ traceability evidence links recorded (tests/runbook/checks).
+
+## Release Sign-Off
+
+- [ ] Technical sign-off complete.
+- [ ] Operational sign-off complete.
+- [ ] V1 completion date recorded.
@@ -0,0 +1,39 @@
+# V1 Release Evidence Log
+
+## Step 5 Quality Gates (2026-07-29)
+
+### Lint
+
+- Command: `python -m ruff check .`
+- Result: ✅ pass
+- Notes: initial findings were auto-fixed (`ruff --fix`) plus small manual line-wrap/annotation adjustments.
+
+### Tests (primary gate)
+
+- Command: `python -m pytest -m "not external" -q`
+- Result: ✅ pass (`[100%]`)
+
+### Tests (external smoke)
+
+- Command: `python -m pytest -m external -q`
+- Result: ✅ pass (`[100%]`)
+
+### Type Check
+
+- Command: `python -m ty check src tests`
+- Result: ⚠️ not passing
+- Summary: existing SQLModel/SQLAlchemy typing incompatibilities and test double typing mismatches remain.
+
+Key current blocker families:
+
+1. SQLModel relationship/query attribute typing (`selectinload`, `order_by`, `.any()`)
+2. SQLAlchemy join clause typing in `services/transcription.py`
+3. Test fake client type mismatch for `OpenRouterTranscriptionProvider(client=...)`
+4. `Settings(**defaults)` typed-dict strictness in `tests/test_config.py`
+
+## Current Gate Status
+
+- Lint: pass
+- Non-external tests: pass
+- External smoke tests: pass
+- Type check: **blocked** (requires dedicated typing cleanup pass)
@@ -0,0 +1,40 @@
+# V1 Traceability Matrix
+
+This matrix provides implementation and validation evidence for V1 requirements (`REQ-0` through `REQ-13`).
+
+Status values:
+
+- `done`: implemented and evidence recorded
+- `in progress`: partially implemented or evidence incomplete
+- `not started`: no implementation/evidence yet
+
+## Requirement Evidence Table
+
+| Requirement | Status | Implementation Evidence | Validation Evidence |
+| --- | --- | --- | --- |
+| REQ-0 | done | End-to-end upload + worker pipeline in `src/transcription/services/store.py`, `src/transcription/worker.py`, `src/transcription/services/workflows.py` | `tests/integration/test_pipeline_flow.py` |
+| REQ-1 | done | Upload UI/page flow in `src/transcription/ui/pages/upload_page.py`, `src/transcription/ui/components/upload.py` | `tests/ui/test_upload_page.py`, `tests/integration/test_pipeline_flow.py` |
+| REQ-2 | done | Async worker execution and provider call orchestration in `src/transcription/worker.py`, `src/transcription/services/workflows.py` | `tests/integration/test_pipeline_flow.py`, `tests/services/test_workflows_reliability.py` |
+| REQ-3 | done | Job lifecycle state model + transitions in `src/transcription/models.py`, `src/transcription/services/jobs.py`, `src/transcription/services/workflows.py` | `tests/services/test_job_service.py`, `tests/ui/test_jobs_page.py` |
+| REQ-4 | done | Persistence of original output and failure detail in `src/transcription/services/transcription.py`, `src/transcription/services/workflows.py` | `tests/integration/test_pipeline_flow.py`, `tests/services/test_workflows_reliability.py` |
+| REQ-5 | done | Status/result inspection via UI pages and API health route in `src/transcription/ui/pages/jobs_page.py`, `src/transcription/api/health.py` | `tests/ui/test_jobs_page.py`, `tests/ui/test_pages_registration.py`, `tests/api/test_health.py` |
+| REQ-6 | done | Background processing trigger/worker notifier and non-blocking workflow in `src/transcription/ui/components/upload.py`, `src/transcription/worker.py` | `tests/test_app.py`, `tests/services/test_workflows_reliability.py` |
+| REQ-7 | done | Lifespan-owned runtime resources in `src/transcription/app.py`, `src/transcription/db/runtime.py` | `tests/test_app.py`, `tests/test_db.py` |
+| REQ-8 | done | Centralized settings/logging initialization in `src/transcription/config.py`, `src/transcription/app.py` | `tests/test_config.py`, `tests/test_app.py` |
+| REQ-9 | done | Containerized runtime baseline in `docker-compose.yml`, `Dockerfile` | `docs/release_checklist_v1.md` (Ops checklist), manual demonstration step |
+| REQ-10 | done | Explicit schema bootstrap policy + runtime controls in `src/transcription/config.py`, `src/transcription/app.py`, `src/transcription/db/operations.py` | `tests/test_db.py`, `tests/test_config.py` |
+| REQ-11 | done | Service/workflow persistence boundaries in `src/transcription/services/*.py`, `src/transcription/services/workflows.py` | `tests/services/test_job_service.py`, `tests/services/test_transcription_service.py` |
+| REQ-12 | done | Prompt artifacts in `prompts/` and loading/validation in `src/transcription/services/transcription.py` | `tests/test_prompts.py` |
+| REQ-13 | done | Optional single revision create/update/view/delete in `src/transcription/services/transcription.py`, `src/transcription/ui/pages/jobs_page.py` | `tests/services/test_transcription_service.py`, `tests/ui/test_jobs_page.py` |
+
+## Operational Evidence (Step 3 Artifacts)
+
+- Runbook: `docs/runbook.md`
+- Migration/backfill/rollback guidance: `docs/migration_v1.md`
+- Release readiness checklist: `docs/release_checklist_v1.md`
+
+## Verification Cadence
+
+- Per change: maintain `tests/test_traceability.py` mappings for touched requirements.
+- Per milestone: update this table status and evidence links.
+- Pre-release: confirm all rows are `done` and non-external suite is green.
@@ -150,6 +150,9 @@ V1 is complete when all of the following are true:

 ### Deliverables
 - V1 release checklist and acceptance evidence.
+- `docs/runbook.md` for incident response and operator workflows.
+- `docs/migration_v1.md` for V1 migration/backfill/rollback guidance.
+- `docs/release_checklist_v1.md` for release sign-off.

 ### Exit Criteria
 - Stakeholder sign-off and launch readiness achieved.