V1 mostly complete except for some testing. Linting in the last step changed nearly every file which is why this commit is so larger.

2026-07-29 17:27:21 -05:00
parent bc21a97019
commit a3b3bab571
37 changed files with 1383 additions and 410 deletions
@@ -0,0 +1,125 @@
+# Version 2 Plan
+
+## Purpose
+
+Version 2 extends the V1 baseline by introducing a production-oriented persistence architecture while preserving current user workflows.
+
+Primary target changes:
+
+- Migrate relational persistence from SQLite to PostgreSQL
+- Introduce optional MongoDB for document-oriented adjunct data (non-canonical)
+
+V1 behavior remains the functional baseline unless explicitly superseded by approved V2 requirements.
+
+---
+
+## V2 Goals
+
+1. **Relational migration complete**
+   - PostgreSQL becomes the default system of record for `Document`, `Source`, `Job`, and `Revision`.
+2. **Operational maturity**
+   - Repeatable migrations, rollback paths, and environment-specific deployment procedures are documented and tested.
+3. **Optional document store integration**
+   - MongoDB is introduced only for clearly scoped use cases that do not replace canonical relational ownership.
+4. **No regression of V1 workflows**
+   - Upload, queue/worker processing, status inspection, original transcription, and optional single revision remain stable.
+
+---
+
+## Non-Goals (V2)
+
+- Replacing SQLModel domain ownership with MongoDB
+- Introducing breaking UI behavior for existing V1 flows
+- Expanding revision cardinality beyond current `0..1` without explicit requirements update
+
+---
+
+## Proposed Scope
+
+### A) PostgreSQL migration (required)
+
+- Add PostgreSQL runtime profile for local/dev/prod
+- Introduce migration toolchain and migration history
+- Convert bootstrap strategy from compatibility patching to explicit migrations
+- Validate model constraints and indexes against PostgreSQL
+- Add operational checks (connectivity, pool, transaction behavior)
+
+### B) MongoDB integration (optional, gated)
+
+- Define approved use cases (for example: denormalized read models, audit/event projections, or search-oriented materializations)
+- Keep canonical write path in relational store
+- Add feature flag/config gate to enable or disable Mongo features
+- Document consistency model and failure behavior
+
+---
+
+## Milestones
+
+## M1 — Requirements and architecture baseline
+
+- Create V2 requirements delta from V1 baseline
+- Define relational/document ownership boundaries
+- Approve migration strategy and cutover approach
+
+**Exit criteria:** signed architecture decision and updated traceability map.
+
+## M2 — PostgreSQL foundation
+
+- Add PostgreSQL environment wiring and secrets strategy
+- Add migration framework and initial schema migration
+- Add CI path using PostgreSQL service container
+
+**Exit criteria:** test suite green on PostgreSQL in CI.
+
+## M3 — Data migration and cutover rehearsal
+
+- Build SQLite -> PostgreSQL migration utility/playbook
+- Rehearse migration on representative datasets
+- Validate rollback/recovery procedures
+
+**Exit criteria:** successful dry-run migration with measured rollback test.
+
+## M4 — MongoDB optional integration
+
+- Implement scoped Mongo use case(s)
+- Add fallback behavior when Mongo unavailable
+- Add tests and operational runbook updates
+
+**Exit criteria:** feature-gated Mongo behavior validated with no V1 flow regressions.
+
+## M5 — Release readiness
+
+- Final regression suite (functional + reliability)
+- Performance and failure-mode checks
+- Production release checklist and sign-off
+
+**Exit criteria:** V2 release approval.
+
+---
+
+## Risks and Mitigations
+
+- **Schema drift risk** -> enforce migration-first policy and CI migration checks.
+- **Dual-store consistency risk** -> keep relational source of truth and explicit projection contracts.
+- **Operational complexity** -> staged rollout, runbooks, and feature flags.
+- **Regression risk in worker lifecycle** -> keep dedicated reliability tests around terminal-state guarantees.
+
+---
+
+## Traceability and Evidence
+
+Maintain a V2 table with:
+
+- requirement/change ID
+- status (`not started` / `in progress` / `done`)
+- implementation PR
+- validation evidence (test names, migration rehearsal logs, runbook references)
+
+---
+
+## Suggested first implementation tasks
+
+1. Create `docs/ver2/adr/` and draft ADR for persistence ownership boundaries.
+2. Add PostgreSQL compose profile and env contract.
+3. Introduce migration tooling and generate initial migration from current schema.
+4. Add CI job for PostgreSQL-backed `pytest -m "not external"`.