# Step 1 Implementation Plan: Architecture Consolidation ## Purpose Align the implemented MVP codebase with the production architecture and V1 constraints documented in: - `docs/architecture.md` - `docs/requirements.md` - `docs/error_handling.md` - `docs/index.md` - `docs/intent.md` - `docs/ver1/ver1.md` (Step 1) This step hardens architecture boundaries and ownership without expanding product scope. --- ## MCP Skill and Guide Inputs Incorporated This plan explicitly incorporates patterns and guardrails from john-stream-mcp resources: 1. `resource://skills/fastapi-uv-docker/document` - App factory and lifespan ownership - Health endpoint and cloud-native baseline expectations - Environment-driven configuration and startup discipline 2. `resource://skills/fastapi-async-sqlalchemy-modernization/document` - Current-state gap audit first - Target runtime model before refactor - Explicit resource lifecycle ownership - Transaction/session boundary clarity - Phased migration with rollback points 3. `resource://skills/nicegui/document` - Clear dependency direction - UI/page registration as composition, not business logic container - Async responsiveness and boundary separation 4. `resource://prompts/greenfield-architecture/document` - Pattern-comparison-first planning - Explicit tradeoffs and staged implementation - Output contract with risks, open questions, and next steps --- ## Current-State Gap Summary (Architecture vs Implementation) Based on docs and current `src/transcription` code: 1. **REQ-7 gap (lifespan-owned resources)** - DB engine/session factory are module globals in `db.py`, not app lifespan-owned. - Worker thread lifecycle is owned by lifespan (good), but DB/provider resource ownership is mixed. 2. **REQ-10 gap (explicit opt-in schema bootstrap)** - `create_all()` is executed unconditionally on startup in `app.py`. 3. **Data store target gap (REQ-9 + architecture baseline)** - Runtime still defaults to SQLite MVP setup; production architecture targets PostgreSQL baseline with optional MongoDB. 4. **Layering clarity gap (architecture layer model)** - Boundaries exist but are not yet formally enforced (interface/app/domain/infra dependency rules are implicit, not codified). 5. **Decision record gap** - No ADR set documenting key V1 architectural decisions and deviations from MVP. --- ## Scope for Step 1 ### In scope 1. Produce architecture conformance audit and decision records. 2. Define and implement target runtime ownership model for core resources. 3. Establish explicit schema bootstrap policy (opt-in in production paths). 4. Consolidate module boundaries and dependency direction rules. 5. Update architecture docs to reflect implemented reality and V1 trajectory. ### Out of scope - Full async SQLAlchemy rewrite (plan and seams only if deferred) - MongoDB feature implementation - New user-facing features - Major worker architecture replacement (in-process worker remains baseline) --- ## Target Architecture Decisions for V1 1. **Keep modular monolith topology** (FastAPI + NiceGUI + in-process worker). 2. **Preserve container-light simplicity guardrails** from `architecture.md`. 3. **Move runtime ownership to lifespan** for: - DB engine/session factory lifecycle - Worker runtime resources - Provider client factory/config lifecycle 4. **Adopt explicit schema bootstrap policy**: - Dev/test: opt-in auto-bootstrap allowed - Production: startup must not mutate schema implicitly 5. **Formalize boundary map**: - Interface (`api`, `ui`) -> Application (`services`) -> Domain (`models/rules`) -> Infrastructure (`db`, `providers`) - No reverse imports --- ## Detailed Work Breakdown ## Phase A — Architecture Audit and Baseline Freeze - [ ] **A1. Produce architecture conformance matrix** - Map each architecture section to current modules/files. - Classify each row: `aligned`, `partial`, `not aligned`. - [ ] **A2. Produce REQ-7/REQ-9/REQ-10 focused gap report** - Explicitly capture current vs required state. - Include operational risk if left unresolved. - [ ] **A3. Freeze MVP architecture baseline** - Record current baseline behavior and known temporary shortcuts. - Link this baseline from `docs/ver1/ver1.md`. ### Deliverables - `docs/ver1/ver1-step1-audit.md` (or equivalent section in this doc) - Architecture conformance table ### Exit Criteria - No architecture changes begin before gap matrix and baseline are approved. --- ## Phase B — Resource Ownership Consolidation (Lifespan-Centric) - [ ] **B1. Define runtime resource ownership contract** - `app.py` lifespan owns resource initialization and cleanup order. - `app.state` carries resource handles/factories. - No hidden module-global side-effect initialization for runtime resources. - [ ] **B2. Refactor DB ownership model** - Replace module-global engine singleton pattern with lifespan-initialized resource model. - Define one canonical session-factory access path for app/worker/services. - [ ] **B3. Normalize worker dependencies** - Ensure worker uses lifespan-owned resources/factories rather than implicit globals. - Preserve deterministic startup/shutdown behavior. - [ ] **B4. Define provider adapter ownership** - Provider client creation strategy is centralized and lifecycle-aware. - Avoid per-call hidden client construction when unnecessary. ### MCP-Guided Guardrails - Use explicit lifecycle composition patterns from `fastapi-async-sqlalchemy-modernization`. - Maintain app-factory + lifespan structure per `fastapi-uv-docker`. - Keep UI registration as composition only per `nicegui`. ### Exit Criteria - Core runtime resources have one owner and one cleanup path. - No critical resource has ambiguous ownership. --- ## Phase C — Schema Bootstrap Policy (REQ-10 Alignment) - [ ] **C1. Define environment-aware bootstrap policy** - `auto_create_schema` (or equivalent) disabled in production by default. - Startup schema mutation is explicit and intentional. - [ ] **C2. Split startup responsibilities** - App startup performs health-critical initialization only. - Schema bootstrap path is moved to explicit command/flag workflow. - [ ] **C3. Update deployment/runbook docs** - Document migration/bootstrap flow for dev, staging, prod. - Ensure policy is testable and auditable. ### Exit Criteria - Normal production startup path does not call schema auto-create implicitly. - Bootstrap behavior is explicit and documented. --- ## Phase D — Module Boundary Enforcement - [ ] **D1. Publish dependency direction rules** - Allowed import directions across `api`, `ui`, `services`, `models/domain`, `db/providers`. - Explicitly disallow reverse dependencies. - [ ] **D2. Reconcile package map with docs** - Ensure docs’ architecture elements match real package layout and naming. - Update docs where intentional deviations remain. - [ ] **D3. Isolate cross-layer responsibilities** - Keep API/UI presentation concerns out of services. - Keep provider/DB specifics out of interface layer. - [ ] **D4. Add lightweight architecture checks** - Add static/import checks and/or review checklist in CI/review process. ### Exit Criteria - Boundary rules are documented and applied. - Architectural drift can be detected during review/CI. --- ## Phase E — Architecture Decision Records (ADRs) - [ ] **E1. Create ADR index** - Add `docs/adr/README.md` with template and status model. - [ ] **E2. Record minimum V1 ADR set** 1. Runtime ownership model (lifespan-owned resources) 2. Schema bootstrap policy (explicit vs implicit) 3. Persistence baseline (PostgreSQL target; SQLite transition strategy) 4. Worker topology (in-process for V1, extension path preserved) - [ ] **E3. Cross-link ADRs** - Link from architecture and V1 docs. ### Exit Criteria - Major architecture decisions are explicit, versioned, and discoverable. --- ## Phase F — Documentation Consolidation - [ ] **F1. Update `docs/architecture.md`** - Reflect real implementation and V1 target state separately. - Mark transitional choices clearly. - [ ] **F2. Update `docs/index.md` navigation consistency** - Ensure architecture/readme references match actual docs/files. - [ ] **F3. Update `docs/requirements.md` traceability notes** - Mark REQ-7/REQ-10 status and verification approach after consolidation. - [ ] **F4. Add Step 1 result summary** - Create `docs/ver1/ver1-step1-results.md` after implementation. ### Exit Criteria - Docs are internally consistent and match runtime architecture reality. --- ## Verification Plan ## Architecture Verification Matrix (Step 1) 1. **Inspection** - Resource ownership map exists and matches code. - Schema bootstrap policy is explicit and environment-aware. - ADRs exist for each key architecture decision. 2. **Automated checks** - Existing test suite remains green. - New/updated tests validate startup policy (no implicit schema mutation in production mode). - Import/dependency-direction checks pass (if introduced). 3. **Demonstration** - App starts in dev mode with explicit expected behavior. - App starts in production mode without mutating schema implicitly. - Worker lifecycle starts/stops cleanly with app lifespan. --- ## Risks and Mitigations 1. **Risk:** Refactor destabilizes MVP behavior **Mitigation:** Phase changes with small PRs and regression checks after each phase. 2. **Risk:** Over-rotation into premature async rewrite **Mitigation:** Keep this step focused on lifecycle ownership and boundaries; defer full async migration unless required. 3. **Risk:** Schema policy changes break local DX **Mitigation:** Keep explicit dev bootstrap path simple and documented. 4. **Risk:** Boundary rules become “doc only” **Mitigation:** Add CI/review enforcement and architecture checklist. --- ## Recommended Implementation Order 1. Phase A — Audit and baseline freeze 2. Phase B — Resource ownership consolidation 3. Phase C — Schema bootstrap policy 4. Phase D — Boundary enforcement 5. Phase E — ADR authoring 6. Phase F — Documentation consolidation This order minimizes risk: diagnose first, then refactor ownership, then lock policy, then enforce boundaries, and finally finalize docs. --- ## Step 1 Completion Checklist - [ ] Architecture conformance matrix approved. - [ ] REQ-7 ownership gaps resolved or explicitly deferred with owner/date. - [ ] REQ-10 explicit bootstrap policy implemented and verified. - [ ] Dependency direction rules documented and enforced. - [ ] ADR set created for all major Step 1 decisions. - [ ] Architecture and index docs updated to match implementation. - [ ] Full test suite passes after consolidation. - [ ] `docs/ver1/ver1-step1-results.md` created with evidence and residual risks. --- ## Handoff to Step 2 Once Step 1 completes, Step 2 (Error Handling & Reliability Hardening) can proceed on stable architecture seams: - consistent lifecycle ownership, - explicit startup policy, - clear module boundaries, - documented architecture decisions.