transcription/docs/ver1/ver1-step1.md

# Step 1 Implementation Plan: Architecture Consolidation

## Purpose

Align the implemented MVP codebase with the production architecture and V1 constraints documented in:

- `docs/architecture.md`
- `docs/requirements.md`
- `docs/error_handling.md`
- `docs/index.md`
- `docs/intent.md`
- `docs/ver1/ver1.md` (Step 1)

This step hardens architecture boundaries and ownership without expanding product scope.

---

## MCP Skill and Guide Inputs Incorporated

This plan explicitly incorporates patterns and guardrails from john-stream-mcp resources:

1. `resource://skills/fastapi-uv-docker/document`
   - App factory and lifespan ownership
   - Health endpoint and cloud-native baseline expectations
   - Environment-driven configuration and startup discipline

2. `resource://skills/fastapi-async-sqlalchemy-modernization/document`
   - Current-state gap audit first
   - Target runtime model before refactor
   - Explicit resource lifecycle ownership
   - Transaction/session boundary clarity
   - Phased migration with rollback points

3. `resource://skills/nicegui/document`
   - Clear dependency direction
   - UI/page registration as composition, not business logic container
   - Async responsiveness and boundary separation

4. `resource://prompts/greenfield-architecture/document`
   - Pattern-comparison-first planning
   - Explicit tradeoffs and staged implementation
   - Output contract with risks, open questions, and next steps

---

## Current-State Gap Summary (Architecture vs Implementation)

Based on docs and current `src/transcription` code:

1. **REQ-7 gap (lifespan-owned resources)**  
   - DB engine/session factory are module globals in `db.py`, not app lifespan-owned.
   - Worker thread lifecycle is owned by lifespan (good), but DB/provider resource ownership is mixed.

2. **REQ-10 gap (explicit opt-in schema bootstrap)**  
   - `create_all()` is executed unconditionally on startup in `app.py`.

3. **Data store target gap (REQ-9 + architecture baseline)**  
   - Runtime still defaults to SQLite MVP setup; production architecture targets PostgreSQL baseline with optional MongoDB.

4. **Layering clarity gap (architecture layer model)**  
   - Boundaries exist but are not yet formally enforced (interface/app/domain/infra dependency rules are implicit, not codified).

5. **Decision record gap**  
   - No ADR set documenting key V1 architectural decisions and deviations from MVP.

---

## Scope for Step 1

### In scope
1. Produce architecture conformance audit and decision records.
2. Define and implement target runtime ownership model for core resources.
3. Establish explicit schema bootstrap policy (opt-in in production paths).
4. Consolidate module boundaries and dependency direction rules.
5. Update architecture docs to reflect implemented reality and V1 trajectory.

### Out of scope
- Full async SQLAlchemy rewrite (plan and seams only if deferred)
- MongoDB feature implementation
- New user-facing features
- Major worker architecture replacement (in-process worker remains baseline)

---

## Target Architecture Decisions for V1

1. **Keep modular monolith topology** (FastAPI + NiceGUI + in-process worker).
2. **Preserve container-light simplicity guardrails** from `architecture.md`.
3. **Move runtime ownership to lifespan** for:
   - DB engine/session factory lifecycle
   - Worker runtime resources
   - Provider client factory/config lifecycle
4. **Adopt explicit schema bootstrap policy**:
   - Dev/test: opt-in auto-bootstrap allowed
   - Production: startup must not mutate schema implicitly
5. **Formalize boundary map**:
   - Interface (`api`, `ui`) -> Application (`services`) -> Domain (`models/rules`) -> Infrastructure (`db`, `providers`)
   - No reverse imports

---

## Detailed Work Breakdown

## Phase A — Architecture Audit and Baseline Freeze

- [ ] **A1. Produce architecture conformance matrix**
  - Map each architecture section to current modules/files.
  - Classify each row: `aligned`, `partial`, `not aligned`.

- [ ] **A2. Produce REQ-7/REQ-9/REQ-10 focused gap report**
  - Explicitly capture current vs required state.
  - Include operational risk if left unresolved.

- [ ] **A3. Freeze MVP architecture baseline**
  - Record current baseline behavior and known temporary shortcuts.
  - Link this baseline from `docs/ver1/ver1.md`.

### Deliverables
- `docs/ver1/ver1-step1-audit.md` (or equivalent section in this doc)
- Architecture conformance table

### Exit Criteria
- No architecture changes begin before gap matrix and baseline are approved.

---

## Phase B — Resource Ownership Consolidation (Lifespan-Centric)

- [ ] **B1. Define runtime resource ownership contract**
  - `app.py` lifespan owns resource initialization and cleanup order.
  - `app.state` carries resource handles/factories.
  - No hidden module-global side-effect initialization for runtime resources.

- [ ] **B2. Refactor DB ownership model**
  - Replace module-global engine singleton pattern with lifespan-initialized resource model.
  - Define one canonical session-factory access path for app/worker/services.

- [ ] **B3. Normalize worker dependencies**
  - Ensure worker uses lifespan-owned resources/factories rather than implicit globals.
  - Preserve deterministic startup/shutdown behavior.

- [ ] **B4. Define provider adapter ownership**
  - Provider client creation strategy is centralized and lifecycle-aware.
  - Avoid per-call hidden client construction when unnecessary.

### MCP-Guided Guardrails
- Use explicit lifecycle composition patterns from `fastapi-async-sqlalchemy-modernization`.
- Maintain app-factory + lifespan structure per `fastapi-uv-docker`.
- Keep UI registration as composition only per `nicegui`.

### Exit Criteria
- Core runtime resources have one owner and one cleanup path.
- No critical resource has ambiguous ownership.

---

## Phase C — Schema Bootstrap Policy (REQ-10 Alignment)

- [ ] **C1. Define environment-aware bootstrap policy**
  - `auto_create_schema` (or equivalent) disabled in production by default.
  - Startup schema mutation is explicit and intentional.

- [ ] **C2. Split startup responsibilities**
  - App startup performs health-critical initialization only.
  - Schema bootstrap path is moved to explicit command/flag workflow.

- [ ] **C3. Update deployment/runbook docs**
  - Document migration/bootstrap flow for dev, staging, prod.
  - Ensure policy is testable and auditable.

### Exit Criteria
- Normal production startup path does not call schema auto-create implicitly.
- Bootstrap behavior is explicit and documented.

---

## Phase D — Module Boundary Enforcement

- [ ] **D1. Publish dependency direction rules**
  - Allowed import directions across `api`, `ui`, `services`, `models/domain`, `db/providers`.
  - Explicitly disallow reverse dependencies.

- [ ] **D2. Reconcile package map with docs**
  - Ensure docs’ architecture elements match real package layout and naming.
  - Update docs where intentional deviations remain.

- [ ] **D3. Isolate cross-layer responsibilities**
  - Keep API/UI presentation concerns out of services.
  - Keep provider/DB specifics out of interface layer.

- [ ] **D4. Add lightweight architecture checks**
  - Add static/import checks and/or review checklist in CI/review process.

### Exit Criteria
- Boundary rules are documented and applied.
- Architectural drift can be detected during review/CI.

---

## Phase E — Architecture Decision Records (ADRs)

- [ ] **E1. Create ADR index**
  - Add `docs/adr/README.md` with template and status model.

- [ ] **E2. Record minimum V1 ADR set**
  1. Runtime ownership model (lifespan-owned resources)
  2. Schema bootstrap policy (explicit vs implicit)
  3. Persistence baseline (PostgreSQL target; SQLite transition strategy)
  4. Worker topology (in-process for V1, extension path preserved)

- [ ] **E3. Cross-link ADRs**
  - Link from architecture and V1 docs.

### Exit Criteria
- Major architecture decisions are explicit, versioned, and discoverable.

---

## Phase F — Documentation Consolidation

- [ ] **F1. Update `docs/architecture.md`**
  - Reflect real implementation and V1 target state separately.
  - Mark transitional choices clearly.

- [ ] **F2. Update `docs/index.md` navigation consistency**
  - Ensure architecture/readme references match actual docs/files.

- [ ] **F3. Update `docs/requirements.md` traceability notes**
  - Mark REQ-7/REQ-10 status and verification approach after consolidation.

- [ ] **F4. Add Step 1 result summary**
  - Create `docs/ver1/ver1-step1-results.md` after implementation.

### Exit Criteria
- Docs are internally consistent and match runtime architecture reality.

---

## Verification Plan

## Architecture Verification Matrix (Step 1)

1. **Inspection**
   - Resource ownership map exists and matches code.
   - Schema bootstrap policy is explicit and environment-aware.
   - ADRs exist for each key architecture decision.

2. **Automated checks**
   - Existing test suite remains green.
   - New/updated tests validate startup policy (no implicit schema mutation in production mode).
   - Import/dependency-direction checks pass (if introduced).

3. **Demonstration**
   - App starts in dev mode with explicit expected behavior.
   - App starts in production mode without mutating schema implicitly.
   - Worker lifecycle starts/stops cleanly with app lifespan.

---

## Risks and Mitigations

1. **Risk:** Refactor destabilizes MVP behavior  
   **Mitigation:** Phase changes with small PRs and regression checks after each phase.

2. **Risk:** Over-rotation into premature async rewrite  
   **Mitigation:** Keep this step focused on lifecycle ownership and boundaries; defer full async migration unless required.

3. **Risk:** Schema policy changes break local DX  
   **Mitigation:** Keep explicit dev bootstrap path simple and documented.

4. **Risk:** Boundary rules become “doc only”  
   **Mitigation:** Add CI/review enforcement and architecture checklist.

---

## Recommended Implementation Order

1. Phase A — Audit and baseline freeze  
2. Phase B — Resource ownership consolidation  
3. Phase C — Schema bootstrap policy  
4. Phase D — Boundary enforcement  
5. Phase E — ADR authoring  
6. Phase F — Documentation consolidation

This order minimizes risk: diagnose first, then refactor ownership, then lock policy, then enforce boundaries, and finally finalize docs.

---

## Step 1 Completion Checklist

- [ ] Architecture conformance matrix approved.
- [ ] REQ-7 ownership gaps resolved or explicitly deferred with owner/date.
- [ ] REQ-10 explicit bootstrap policy implemented and verified.
- [ ] Dependency direction rules documented and enforced.
- [ ] ADR set created for all major Step 1 decisions.
- [ ] Architecture and index docs updated to match implementation.
- [ ] Full test suite passes after consolidation.
- [ ] `docs/ver1/ver1-step1-results.md` created with evidence and residual risks.

---

## Handoff to Step 2

Once Step 1 completes, Step 2 (Error Handling & Reliability Hardening) can proceed on stable architecture seams:

- consistent lifecycle ownership,
- explicit startup policy,
- clear module boundaries,
- documented architecture decisions.