generated from john/python-template
Ver1 Implementation Plan, and a detailed impl plan for step 1.
This commit is contained in:
Jim Lancaster
@@ -0,0 +1,309 @@
|
||||
# Step 1 Implementation Plan: Architecture Consolidation
|
||||
|
||||
## Purpose
|
||||
|
||||
Align the implemented MVP codebase with the production architecture and V1 constraints documented in:
|
||||
|
||||
- `docs/architecture.md`
|
||||
- `docs/requirements.md`
|
||||
- `docs/error_handling.md`
|
||||
- `docs/index.md`
|
||||
- `docs/intent.md`
|
||||
- `docs/ver1/ver1.md` (Step 1)
|
||||
|
||||
This step hardens architecture boundaries and ownership without expanding product scope.
|
||||
|
||||
---
|
||||
|
||||
## MCP Skill and Guide Inputs Incorporated
|
||||
|
||||
This plan explicitly incorporates patterns and guardrails from john-stream-mcp resources:
|
||||
|
||||
1. `resource://skills/fastapi-uv-docker/document`
|
||||
- App factory and lifespan ownership
|
||||
- Health endpoint and cloud-native baseline expectations
|
||||
- Environment-driven configuration and startup discipline
|
||||
|
||||
2. `resource://skills/fastapi-async-sqlalchemy-modernization/document`
|
||||
- Current-state gap audit first
|
||||
- Target runtime model before refactor
|
||||
- Explicit resource lifecycle ownership
|
||||
- Transaction/session boundary clarity
|
||||
- Phased migration with rollback points
|
||||
|
||||
3. `resource://skills/nicegui/document`
|
||||
- Clear dependency direction
|
||||
- UI/page registration as composition, not business logic container
|
||||
- Async responsiveness and boundary separation
|
||||
|
||||
4. `resource://prompts/greenfield-architecture/document`
|
||||
- Pattern-comparison-first planning
|
||||
- Explicit tradeoffs and staged implementation
|
||||
- Output contract with risks, open questions, and next steps
|
||||
|
||||
---
|
||||
|
||||
## Current-State Gap Summary (Architecture vs Implementation)
|
||||
|
||||
Based on docs and current `src/transcription` code:
|
||||
|
||||
1. **REQ-7 gap (lifespan-owned resources)**
|
||||
- DB engine/session factory are module globals in `db.py`, not app lifespan-owned.
|
||||
- Worker thread lifecycle is owned by lifespan (good), but DB/provider resource ownership is mixed.
|
||||
|
||||
2. **REQ-10 gap (explicit opt-in schema bootstrap)**
|
||||
- `create_all()` is executed unconditionally on startup in `app.py`.
|
||||
|
||||
3. **Data store target gap (REQ-9 + architecture baseline)**
|
||||
- Runtime still defaults to SQLite MVP setup; production architecture targets PostgreSQL baseline with optional MongoDB.
|
||||
|
||||
4. **Layering clarity gap (architecture layer model)**
|
||||
- Boundaries exist but are not yet formally enforced (interface/app/domain/infra dependency rules are implicit, not codified).
|
||||
|
||||
5. **Decision record gap**
|
||||
- No ADR set documenting key V1 architectural decisions and deviations from MVP.
|
||||
|
||||
---
|
||||
|
||||
## Scope for Step 1
|
||||
|
||||
### In scope
|
||||
1. Produce architecture conformance audit and decision records.
|
||||
2. Define and implement target runtime ownership model for core resources.
|
||||
3. Establish explicit schema bootstrap policy (opt-in in production paths).
|
||||
4. Consolidate module boundaries and dependency direction rules.
|
||||
5. Update architecture docs to reflect implemented reality and V1 trajectory.
|
||||
|
||||
### Out of scope
|
||||
- Full async SQLAlchemy rewrite (plan and seams only if deferred)
|
||||
- MongoDB feature implementation
|
||||
- New user-facing features
|
||||
- Major worker architecture replacement (in-process worker remains baseline)
|
||||
|
||||
---
|
||||
|
||||
## Target Architecture Decisions for V1
|
||||
|
||||
1. **Keep modular monolith topology** (FastAPI + NiceGUI + in-process worker).
|
||||
2. **Preserve container-light simplicity guardrails** from `architecture.md`.
|
||||
3. **Move runtime ownership to lifespan** for:
|
||||
- DB engine/session factory lifecycle
|
||||
- Worker runtime resources
|
||||
- Provider client factory/config lifecycle
|
||||
4. **Adopt explicit schema bootstrap policy**:
|
||||
- Dev/test: opt-in auto-bootstrap allowed
|
||||
- Production: startup must not mutate schema implicitly
|
||||
5. **Formalize boundary map**:
|
||||
- Interface (`api`, `ui`) -> Application (`services`) -> Domain (`models/rules`) -> Infrastructure (`db`, `providers`)
|
||||
- No reverse imports
|
||||
|
||||
---
|
||||
|
||||
## Detailed Work Breakdown
|
||||
|
||||
## Phase A — Architecture Audit and Baseline Freeze
|
||||
|
||||
- [ ] **A1. Produce architecture conformance matrix**
|
||||
- Map each architecture section to current modules/files.
|
||||
- Classify each row: `aligned`, `partial`, `not aligned`.
|
||||
|
||||
- [ ] **A2. Produce REQ-7/REQ-9/REQ-10 focused gap report**
|
||||
- Explicitly capture current vs required state.
|
||||
- Include operational risk if left unresolved.
|
||||
|
||||
- [ ] **A3. Freeze MVP architecture baseline**
|
||||
- Record current baseline behavior and known temporary shortcuts.
|
||||
- Link this baseline from `docs/ver1/ver1.md`.
|
||||
|
||||
### Deliverables
|
||||
- `docs/ver1/ver1-step1-audit.md` (or equivalent section in this doc)
|
||||
- Architecture conformance table
|
||||
|
||||
### Exit Criteria
|
||||
- No architecture changes begin before gap matrix and baseline are approved.
|
||||
|
||||
---
|
||||
|
||||
## Phase B — Resource Ownership Consolidation (Lifespan-Centric)
|
||||
|
||||
- [ ] **B1. Define runtime resource ownership contract**
|
||||
- `app.py` lifespan owns resource initialization and cleanup order.
|
||||
- `app.state` carries resource handles/factories.
|
||||
- No hidden module-global side-effect initialization for runtime resources.
|
||||
|
||||
- [ ] **B2. Refactor DB ownership model**
|
||||
- Replace module-global engine singleton pattern with lifespan-initialized resource model.
|
||||
- Define one canonical session-factory access path for app/worker/services.
|
||||
|
||||
- [ ] **B3. Normalize worker dependencies**
|
||||
- Ensure worker uses lifespan-owned resources/factories rather than implicit globals.
|
||||
- Preserve deterministic startup/shutdown behavior.
|
||||
|
||||
- [ ] **B4. Define provider adapter ownership**
|
||||
- Provider client creation strategy is centralized and lifecycle-aware.
|
||||
- Avoid per-call hidden client construction when unnecessary.
|
||||
|
||||
### MCP-Guided Guardrails
|
||||
- Use explicit lifecycle composition patterns from `fastapi-async-sqlalchemy-modernization`.
|
||||
- Maintain app-factory + lifespan structure per `fastapi-uv-docker`.
|
||||
- Keep UI registration as composition only per `nicegui`.
|
||||
|
||||
### Exit Criteria
|
||||
- Core runtime resources have one owner and one cleanup path.
|
||||
- No critical resource has ambiguous ownership.
|
||||
|
||||
---
|
||||
|
||||
## Phase C — Schema Bootstrap Policy (REQ-10 Alignment)
|
||||
|
||||
- [ ] **C1. Define environment-aware bootstrap policy**
|
||||
- `auto_create_schema` (or equivalent) disabled in production by default.
|
||||
- Startup schema mutation is explicit and intentional.
|
||||
|
||||
- [ ] **C2. Split startup responsibilities**
|
||||
- App startup performs health-critical initialization only.
|
||||
- Schema bootstrap path is moved to explicit command/flag workflow.
|
||||
|
||||
- [ ] **C3. Update deployment/runbook docs**
|
||||
- Document migration/bootstrap flow for dev, staging, prod.
|
||||
- Ensure policy is testable and auditable.
|
||||
|
||||
### Exit Criteria
|
||||
- Normal production startup path does not call schema auto-create implicitly.
|
||||
- Bootstrap behavior is explicit and documented.
|
||||
|
||||
---
|
||||
|
||||
## Phase D — Module Boundary Enforcement
|
||||
|
||||
- [ ] **D1. Publish dependency direction rules**
|
||||
- Allowed import directions across `api`, `ui`, `services`, `models/domain`, `db/providers`.
|
||||
- Explicitly disallow reverse dependencies.
|
||||
|
||||
- [ ] **D2. Reconcile package map with docs**
|
||||
- Ensure docs’ architecture elements match real package layout and naming.
|
||||
- Update docs where intentional deviations remain.
|
||||
|
||||
- [ ] **D3. Isolate cross-layer responsibilities**
|
||||
- Keep API/UI presentation concerns out of services.
|
||||
- Keep provider/DB specifics out of interface layer.
|
||||
|
||||
- [ ] **D4. Add lightweight architecture checks**
|
||||
- Add static/import checks and/or review checklist in CI/review process.
|
||||
|
||||
### Exit Criteria
|
||||
- Boundary rules are documented and applied.
|
||||
- Architectural drift can be detected during review/CI.
|
||||
|
||||
---
|
||||
|
||||
## Phase E — Architecture Decision Records (ADRs)
|
||||
|
||||
- [ ] **E1. Create ADR index**
|
||||
- Add `docs/adr/README.md` with template and status model.
|
||||
|
||||
- [ ] **E2. Record minimum V1 ADR set**
|
||||
1. Runtime ownership model (lifespan-owned resources)
|
||||
2. Schema bootstrap policy (explicit vs implicit)
|
||||
3. Persistence baseline (PostgreSQL target; SQLite transition strategy)
|
||||
4. Worker topology (in-process for V1, extension path preserved)
|
||||
|
||||
- [ ] **E3. Cross-link ADRs**
|
||||
- Link from architecture and V1 docs.
|
||||
|
||||
### Exit Criteria
|
||||
- Major architecture decisions are explicit, versioned, and discoverable.
|
||||
|
||||
---
|
||||
|
||||
## Phase F — Documentation Consolidation
|
||||
|
||||
- [ ] **F1. Update `docs/architecture.md`**
|
||||
- Reflect real implementation and V1 target state separately.
|
||||
- Mark transitional choices clearly.
|
||||
|
||||
- [ ] **F2. Update `docs/index.md` navigation consistency**
|
||||
- Ensure architecture/readme references match actual docs/files.
|
||||
|
||||
- [ ] **F3. Update `docs/requirements.md` traceability notes**
|
||||
- Mark REQ-7/REQ-10 status and verification approach after consolidation.
|
||||
|
||||
- [ ] **F4. Add Step 1 result summary**
|
||||
- Create `docs/ver1/ver1-step1-results.md` after implementation.
|
||||
|
||||
### Exit Criteria
|
||||
- Docs are internally consistent and match runtime architecture reality.
|
||||
|
||||
---
|
||||
|
||||
## Verification Plan
|
||||
|
||||
## Architecture Verification Matrix (Step 1)
|
||||
|
||||
1. **Inspection**
|
||||
- Resource ownership map exists and matches code.
|
||||
- Schema bootstrap policy is explicit and environment-aware.
|
||||
- ADRs exist for each key architecture decision.
|
||||
|
||||
2. **Automated checks**
|
||||
- Existing test suite remains green.
|
||||
- New/updated tests validate startup policy (no implicit schema mutation in production mode).
|
||||
- Import/dependency-direction checks pass (if introduced).
|
||||
|
||||
3. **Demonstration**
|
||||
- App starts in dev mode with explicit expected behavior.
|
||||
- App starts in production mode without mutating schema implicitly.
|
||||
- Worker lifecycle starts/stops cleanly with app lifespan.
|
||||
|
||||
---
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
1. **Risk:** Refactor destabilizes MVP behavior
|
||||
**Mitigation:** Phase changes with small PRs and regression checks after each phase.
|
||||
|
||||
2. **Risk:** Over-rotation into premature async rewrite
|
||||
**Mitigation:** Keep this step focused on lifecycle ownership and boundaries; defer full async migration unless required.
|
||||
|
||||
3. **Risk:** Schema policy changes break local DX
|
||||
**Mitigation:** Keep explicit dev bootstrap path simple and documented.
|
||||
|
||||
4. **Risk:** Boundary rules become “doc only”
|
||||
**Mitigation:** Add CI/review enforcement and architecture checklist.
|
||||
|
||||
---
|
||||
|
||||
## Recommended Implementation Order
|
||||
|
||||
1. Phase A — Audit and baseline freeze
|
||||
2. Phase B — Resource ownership consolidation
|
||||
3. Phase C — Schema bootstrap policy
|
||||
4. Phase D — Boundary enforcement
|
||||
5. Phase E — ADR authoring
|
||||
6. Phase F — Documentation consolidation
|
||||
|
||||
This order minimizes risk: diagnose first, then refactor ownership, then lock policy, then enforce boundaries, and finally finalize docs.
|
||||
|
||||
---
|
||||
|
||||
## Step 1 Completion Checklist
|
||||
|
||||
- [ ] Architecture conformance matrix approved.
|
||||
- [ ] REQ-7 ownership gaps resolved or explicitly deferred with owner/date.
|
||||
- [ ] REQ-10 explicit bootstrap policy implemented and verified.
|
||||
- [ ] Dependency direction rules documented and enforced.
|
||||
- [ ] ADR set created for all major Step 1 decisions.
|
||||
- [ ] Architecture and index docs updated to match implementation.
|
||||
- [ ] Full test suite passes after consolidation.
|
||||
- [ ] `docs/ver1/ver1-step1-results.md` created with evidence and residual risks.
|
||||
|
||||
---
|
||||
|
||||
## Handoff to Step 2
|
||||
|
||||
Once Step 1 completes, Step 2 (Error Handling & Reliability Hardening) can proceed on stable architecture seams:
|
||||
|
||||
- consistent lifecycle ownership,
|
||||
- explicit startup policy,
|
||||
- clear module boundaries,
|
||||
- documented architecture decisions.
|
||||
Reference in New Issue
Block a user