generated from john/python-template
434 lines
13 KiB
Markdown
434 lines
13 KiB
Markdown
# Step 3 Implementation Plan: Functional Completion by Requirement Domain
|
|
|
|
## Purpose
|
|
|
|
Implement **Ver1 Step 3** from `docs/ver1/ver1.md` by completing all in-scope V1 functional requirements in a practical, user-first order while preserving:
|
|
|
|
- personal-scale operation
|
|
- single-operator workflow
|
|
- private-network deployment assumptions
|
|
- low operational overhead
|
|
- clean architecture boundaries
|
|
|
|
Primary governing docs:
|
|
|
|
- `docs/ver1/ver1.md` (Step 3 objective and sequencing)
|
|
- `docs/architecture.md` (module boundaries, workflow, simplicity guardrails)
|
|
- `docs/requirements.md` (REQ-0 through REQ-12 traceability)
|
|
- `docs/error_handling.md` (error contract across boundaries)
|
|
- `docs/intent.md` (verbatim transcription policy and prompt curation)
|
|
- `docs/ver1/ver1-step1-2-carry-forward-checklist.md` (Step 1/2 carry-forward integration)
|
|
- `docs/ver1/ver1-step2-error-path-inventory.md` (failure-path coverage visibility)
|
|
|
|
---
|
|
|
|
## MCP Resources Reviewed and Applied
|
|
|
|
All resources on `john-stream-mcp` were reviewed. Step 3 applies the following guidance directly:
|
|
|
|
1. `resource://skills/nicegui/document`
|
|
- modular page registration
|
|
- one-way dependency flow (`ui/api -> services -> infra`)
|
|
- async-first UI responsiveness expectations
|
|
|
|
2. `resource://skills/nicegui-ui-customization/document`
|
|
- reusable UI component extraction for repeated patterns
|
|
- in-flight guards and explicit success/failure user feedback
|
|
- event-driven updates over ad-hoc polling
|
|
|
|
3. `resource://skills/fastapi-async-sqlalchemy-modernization/document`
|
|
- explicit transaction/session boundaries
|
|
- deterministic resource ownership and cleanup continuity from Step 1
|
|
- incremental migration strategy with rollback-aware checkpoints
|
|
|
|
4. `resource://skills/pydantic-settings/document`
|
|
- typed configuration as single source of runtime truth
|
|
- explicit source precedence and environment-safe defaults
|
|
|
|
5. `resource://skills/python-logging-dictconfig/document`
|
|
- centralized startup-only logging configuration
|
|
- named logger discipline and boundary-level structured fields
|
|
|
|
6. `resource://skills/pytesting/document`
|
|
- deterministic test structure and marker discipline
|
|
- behavior-first tests with clear fast-path and full-suite validation
|
|
|
|
7. `resource://skills/fastapi-uv-docker/document`
|
|
- health endpoint and runtime startup/shutdown hygiene
|
|
- compose/deployment readiness constraints relevant to functional completion
|
|
|
|
8. `resource://skills/python-typing/document`
|
|
- modern typing updates where touched by Step 3 work
|
|
|
|
9. `resource://skills/ruff-linting-formating/document`
|
|
- maintain lint/format consistency in all modified modules
|
|
|
|
10. `resource://prompts/greenfield-architecture/document`
|
|
- explicit staged delivery with tradeoff-aware sequencing and test strategy
|
|
|
|
11. `resource://prompts/pytest-scaffold/document`
|
|
12. `resource://prompts/pytest-fill-scaffold/document`
|
|
- structure-first test planning, then deterministic implementation fill-in
|
|
|
|
Resources reviewed but not directly in Step 3 execution scope (no changes required now):
|
|
|
|
- `copilot-customization`, `mcp-details`, `vscode-configuration`, `zensical-docs`
|
|
- prompts: `authoring`, `mcp-consumer-repo-shim`
|
|
|
|
---
|
|
|
|
## Step 3 Success Criteria
|
|
|
|
Step 3 is complete when:
|
|
|
|
1. All Step 3-targeted requirement slices are implemented and verified.
|
|
2. Functional behavior is available through UI/API where required.
|
|
3. Core data integrity and state transitions are deterministic.
|
|
4. Error behavior follows `docs/error_handling.md` contracts.
|
|
5. Carry-forward Step 1/2 items mapped to Step 3 are updated with evidence.
|
|
|
|
---
|
|
|
|
## Requirement-Slice Execution Model (Applied to Every Slice)
|
|
|
|
For each slice, execute this sequence:
|
|
|
|
1. Confirm contract/schema and boundary ownership.
|
|
2. Implement service/domain logic.
|
|
3. Implement persistence/state transitions.
|
|
4. Integrate API and/or UI behavior.
|
|
5. Add/update unit + integration + targeted end-to-end tests.
|
|
6. Update docs and traceability artifacts.
|
|
|
|
Definition of done per slice:
|
|
|
|
- behavior is functional
|
|
- tests pass in intended marker lanes
|
|
- error pathways are classified and surfaced correctly
|
|
- requirement traceability is updated with evidence
|
|
|
|
---
|
|
|
|
## Detailed Workstreams
|
|
|
|
## Workstream A — Functional Baseline Audit and Slice Backlog Lock
|
|
|
|
### Goals
|
|
|
|
- establish exact Step 3 functional delta from current implementation
|
|
- lock a practical slice backlog before coding
|
|
|
|
### Tasks
|
|
|
|
1. Build Step 3 requirement matrix (REQ -> current status -> gap -> target slice).
|
|
2. Map each gap to one of these domains:
|
|
- Upload and lifecycle integrity
|
|
- Review and revision history
|
|
- Search over accepted transcripts
|
|
- Export workflows
|
|
- Prompt asset management behavior
|
|
- API/UI parity and status visibility
|
|
3. Align each slice with architecture boundary ownership and persistence strategy.
|
|
4. Link open carry-forward items from checklist:
|
|
- CF-A1, CF-A3 (architecture continuity in Step 3)
|
|
- CF-C1, CF-C2 (traceability/execution continuity)
|
|
|
|
### Deliverables
|
|
|
|
- Step 3 requirement-slice matrix (appendix in this doc or separate artifact)
|
|
- prioritized slice backlog with owner and validation method
|
|
|
|
### Exit Criteria
|
|
|
|
- every Step 3 slice maps to REQ IDs and a validation method
|
|
- no ambiguous ownership remains for in-scope slices
|
|
|
|
---
|
|
|
|
## Workstream B — Core End-User Flows (Upload -> Transcribe -> Review)
|
|
|
|
### Related Requirements
|
|
|
|
- REQ-0, REQ-1, REQ-2, REQ-3, REQ-4, REQ-5, REQ-6, REQ-12
|
|
|
|
### Goals
|
|
|
|
- guarantee end-to-end reliability and usability of the primary user flow
|
|
- ensure review experience supports transcript acceptance and correction
|
|
|
|
### Tasks
|
|
|
|
1. Validate and close any lifecycle-state gaps:
|
|
- enforce valid transitions (`queued -> processing -> transcribed/failed/completed`)
|
|
- ensure transition visibility in UI/API
|
|
2. Review experience completion:
|
|
- transcript detail display stability
|
|
- failure detail readability and actionability
|
|
- acceptance/edit path for human review
|
|
3. Ensure prompt-asset integration remains file-based and auditable:
|
|
- one prompt per Markdown file
|
|
- prompt selection/usage traceability in job outcomes (if available in model)
|
|
4. Confirm worker/UI interactions remain responsive under long-running jobs:
|
|
- in-flight guards
|
|
- clear status refresh behavior
|
|
|
|
### Deliverables
|
|
|
|
- complete end-user flow behavior with stable lifecycle visibility
|
|
- test coverage for happy path and failure path
|
|
|
|
### Exit Criteria
|
|
|
|
- user can run upload -> process -> review reliably
|
|
- failed and successful outcomes are both actionable and traceable
|
|
|
|
---
|
|
|
|
## Workstream C — Revision History and Provenance Completion
|
|
|
|
### Related Requirements
|
|
|
|
- REQ-3, REQ-4, REQ-5, REQ-11
|
|
|
|
### Goals
|
|
|
|
- finalize immutable transcript revision behavior and provenance consistency
|
|
|
|
### Tasks
|
|
|
|
1. Define/confirm revision invariants:
|
|
- append-only revision history
|
|
- clear current/accepted revision indicator
|
|
2. Persist revision events consistently through service layer boundaries.
|
|
3. Ensure UI/API expose revision timeline and selected revision details.
|
|
4. Align error handling for revision conflicts and missing resources.
|
|
|
|
### Deliverables
|
|
|
|
- revision-history feature completeness
|
|
- provenance and history read-path coverage
|
|
|
|
### Exit Criteria
|
|
|
|
- transcript edits produce deterministic revision records
|
|
- previous revisions remain inspectable
|
|
|
|
---
|
|
|
|
## Workstream D — Search Completion (Accepted Transcript Scope)
|
|
|
|
### Related Requirements
|
|
|
|
- REQ-0, REQ-5, REQ-11
|
|
|
|
### Goals
|
|
|
|
- provide practical search over accepted transcripts for personal corpus usage
|
|
|
|
### Tasks
|
|
|
|
1. Finalize searchable scope and indexing rules (accepted/current text only).
|
|
2. Implement service-backed search query behavior.
|
|
3. Expose search in UI/API with clear result metadata (document/job/revision context).
|
|
4. Add guardrails for empty/no-result/error scenarios with actionable messaging.
|
|
|
|
### Deliverables
|
|
|
|
- functional search pathway with deterministic results for accepted text
|
|
|
|
### Exit Criteria
|
|
|
|
- operator can find transcripts reliably by text queries
|
|
- no-result and error states are clear and non-silent
|
|
|
|
---
|
|
|
|
## Workstream E — Export Completion
|
|
|
|
### Related Requirements
|
|
|
|
- REQ-0, REQ-4, REQ-5, REQ-11
|
|
|
|
### Goals
|
|
|
|
- deliver practical export of transcript data for personal archive use
|
|
|
|
### Tasks
|
|
|
|
1. Finalize export contract (format, included fields, scope filters).
|
|
2. Implement export service with deterministic data mapping.
|
|
3. Add UI/API trigger path and user-visible completion/failure feedback.
|
|
4. Validate export integrity against persisted source-of-record entities.
|
|
|
|
### Deliverables
|
|
|
|
- end-to-end export capability with operator-visible outcomes
|
|
|
|
### Exit Criteria
|
|
|
|
- export output is complete, consistent, and usable for downstream personal archive workflows
|
|
|
|
---
|
|
|
|
## Workstream F — API/UI Parity and Interaction Hardening
|
|
|
|
### Related Requirements
|
|
|
|
- REQ-5 plus cross-cutting REQ-2/3/4
|
|
|
|
### Goals
|
|
|
|
- ensure UI and API expose coherent feature behavior and error contracts
|
|
|
|
### Tasks
|
|
|
|
1. Verify API/UI parity matrix for each Step 3 slice.
|
|
2. Standardize interaction behavior:
|
|
- loading and in-flight states
|
|
- success/failure notifications
|
|
- stable error_id visibility where user-facing
|
|
3. Ensure route/page modules remain composition-focused (business logic in services).
|
|
|
|
### Deliverables
|
|
|
|
- API/UI parity checklist with resolved gaps
|
|
|
|
### Exit Criteria
|
|
|
|
- no major flow exists in one interface with conflicting semantics in the other
|
|
|
|
---
|
|
|
|
## Workstream G — Carry-Forward Integration During Step 3
|
|
|
|
### Goals
|
|
|
|
- close Step 1/2 follow-ups that are Step 3-owned
|
|
|
|
### Tasks
|
|
|
|
1. Update checklist item CF-A1 as Step 3 slices touch runtime resources.
|
|
2. Update checklist item CF-A3 with lightweight boundary enforcement evidence.
|
|
3. Update CF-C1/CF-C2 traceability mapping with Step 3 outcomes.
|
|
|
|
### Deliverables
|
|
|
|
- updated `docs/ver1/ver1-step1-2-carry-forward-checklist.md` evidence entries
|
|
|
|
### Exit Criteria
|
|
|
|
- Step 3-owned carry-forward items are either completed or explicitly routed with evidence
|
|
|
|
---
|
|
|
|
## Test and Validation Plan
|
|
|
|
Apply `pytesting` guidance with deterministic, behavior-focused coverage.
|
|
|
|
### Validation Lanes
|
|
|
|
1. Structure/collection:
|
|
- `uv run pytest --collect-only -q`
|
|
2. Fast feedback lane:
|
|
- `uv run pytest -m unit -q`
|
|
3. Main verification lane:
|
|
- `uv run pytest -m "not external" -q`
|
|
4. Full suite:
|
|
- `uv run pytest -q`
|
|
|
|
### Required Coverage Areas
|
|
|
|
- lifecycle transition invariants
|
|
- revision history invariants
|
|
- search query behavior and result mapping
|
|
- export integrity and failure handling
|
|
- UI interaction guards and actionable failure feedback
|
|
- API envelope and status consistency for new/changed flows
|
|
|
|
### Test Design Rules
|
|
|
|
- one behavior target per test
|
|
- minimize heavy mocking; prefer real-path behavior checks where practical
|
|
- keep markers explicit and strict
|
|
|
|
---
|
|
|
|
## Logging, Error, and Config Guardrails for Step 3 Changes
|
|
|
|
1. Logging
|
|
- keep centralized startup logging config (`dictConfig`) as canonical
|
|
- include required error fields at boundary failures (`error_id`, `category`, `operation`, identifiers where available)
|
|
|
|
2. Error handling
|
|
- preserve taxonomy stability from `docs/error_handling.md`
|
|
- map any new failure pathways into existing categories
|
|
- surface actionable suggestions in UI/API
|
|
|
|
3. Configuration
|
|
- use typed settings and avoid ad-hoc env reads in business modules
|
|
- keep environment behavior explicit and documented
|
|
|
|
---
|
|
|
|
## Implementation Order (Detailed)
|
|
|
|
1. Workstream A: audit and backlog lock
|
|
2. Workstream B: core flow completion
|
|
3. Workstream C: revision/provenance completion
|
|
4. Workstream D: search completion
|
|
5. Workstream E: export completion
|
|
6. Workstream F: API/UI parity hardening
|
|
7. Workstream G: carry-forward integration updates
|
|
8. Full validation pass + docs/traceability updates
|
|
|
|
---
|
|
|
|
## Deliverables
|
|
|
|
1. Step 3 requirement-slice matrix with REQ mapping and evidence links
|
|
2. implemented Step 3 functional slices across service/persistence/API/UI
|
|
3. updated tests and passing validation lanes
|
|
4. updated carry-forward checklist entries (`CF-A1`, `CF-A3`, `CF-C1`, `CF-C2` as applicable)
|
|
5. Step 3 results document (`docs/ver1/ver1-step3-results.md`)
|
|
|
|
---
|
|
|
|
## Risks and Mitigations
|
|
|
|
1. **Risk:** Scope creep from optional enhancements during feature completion
|
|
- **Mitigation:** enforce REQ-mapped slice backlog and defer non-REQ enhancements
|
|
|
|
2. **Risk:** Functional parity drift between UI and API
|
|
- **Mitigation:** maintain parity matrix and verify both surfaces per slice
|
|
|
|
3. **Risk:** Data-model changes introduce migration surprises
|
|
- **Mitigation:** coordinate with Step 4 runbook expectations early and test on representative data
|
|
|
|
4. **Risk:** Reliability regressions while adding functionality
|
|
- **Mitigation:** run full error-path regression checks and keep Step 2 contracts intact
|
|
|
|
---
|
|
|
|
## Step 3 Completion Checklist
|
|
|
|
- [ ] Step 3 requirement-slice matrix completed and linked to REQ IDs.
|
|
- [ ] Core end-user flow is functionally complete and verified.
|
|
- [ ] Revision history/provenance behavior is complete and test-covered.
|
|
- [ ] Search over accepted transcripts is complete and test-covered.
|
|
- [ ] Export flow is complete and test-covered.
|
|
- [ ] API/UI parity checklist has no unresolved high-impact gaps.
|
|
- [ ] Step 3-owned carry-forward items are updated with evidence.
|
|
- [ ] Validation lanes pass (`collect-only`, unit, non-external, full).
|
|
- [ ] `docs/ver1/ver1-step3-results.md` is created with evidence and residual follow-ups.
|
|
|
|
---
|
|
|
|
## Handoff to Step 4
|
|
|
|
Step 3 completion enables Step 4 (Data Model and Migration Safety) with:
|
|
|
|
- finalized functional domain behavior
|
|
- stable persistence expectations
|
|
- traceable requirement evidence
|
|
- clarified migration-impact surface
|