ver1 - Step 5 implementation plan

2026-06-26 15:25:57 -05:00
parent dd80cd60cf
commit c9682b0399
2 changed files with 579 additions and 0 deletions
@@ -0,0 +1,120 @@
+# Ver1 Step 5 Results: Private-Network Safety Baseline
+
+## Summary
+
+Step 5 implementation status: **in progress**.
+
+This document records completed private-network safety controls, validation evidence, and residual risks for Ver1 Step 5.
+
+Implemented in this step:
+
+1. _TBD_
+2. _TBD_
+3. _TBD_
+
+---
+
+## Implemented Changes
+
+### 1) Security assumptions and threat model
+
+_TBD_
+
+### 2) Single-operator access control baseline
+
+_TBD_
+
+### 3) Input validation and safe-output hardening
+
+_TBD_
+
+### 4) Secret handling and configuration safety
+
+_TBD_
+
+### 5) Dependency/security scanning baseline
+
+_TBD_
+
+---
+
+## Test and Verification Evidence
+
+### Added/Updated Tests
+
+1. _TBD_
+2. _TBD_
+3. _TBD_
+
+### Validation Runs
+
+Run and record outcomes:
+
+- `uv run pytest --collect-only -q` -> _TBD_
+- `uv run pytest -m unit -q` -> _TBD_
+- `uv run pytest -m "not external" -q` -> _TBD_
+- `uv run pytest -q` -> _TBD_
+
+### Security Scan Evidence
+
+Record scan commands and outcomes:
+
+- dependency scan command(s): _TBD_
+- static/security lint command(s): _TBD_
+- critical/high findings: _TBD_
+- remediation/defer decisions: _TBD_
+
+---
+
+## Requirement Traceability (Step 5)
+
+| Step 5 Area | REQ Coverage | Status | Evidence |
+| --- | --- | --- | --- |
+| Private-network and single-operator safety posture | REQ-9 | _TBD_ | _TBD_ |
+| Access control behavior at UI/API boundaries | REQ-5, REQ-7 | _TBD_ | _TBD_ |
+| Input validation and safe user-facing error behavior | REQ-1, REQ-2, REQ-5 | _TBD_ | _TBD_ |
+| Config and startup safety controls | REQ-8, REQ-10 | _TBD_ | _TBD_ |
+| Persistence and domain integrity continuity | REQ-11, REQ-12 | _TBD_ | _TBD_ |
+
+---
+
+## Operational Artifacts Produced
+
+- `docs/ver1/ver1-step5.md`
+- `docs/ver1/ver1-step5-results.md`
+- _TBD additional artifacts_
+
+---
+
+## Risks, Exceptions, and Follow-Ups
+
+1. _TBD_
+2. _TBD_
+3. _TBD_
+
+Open follow-ups to carry forward:
+
+- _TBD_
+
+---
+
+## Step 5 Exit Assessment
+
+- Private-network assumptions and controls: **_TBD_**
+- Access-control baseline effectiveness: **_TBD_**
+- Validation and safe-output safety: **_TBD_**
+- Secret handling and config safety: **_TBD_**
+- Dependency/security risk closure: **_TBD_**
+- Test and regression safety: **_TBD_**
+
+Step 5 completion status: **_TBD_**
+
+---
+
+## Handoff to Step 6
+
+Once Step 5 is marked complete, Step 6 can proceed with:
+
+- clearer operational security assumptions for logs/runbooks
+- hardened boundary behavior for diagnosis and support
+- reduced risk posture for personal-scale ongoing operations
@@ -0,0 +1,459 @@
+# Step 5 Implementation Plan: Private-Network Safety Baseline
+
+## Purpose
+
+Implement **Ver1 Step 5** from `docs/ver1/ver1.md` by applying right-sized security controls for a single-user system running on a trusted private network.
+
+Step 5 focuses on practical risk reduction without introducing unnecessary complexity, while preserving:
+
+- personal-scale operational simplicity
+- single-operator workflow
+- explicit boundary ownership from `docs/architecture.md`
+- safety and diagnostics behavior defined in `docs/error_handling.md`
+
+Primary governing docs:
+
+- `docs/ver1/ver1.md` (Step 5 objective and sequencing)
+- `docs/architecture.md` (deployment model and module boundaries)
+- `docs/error_handling.md` (safe user output and diagnostic boundaries)
+- `docs/requirements.md` (REQ-1, REQ-2, REQ-5, REQ-7, REQ-8, REQ-9, REQ-10, REQ-11, REQ-12)
+- `docs/intent.md` (domain integrity priorities)
+
+---
+
+## MCP Resources Reviewed and Applied
+
+All currently available resources on `john-stream-mcp` were reviewed. Step 5 applies the following guidance directly:
+
+1. `resource://skills/pydantic-settings/document`
+   - typed security-related runtime settings
+   - explicit env/source precedence
+   - fail-fast handling for missing/invalid required values
+
+2. `resource://skills/fastapi-uv-docker/document`
+   - environment and deployment safety defaults
+   - startup/health posture and container hygiene assumptions
+   - local secret handling expectations
+
+3. `resource://skills/pytesting/document`
+   - deterministic security-behavior test lanes
+   - marker discipline and behavior-first assertions
+
+4. `resource://skills/python-logging-dictconfig/document`
+   - centralized logging discipline
+   - avoid leaking sensitive values in logs
+
+5. `resource://skills/nicegui-ui-customization/document`
+   - user-safe failure messaging in UI
+   - resilient interaction behavior and clear error feedback
+
+6. `resource://skills/ruff-linting-formating/document`
+   - keep lint quality baseline stable during safety changes
+
+Planning methodology input:
+
+7. `resource://prompts/greenfield-architecture/document`
+   - explicit tradeoff-oriented staging
+   - scope discipline for minimally sufficient security controls
+
+Reviewed but not directly Step 5 execution-critical:
+
+- skills: `copilot-customization`, `fastapi-async-sqlalchemy-modernization`, `mcp-details`, `nicegui`, `python-typing`, `vscode-configuration`, `zensical-docs`
+- prompts: `authoring`, `mcp-consumer-repo-shim`, `pytest-scaffold`, `pytest-fill-scaffold`
+
+---
+
+## Current-State Gap Summary (Step 5 Scope)
+
+Based on current implementation and prior Step outputs:
+
+1. **Private-network assumptions are implicit, not fully codified**
+   - Need explicit, documented security posture and operator constraints.
+
+2. **Access control for UI/API is minimal or absent**
+   - Step 5 requires basic single-operator gating appropriate for private-network use.
+
+3. **Input validation baseline exists but needs security-oriented audit closure**
+   - Upload and API validation should be verified for abuse-resistant boundaries.
+
+4. **Safe error output baseline exists (Step 2), but needs security confirmation pass**
+   - Must ensure no sensitive internals leak through API/UI error payloads.
+
+5. **Secret handling documentation needs formalization in Step 5 artifacts**
+   - Local workflow should clearly prohibit secrets in repo-tracked files and logs.
+
+6. **Dependency/security scanning is not yet formalized as a recurring gate**
+   - Step 5 requires lightweight scanning and triage of high-risk findings.
+
+---
+
+## Scope for Step 5
+
+### In scope
+
+1. Codify private-network and single-operator security assumptions in docs and config.
+2. Add basic access control for UI/API actions (right-sized for trusted network model).
+3. Audit and harden input-validation boundaries (upload, API params/payloads, operational flags).
+4. Verify safe error surface behavior (UI/API) and prevent sensitive leak paths.
+5. Formalize local secret handling policy and usage examples.
+6. Add lightweight dependency/security scan workflow and triage policy.
+7. Add Step 5 verification tests and results artifact.
+
+### Out of scope
+
+- Internet-facing zero-trust security architecture
+- Enterprise IAM/SSO/role systems
+- Full cryptographic key-management infrastructure
+- Major security product integrations beyond lightweight V1 needs
+
+---
+
+## Target Decisions for Step 5
+
+1. **Threat model is explicitly private-network + single operator**
+   - Security controls are right-sized to this posture and documented as assumptions.
+
+2. **Access control is required, even in private network mode**
+   - Basic gate (single shared operator credential/token) protects UI/API mutation paths.
+
+3. **Validation and output safety are strict defaults**
+   - Reject invalid inputs early; never expose sensitive internals in user-facing outputs.
+
+4. **Secrets are runtime-only**
+   - No secrets committed to source control; no plaintext secret logging.
+
+5. **Security scanning is lightweight but mandatory**
+   - Add recurring dependency/security checks with high-risk triage and closure workflow.
+
+6. **No security control may violate Step 1–4 operational simplicity guardrails**
+   - Preserve deployability and maintainability for personal-scale use.
+
+---
+
+## Detailed Work Breakdown
+
+## Phase A — Security Posture Definition and Gap Lock
+
+- [ ] **A1. Define Step 5 threat model**
+  - trusted private network
+  - single operator
+  - local deployment assumptions
+  - explicit out-of-scope threat classes
+
+- [ ] **A2. Produce security baseline checklist**
+  - access control
+  - validation boundaries
+  - safe error behavior
+  - secret handling
+  - dependency risk checks
+
+- [ ] **A3. Map controls to architecture boundaries**
+  - UI
+  - API
+  - service
+  - config/runtime
+  - operator runbooks
+
+### Deliverables
+
+- `docs/ver1/ver1-step5-security-assumptions.md` (recommended)
+- Step 5 control matrix (control -> owner -> validation method)
+
+### Exit Criteria
+
+- private-network safety posture is explicit and approved
+- each in-scope control has boundary ownership and verification path
+
+---
+
+## Phase B — Basic Single-Operator Access Control
+
+- [ ] **B1. Select access mechanism**
+  - minimal approach suitable for private-network model
+  - explicitly document tradeoffs and operator ergonomics
+
+- [ ] **B2. Protect mutating operations first**
+  - upload/create/accept/export-trigger endpoints
+  - UI actions that trigger persistence changes
+
+- [ ] **B3. Protect read operations as policy requires**
+  - determine read-path gating expectations and apply consistently
+
+- [ ] **B4. Add clear unauthorized behavior contract**
+  - stable API status and safe message
+  - UI feedback with actionable operator guidance
+
+### Deliverables
+
+- access-control policy and implementation notes
+- unauthorized behavior matrix (UI/API)
+
+### Exit Criteria
+
+- unauthorized actions are blocked consistently
+- authorized operator flows remain usable and deterministic
+
+---
+
+## Phase C — Input Validation and Safe Output Hardening
+
+- [ ] **C1. Validation audit for all entry points**
+  - file uploads (type/size/content guards)
+  - route/query/body constraints
+  - service-layer invariants
+
+- [ ] **C2. Normalize validation failures to canonical taxonomy**
+  - `validation_error` vs `user_input_error` consistency
+
+- [ ] **C3. Confirm safe error output policy under security lens**
+  - no stack traces/secrets/internal paths in UI/API default outputs
+  - preserve error reference IDs for traceability
+
+- [ ] **C4. Add abuse-resistant guardrails where practical**
+  - basic request-size and payload-shape constraints
+  - anti-duplication interaction safeguards (where missing)
+
+### Deliverables
+
+- validation-path inventory and hardening checklist
+- safe-output verification notes
+
+### Exit Criteria
+
+- input boundaries are deterministic and tested
+- user-facing error outputs remain safe and actionable
+
+---
+
+## Phase D — Secrets Handling and Configuration Safety
+
+- [ ] **D1. Define secret handling policy**
+  - where secrets are allowed (runtime env only)
+  - where secrets are prohibited (source files, docs examples beyond placeholders)
+
+- [ ] **D2. Enforce settings expectations**
+  - required secret fields fail fast
+  - avoid fallback defaults that silently weaken safety
+
+- [ ] **D3. Add operator documentation for local secret workflow**
+  - how to set environment values safely
+  - how to rotate/update credentials locally
+
+- [ ] **D4. Validate logging does not leak secret values**
+  - startup/config logs
+  - error logs for provider/config failures
+
+### Deliverables
+
+- secret-handling section in runbook/README/docs
+- settings and logging safety verification notes
+
+### Exit Criteria
+
+- no secret leakage paths remain in normal operations
+- operator can configure secrets safely using docs only
+
+---
+
+## Phase E — Dependency and Security Scanning Baseline
+
+- [ ] **E1. Select lightweight scanning commands for V1**
+  - dependency vulnerability scan
+  - optional static security scan if practical
+
+- [ ] **E2. Define triage policy for findings**
+  - severity classification
+  - required closure criteria for Step 5 completion
+
+- [ ] **E3. Run scans and capture evidence**
+  - record command outputs/summaries
+  - remediate or formally defer with risk notes
+
+- [ ] **E4. Add recurring execution guidance**
+  - local pre-release checklist integration
+  - future CI gate handoff for Step 7/9
+
+### Deliverables
+
+- Step 5 scan report artifact (recommended)
+- triage log of resolved/deferred findings
+
+### Exit Criteria
+
+- no unresolved critical vulnerabilities in Step 5 scope
+- high-risk findings are resolved or explicitly risk-accepted with rationale
+
+---
+
+## Phase F — Verification and Test Expansion
+
+Apply `pytesting` guidance (deterministic, behavior-first, strict markers).
+
+- [ ] **F1. Access-control tests**
+  - unauthorized requests are rejected as expected
+  - authorized operator requests succeed
+
+- [ ] **F2. Validation and abuse-boundary tests**
+  - invalid payloads rejected with stable category/status
+  - file-type/size constraints enforced
+
+- [ ] **F3. Safe-output tests**
+  - API/UI error responses avoid sensitive details
+  - error IDs and suggestions remain present
+
+- [ ] **F4. Config/secret safety tests**
+  - required secrets fail fast when missing
+  - no unsafe fallback behavior introduced
+
+### Validation Commands
+
+- `uv run pytest --collect-only -q`
+- `uv run pytest -m unit -q`
+- `uv run pytest -m "not external" -q`
+- `uv run pytest -q`
+
+### Exit Criteria
+
+- Step 5 safety behavior is test-covered and passing
+- no regression in core upload/transcribe/review workflows
+
+---
+
+## Phase G — Documentation and Risk Closure
+
+- [ ] **G1. Create Step 5 results artifact**
+  - `docs/ver1/ver1-step5-results.md`
+
+- [ ] **G2. Update operator-facing docs**
+  - security assumptions and local deployment cautions
+  - credential handling and recovery basics
+
+- [ ] **G3. Update traceability and carry-forward notes**
+  - map Step 5 controls to REQ and evidence
+
+### Deliverables
+
+- `docs/ver1/ver1-step5-results.md`
+- updated security assumptions checklist and risk summary
+
+### Exit Criteria
+
+- Step 5 controls and residual risks are fully documented
+- handoff is ready for Step 6 observability and Step 7 quality gates
+
+---
+
+## Recommended Implementation Order
+
+1. Phase A — posture definition and gap lock
+2. Phase B — access control baseline
+3. Phase C — validation/output hardening
+4. Phase D — secrets and config safety
+5. Phase E — dependency/security scan baseline
+6. Phase F — test expansion and verification
+7. Phase G — docs and risk closure
+
+This order reduces risk by locking assumptions first, then applying controls at highest-impact boundaries before final verification and documentation.
+
+---
+
+## Step 5 Execution Checklist (Phase-by-Phase)
+
+Use this checklist to execute Step 5 in implementation order and record progress/evidence.
+
+### Phase A — Security Posture Definition and Gap Lock
+
+- [ ] Publish `docs/ver1/ver1-step5-security-assumptions.md`.
+- [ ] Record explicit in-scope and out-of-scope threat classes.
+- [ ] Produce Step 5 control matrix (control, owner, validation method).
+- [ ] Confirm boundary ownership for each control (UI/API/service/config/docs).
+
+### Phase B — Basic Single-Operator Access Control
+
+- [ ] Choose and document access mechanism (with rationale and tradeoffs).
+- [ ] Implement enforcement for mutating API operations.
+- [ ] Implement corresponding UI-side access behavior for protected actions.
+- [ ] Decide and enforce read-path protection policy.
+- [ ] Add unauthorized API/UI contract tests.
+
+### Phase C — Input Validation and Safe Output Hardening
+
+- [ ] Complete input-validation inventory for upload/API/service boundaries.
+- [ ] Tighten payload/file constraints where gaps are found.
+- [ ] Ensure validation failure categories match `docs/error_handling.md`.
+- [ ] Verify user-facing errors remain safe, actionable, and traceable.
+- [ ] Add regression tests for invalid/boundary inputs.
+
+### Phase D — Secrets Handling and Configuration Safety
+
+- [ ] Document secrets policy (runtime-only, no repo storage).
+- [ ] Verify required secret settings fail fast when missing.
+- [ ] Audit logs for accidental secret leakage risk paths.
+- [ ] Update operator docs for local secret setup/rotation workflow.
+- [ ] Add tests for config safety expectations where practical.
+
+### Phase E — Dependency and Security Scanning Baseline
+
+- [ ] Select scanning commands and record tool versions.
+- [ ] Run baseline scans and capture outputs.
+- [ ] Triage findings by severity and exploitability in private-network context.
+- [ ] Resolve/mitigate critical findings; document accepted residual risk.
+- [ ] Add recurring scan guidance for release workflow handoff.
+
+### Phase F — Verification and Test Expansion
+
+- [ ] Run `uv run pytest --collect-only -q`.
+- [ ] Run `uv run pytest -m unit -q`.
+- [ ] Run `uv run pytest -m "not external" -q`.
+- [ ] Run `uv run pytest -q`.
+- [ ] Confirm no regressions in upload/transcribe/review core flows.
+
+### Phase G — Documentation and Risk Closure
+
+- [ ] Complete `docs/ver1/ver1-step5-results.md` with evidence.
+- [ ] Update docs/README/runbooks with final Step 5 security posture.
+- [ ] Record REQ traceability updates and residual risks.
+- [ ] Confirm Step 5 completion checklist items are all closed.
+
+---
+
+## Risks and Mitigations
+
+1. **Risk:** Over-engineering beyond private-network needs
+   - **Mitigation:** enforce Step 5 scope discipline and threat-model constraints.
+
+2. **Risk:** Access controls disrupt operator usability
+   - **Mitigation:** keep mechanism minimal and test primary workflows thoroughly.
+
+3. **Risk:** Sensitive details leak through errors/logging
+   - **Mitigation:** apply safe-output and log-sanitization checks with tests.
+
+4. **Risk:** Unpatched dependency vulnerabilities remain invisible
+   - **Mitigation:** formalize scan + triage + evidence capture workflow.
+
+5. **Risk:** Secret handling remains ad hoc
+   - **Mitigation:** fail-fast settings + explicit operator documentation + review checks.
+
+---
+
+## Step 5 Completion Checklist
+
+- [ ] Private-network and single-operator security assumptions are documented.
+- [ ] Basic single-operator access control is implemented and verified.
+- [ ] Input-validation boundaries are audited, hardened, and test-covered.
+- [ ] UI/API error output safety is confirmed under security tests.
+- [ ] Secret handling policy and local workflow docs are complete.
+- [ ] Dependency/security scans are run; critical findings are resolved.
+- [ ] Step 5 tests pass across all validation lanes.
+- [ ] `docs/ver1/ver1-step5-results.md` is completed with evidence and residual risks.
+
+---
+
+## Handoff to Step 6
+
+Step 5 completion enables Step 6 (Minimal Observability & Operability) with:
+
+- explicit security assumptions for operator context
+- access and validation controls suitable for private-network operation
+- safer runtime/configuration handling for ongoing operations
+- dependency-risk visibility feeding release-readiness gates