diff --git a/docs/index.md b/docs/index.md index f8c4be4..52fd35f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -6,7 +6,7 @@ This project is a production application for transcribing and preserving histori Read [architecture.md](architecture.md) first. -Then review [ver1/ver1.md](ver1/ver1.md) for completion scope and [ver1/ver1-step1-results.md](ver1/ver1-step1-results.md) for current architecture-consolidation status. +Then review [ver1/ver1.md](ver1/ver1.md) for completion scope. The architecture page is the primary technical reference and defines: @@ -41,10 +41,8 @@ This operating model keeps deployment and maintenance simple while preserving cl ## Documentation Map -- Architecture and technical design: [architecture.md](architecture.md) - Version 1 implementation plan: [ver1/ver1.md](ver1/ver1.md) -- Version 1 Step 1 plan: [ver1/ver1-step1.md](ver1/ver1-step1.md) -- Version 1 Step 1 results: [ver1/ver1-step1-results.md](ver1/ver1-step1-results.md) +- Architecture and technical design: [architecture.md](architecture.md) - Architecture decision records (ADR index): [adr/README.md](adr/README.md) - Runtime and deployment requirements: [requirements.md](requirements.md) - Error handling policy and operational guidance: [error_handling.md](error_handling.md) diff --git a/docs/ver1/ver1-step6.md b/docs/ver1/ver1-step6.md new file mode 100644 index 0000000..381d571 --- /dev/null +++ b/docs/ver1/ver1-step6.md @@ -0,0 +1,206 @@ +## Step 6 Goal (from `docs/ver1/ver1.md`) +Implement **minimal observability & operability** so a single operator can quickly diagnose and recover from common failures. + +--- + +## 1) Current-State Assessment (what already exists) + +### Already in place +- Central startup logging initialization via `setup_logging()` and `dictConfig` (`src/transcription/config.py`, `src/transcription/app.py`). +- Error taxonomy and `error_id` envelope contract (`src/transcription/errors.py`) aligned with `docs/error_handling.md`. +- Error handling for API and worker includes category + error IDs in some paths (`src/transcription/api/errors.py`, `src/transcription/worker.py`). +- Basic health endpoint `/healthz` (`src/transcription/api/health.py`). +- UI error display already shows actionable message + error reference (`src/transcription/ui/error_presenter.py`). + +### Gaps to close for Step 6 +1. **Structured logging is inconsistent** (many logs are free-form text with embedded key/value; no enforced schema). +2. **Boundary coverage is incomplete** (UI/service/API/worker don’t all emit consistent operation logs). +3. `/healthz` is very basic; no lightweight readiness/startup diagnostics endpoint/reporting. +4. No concise **operator runbook** yet (start/stop, log interpretation, recovery playbooks). +5. Minimal counters/timings are not yet standardized. + +--- + +## 2) MCP Guidance Incorporated (relevant items) + +From `john-stream-mcp`, these are directly applied: + +- **`python-logging-dictconfig`**: keep one centralized `dictConfig`, configure once at startup, named loggers in modules. +- **`fastapi-async-sqlalchemy-modernization`**: include observability + health/readiness checks; explicit lifecycle and deterministic startup/shutdown checks. +- **`fastapi-uv-docker`**: keep `/healthz`; add practical readiness/ops checks for deployment clarity. +- **`pytesting`**: deterministic tests, concise structure, validation lanes (`collect-only`, `unit`, `not external`, full). +- **`pydantic-settings`**: keep typed settings as single source for logging/health behavior flags. +- **`nicegui` + `nicegui-ui-customization`**: preserve clear, actionable user-facing error feedback and non-blocking UI flows. +- **`zensical-docs`**: produce focused, navigable operator docs. + +(Other MCP resources were reviewed but are not core to Step 6 implementation scope.) + +--- + +## 3) Detailed Implementation Plan for Step 6 + +## Workstream A — Structured Logging Contract + +### A1. Define a canonical log event schema +Create a project log schema (doc + code-level constants) with required keys: +- `timestamp` (UTC) +- `level` +- `logger` +- `operation` +- `event` +- `error_id` (when error) +- `category` (when error) +- `exception_type` (when error) +- `job_id`, `document_id` (when relevant) +- optional: `duration_ms`, `retry_count`, `status` + +### A2. Standardize log emission helpers +Add small logging helpers (or adapter utilities) to reduce drift: +- `log_operation_start(...)` +- `log_operation_success(...)` +- `log_operation_error(...)` + +Keep this minimal and avoid heavy observability frameworks. + +### A3. Update formatter to structured output +Use `dictConfig` to emit either: +- JSON lines (preferred for structure), or +- strict key-value line format with fixed fields. + +**Recommendation:** JSON lines to satisfy “structured logging” unambiguously while still simple. + +--- + +## Workstream B — Boundary-by-Boundary Instrumentation + +### B1. API boundary (`src/transcription/api/*`) +- Add request-level operation logs for key routes (`upload.submit`, `jobs.list`, `jobs.get`, etc.). +- Ensure API exception handler logs always include `error_id`, `category`, `operation`, `exception_type`. + +### B2. Service boundary (`src/transcription/services/*`) +- Add operation logs around: + - upload validation/persist, + - transcription orchestration, + - revision add/accept, + - search/export. +- Add timing (`duration_ms`) for high-value operations only. + +### B3. Worker boundary (`src/transcription/worker.py`) +- Standardize all worker log events to schema. +- Ensure retry logs include: `retriable`, `retry_count`, `max_retries`, `backoff_seconds`. +- Ensure terminal failure logs include error contract fields. + +### B4. UI boundary (`src/transcription/ui/*`) +- Keep user-safe UI messages as-is. +- Add backend/UI logger events for user-triggered failures (operation + error_id + category) so UI-visible errors correlate to server logs. + +--- + +## Workstream C — Health, Readiness, Startup Operability + +### C1. Keep `/healthz` lightweight +- Return “process is running” status quickly. + +### C2. Add lightweight `/readyz` +Include small checks: +- DB connectivity ping. +- Worker thread alive check. +- Optional prompt directory existence check. + +Return structured status payload with per-check pass/fail. + +### C3. Startup self-check summary log +At startup, emit one concise ops summary event: +- environment +- schema validation result +- worker started +- directories checked +- bootstrap/migration mode flags + +--- + +## Workstream D — Minimal Counters & Timings + +Add only high-value diagnostics: +1. `worker_jobs_processed_total` +2. `worker_jobs_failed_total` +3. `worker_retries_total` +4. `transcription_duration_ms` (per job) +5. `upload_persist_duration_ms` (per upload path) + +Implementation can be log-derived counters (no external metrics backend required). + +--- + +## Workstream E — Operator Runbook + +Create concise runbook doc (recommended: `docs/ver1/ver1-step6-operator-runbook.md`) with: + +1. **Start/Stop** + - local `uv` run mode + - docker compose mode (if applicable) + +2. **Where logs are** + - stdout, docker logs commands, filtering by `error_id` / `operation`. + +3. **Common failure patterns → recovery** + - provider timeout + - auth denied + - missing prompt dir + - DB unavailable + - job stuck/failed with retry exhausted + +4. **Recovery procedures** + - restart sequence + - verify health/readiness + - when to requeue/re-upload + +5. **Escalation artifacts** + - capture timestamp + error_id + operation + job_id/document_id + +Also update `README.md` with short links to the runbook. + +--- + +## Workstream F — Verification & Quality Gates + +### Tests to add/update +- `tests/api/test_health.py` + - `/healthz` baseline + - `/readyz` pass/fail behavior +- `tests/api/test_error_responses.py` / `tests/api/test_routes.py` + - logs include `error_id/category/operation` on failures +- `tests/services/test_worker.py` + - retry/failure log fields + timing presence +- `tests/ui/*` + - ensure UI error correlation path includes operation/ref id behavior + +### Validation commands (per MCP pytest guidance) +- `uv run pytest --collect-only -q` +- `uv run pytest -m unit -q` +- `uv run pytest -m "not external" -q` +- `uv run pytest -q` + +--- + +## 4) Traceability to Governing Docs + +- **`docs/ver1/ver1.md` Step 6:** all 5 implementation bullets covered. +- **`docs/error_handling.md`:** logging contract fields and error taxonomy continuity enforced. +- **`docs/architecture.md`:** respects modular boundaries, in-process worker model, low-complexity ops. +- **`docs/requirements.md`:** + - REQ-8 (startup logging/config centralization) strengthened, + - REQ-5 (status visibility) improved operationally, + - REQ-7 lifecycle ownership observability improved. +- **`docs/intent.md`:** keeps operation simple for personal-scale archival workflow. + +--- + +## 5) Suggested Execution Order (low risk) + +1. Logging schema + formatter + helpers +2. Worker/API instrumentation (highest value) +3. Service/UI instrumentation +4. `/readyz` + startup summary check +5. Runbook + README links +6. Tests + Step 6 results artifact (`docs/ver1/ver1-step6-results.md`) diff --git a/uploads/10176990-fb97-4463-8065-00b57c3c8ea5_Hig's postcards to Zenna - October 24, 1924.jpg b/uploads/10176990-fb97-4463-8065-00b57c3c8ea5_Hig's postcards to Zenna - October 24, 1924.jpg new file mode 100644 index 0000000..dbcab73 Binary files /dev/null and b/uploads/10176990-fb97-4463-8065-00b57c3c8ea5_Hig's postcards to Zenna - October 24, 1924.jpg differ