transcription/docs/ver1/ver1-step6.md

Step 6 Goal (from docs/ver1/ver1.md)

Implement minimal observability & operability so a single operator can quickly diagnose and recover from common failures.

---

1) Current-State Assessment (what already exists)

Already in place

• Central startup logging initialization via setup_logging() and dictConfig (src/transcription/config.py, src/transcription/app.py).
• Error taxonomy and error_id envelope contract (src/transcription/errors.py) aligned with docs/error_handling.md.
• Error handling for API and worker includes category + error IDs in some paths (src/transcription/api/errors.py, src/transcription/worker.py).
• Basic health endpoint /healthz (src/transcription/api/health.py).
• UI error display already shows actionable message + error reference (src/transcription/ui/error_presenter.py).

Gaps to close for Step 6

1. Structured logging is inconsistent (many logs are free-form text with embedded key/value; no enforced schema).
2. Boundary coverage is incomplete (UI/service/API/worker don’t all emit consistent operation logs).
3. /healthz is very basic; no lightweight readiness/startup diagnostics endpoint/reporting.
4. No concise operator runbook yet (start/stop, log interpretation, recovery playbooks).
5. Minimal counters/timings are not yet standardized.

---

2) MCP Guidance Incorporated (relevant items)

From john-stream-mcp, these are directly applied:

• python-logging-dictconfig: keep one centralized dictConfig, configure once at startup, named loggers in modules.
• fastapi-async-sqlalchemy-modernization: include observability + health/readiness checks; explicit lifecycle and deterministic startup/shutdown checks.
• fastapi-uv-docker: keep /healthz; add practical readiness/ops checks for deployment clarity.
• pytesting: deterministic tests, concise structure, validation lanes (collect-only, unit, not external, full).
• pydantic-settings: keep typed settings as single source for logging/health behavior flags.
• nicegui + nicegui-ui-customization: preserve clear, actionable user-facing error feedback and non-blocking UI flows.
• zensical-docs: produce focused, navigable operator docs.

(Other MCP resources were reviewed but are not core to Step 6 implementation scope.)

---

3) Detailed Implementation Plan for Step 6

Workstream A — Structured Logging Contract

A1. Define a canonical log event schema

Create a project log schema (doc + code-level constants) with required keys:

• timestamp (UTC)
• level
• logger
• operation
• event
• error_id (when error)
• category (when error)
• exception_type (when error)
• job_id, document_id (when relevant)
• optional: duration_ms, retry_count, status

A2. Standardize log emission helpers

Add small logging helpers (or adapter utilities) to reduce drift:

• log_operation_start(...)
• log_operation_success(...)
• log_operation_error(...)

Keep this minimal and avoid heavy observability frameworks.

A3. Update formatter to structured output

Use dictConfig to emit either:

• JSON lines (preferred for structure), or
• strict key-value line format with fixed fields.

Recommendation: JSON lines to satisfy “structured logging” unambiguously while still simple.

---

Workstream B — Boundary-by-Boundary Instrumentation

B1. API boundary (src/transcription/api/*)

• Add request-level operation logs for key routes (upload.submit, jobs.list, jobs.get, etc.).
• Ensure API exception handler logs always include error_id, category, operation, exception_type.

B2. Service boundary (src/transcription/services/*)

• Add operation logs around:
  • upload validation/persist,
  • transcription orchestration,
  • revision add/accept,
  • search/export.
• Add timing (duration_ms) for high-value operations only.

B3. Worker boundary (src/transcription/worker.py)

• Standardize all worker log events to schema.
• Ensure retry logs include: retriable, retry_count, max_retries, backoff_seconds.
• Ensure terminal failure logs include error contract fields.

B4. UI boundary (src/transcription/ui/*)

• Keep user-safe UI messages as-is.
• Add backend/UI logger events for user-triggered failures (operation + error_id + category) so UI-visible errors correlate to server logs.

---

Workstream C — Health, Readiness, Startup Operability

C1. Keep /healthz lightweight

• Return “process is running” status quickly.

C2. Add lightweight /readyz

Include small checks:

• DB connectivity ping.
• Worker thread alive check.
• Optional prompt directory existence check.

Return structured status payload with per-check pass/fail.

C3. Startup self-check summary log

At startup, emit one concise ops summary event:

• environment
• schema validation result
• worker started
• directories checked
• bootstrap/migration mode flags

---

Workstream D — Minimal Counters & Timings

Add only high-value diagnostics:

1. worker_jobs_processed_total
2. worker_jobs_failed_total
3. worker_retries_total
4. transcription_duration_ms (per job)
5. upload_persist_duration_ms (per upload path)

Implementation can be log-derived counters (no external metrics backend required).

---

Workstream E — Operator Runbook

Create concise runbook doc (recommended: docs/ver1/ver1-step6-operator-runbook.md) with:

1. Start/Stop

   • local uv run mode
   • docker compose mode (if applicable)

2. Where logs are

   • stdout, docker logs commands, filtering by error_id / operation.

3. Common failure patterns → recovery

   • provider timeout
   • auth denied
   • missing prompt dir
   • DB unavailable
   • job stuck/failed with retry exhausted

4. Recovery procedures

   • restart sequence
   • verify health/readiness
   • when to requeue/re-upload

5. Escalation artifacts

   • capture timestamp + error_id + operation + job_id/document_id

Also update README.md with short links to the runbook.

---

Workstream F — Verification & Quality Gates

Tests to add/update

• tests/api/test_health.py
  • /healthz baseline
  • /readyz pass/fail behavior
• tests/api/test_error_responses.py / tests/api/test_routes.py
  • logs include error_id/category/operation on failures
• tests/services/test_worker.py
  • retry/failure log fields + timing presence
• tests/ui/*
  • ensure UI error correlation path includes operation/ref id behavior

Validation commands (per MCP pytest guidance)

• uv run pytest --collect-only -q
• uv run pytest -m unit -q
• uv run pytest -m "not external" -q
• uv run pytest -q

---

4) Traceability to Governing Docs

• docs/ver1/ver1.md Step 6: all 5 implementation bullets covered.
• docs/error_handling.md: logging contract fields and error taxonomy continuity enforced.
• docs/architecture.md: respects modular boundaries, in-process worker model, low-complexity ops.
• docs/requirements.md:
  • REQ-8 (startup logging/config centralization) strengthened,
  • REQ-5 (status visibility) improved operationally,
  • REQ-7 lifecycle ownership observability improved.
• docs/intent.md: keeps operation simple for personal-scale archival workflow.

---

5) Suggested Execution Order (low risk)

1. Logging schema + formatter + helpers
2. Worker/API instrumentation (highest value)
3. Service/UI instrumentation
4. /readyz + startup summary check
5. Runbook + README links
6. Tests + Step 6 results artifact (docs/ver1/ver1-step6-results.md)