20 changed files with 1867 additions and 189 deletions
@@ -0,0 +1,73 @@
# Ver1 Step 1/2 Carry-Forward Checklist
## Purpose
Track open Step 1 and Step 2 follow-ups through later V1 steps, with lightweight verification evidence and requirement traceability.
This artifact implements the carry-forward approach defined in:
- `docs/ver1/ver1-step1-2_revised.md`
Historical records remain unchanged:
- `docs/ver1/ver1-step1.md`
- `docs/ver1/ver1-step1-results.md`
- `docs/ver1/ver1-step2.md`
- `docs/ver1/ver1-step2-results.md`
---
## Status Legend
- `not started`
- `in progress`
- `done`
- `deferred`
---
## Carry-Forward Mapping Matrix
| ID | Carry-Forward Task | Source | Related REQ | Owning V1 Step(s) | Validation Method | Status | Evidence Link/Note |
| --- | --- | --- | --- | --- | --- | --- | --- |
| CF-A1 | Confirm remaining implicit/global runtime ownership and lift only high-impact resources to lifespan ownership | Step 1 residual follow-up | REQ-7 | Step 3, Step 9 | Inspection + test | in progress | Step 3 added `services/library.py` and `api/routes.py` using existing service/session access patterns; no new module-global runtime resource ownership introduced. Reconfirm in Step 9 release readiness. |
| CF-A2 | Finalize migration + rollback runbook usage and rehearse on representative local data | Step 1 residual follow-up | REQ-10 | Step 4, Step 9 | Demonstration + test | not started | |
| CF-A3 | Maintain lightweight boundary enforcement (review checklist and/or simple import checks) | Step 1 residual follow-up | REQ-7, REQ-11 | Step 3, Step 7 | Inspection | in progress | Step 3 implementation keeps UI/API composition thin and pushes revision/search/export logic to `services/library.py`; continue with Step 7 checks. |
| CF-B1 | Build compact error-path inventory for major failure paths and category mapping | Step 2 governance follow-up | REQ-2, REQ-3, REQ-4, REQ-5 | Step 6, Step 7 | Inspection | not started | Use `docs/ver1/ver1-step2-error-path-inventory.md` |
| CF-B2 | Standardize required logging fields at critical boundary handoffs | Step 2 residual follow-up | REQ-3, REQ-4, REQ-8 | Step 6 | Inspection + test | not started | |
| CF-B3 | Revisit retry backoff strategy only if observed runtime behavior justifies extra complexity | Step 2 residual follow-up | REQ-2, REQ-6 | Step 6, Step 8 | Analysis + test | deferred | Keep fixed backoff unless evidence suggests change |
| CF-C1 | Integrate Step 1/2 completed outcomes and open follow-ups into V1 traceability tracking | Revision-plan workstream | REQ-0..REQ-12 (traceability) | Step 3, Step 10 | Inspection | done | Step 3 artifacts added: `docs/ver1/ver1-step3.md`, `docs/ver1/ver1-step3-results.md`, and this checklist updated with Step 3 evidence and routing. |
| CF-C2 | Keep carry-forward routing aligned with revised V1 plan (architecture via 3/4/9, reliability via 6/7) | Revision-plan workstream | REQ-0..REQ-12 (execution alignment) | Step 3+ | Inspection | in progress | Step 3 execution followed routing: functional features implemented in Step 3; migration/rollback items remain in Step 4/9; logging/error-path standardization remains Step 6/7. |
---
## Execution Notes
### Step 3 (Functional Completion)
- Use CF-A1 and CF-A3 during requirement-slice implementation reviews.
- Record any discovered boundary/runtime ownership gaps in this checklist.
### Step 4 (Data Model and Migration Safety)
- Execute CF-A2 rehearsal and link evidence (commands, runbook notes, outcomes).
### Step 6 (Minimal Observability & Operability)
- Execute CF-B1 and CF-B2 with focused artifacts and log-field verification.
### Step 7 (Test Coverage and Practical Quality Gates)
- Add/verify tests supporting CF-A3 and CF-B1/B2 where meaningful.
### Step 8 (Performance Validation)
- Reassess CF-B3 only if retries/backoff are observed to cause practical issues.
### Step 9 (Release Readiness)
- Reconfirm CF-A1/A2 readiness in release checklist and rollback drill.
### Step 10 (Documentation Completion)
- Ensure final V1 docs reference outcomes from this checklist where relevant.
---
## Acceptance Check for Carry-Forward Completion
- [ ] Historical Step 1/2 documents remain unchanged.
- [ ] Every open Step 1/2 follow-up has an owning V1 step and validation method.
- [ ] Evidence links are recorded for each completed carry-forward item.
- [ ] No carry-forward item introduces unnecessary complexity for personal-scale operation.
+166
View File
@@ -0,0 +1,166 @@
# Ver1 Step 1 & Step 2 Revision Plan (Additive)
## Purpose
Define a **targeted implementation follow-through plan** for Step 1 and Step 2 outcomes so remaining V1 work stays aligned with `docs/ver1/ver1.md`:
- personal-scale operation
- single operator
- private-network assumptions
- low operational overhead
- practical, testable controls
This document is additive and does **not** replace or revise historical Step 1/Step 2 records.
---
## Source Documents Reviewed
- `docs/ver1/ver1.md`
- `docs/ver1/ver1-step1.md`
- `docs/ver1/ver1-step1-results.md`
- `docs/ver1/ver1-step2.md`
- `docs/ver1/ver1-step2-results.md`
- `docs/architecture.md`
- `docs/error_handling.md`
- `docs/requirements.md`
- `docs/index.md`
- `docs/intent.md`
---
## Revision Goals
1. Preserve all completed Step 1/Step 2 technical hardening work.
2. Keep historical Step 1/Step 2 documents unchanged.
3. Convert residual risks/follow-ups into concrete implementation tasks for subsequent V1 steps.
4. Preserve traceability to requirements and implemented evidence.
5. Maintain alignment with personal-scale architecture and operating model.
---
## Scope
### In Scope
- Define carry-forward implementation tasks based on Step 1/2 residual risks and open items.
- Map carry-forward tasks to later V1 steps (especially Steps 3, 4, 6, 7, and 9).
- Define lightweight verification evidence expected for each carry-forward task.
- Update V1 traceability references to include completed Step 1/2 outcomes and deferred follow-ups.
### Out of Scope
- Simplifying tone/structure of existing Step 1/2 documents
- Clarifying or rewriting historical Step 1/2 plan/results content
- Editing `docs/ver1/ver1-step1.md`
- Editing `docs/ver1/ver1-step1-results.md`
- Editing `docs/ver1/ver1-step2.md`
- Editing `docs/ver1/ver1-step2-results.md`
- Re-implementing Step 1/2 code changes
- Rewriting `docs/ver1/ver1.md`
- Deleting historical sections/results
- Altering requirements IDs or architecture principles
---
## Carry-Forward Implementation Plan
## Workstream A — Close Step 1 follow-ups through later V1 steps
### A1) Runtime ownership completion (REQ-7 continuity)
- Confirm whether any remaining runtime resources still use implicit/global ownership.
- Move only high-impact remaining resources to explicit lifespan ownership when needed.
- Keep ownership model simple and documented.
### A2) Schema/migration operations readiness (REQ-10 continuity)
- Finalize practical migration + rollback runbook usage in Step 4 execution.
- Rehearse upgrade and rollback on representative local data.
- Keep production startup free from implicit schema mutation.
### A3) Boundary enforcement (lightweight only)
- Keep architecture boundary checks lightweight (review checklist and/or simple import checks).
- Avoid heavy governance tooling unless clear recurring drift appears.
### Expected Outcome
Step 1 architecture hardening remains intact and is completed pragmatically where open items remain.
---
## Workstream B — Close Step 2 follow-ups through later V1 steps
### B1) Error-path inventory and coverage visibility
- Create a compact error-path inventory artifact (or equivalent matrix section) covering major failure paths.
- Ensure each critical path maps to category, retriable policy, and surfaced behavior.
### B2) Logging field consistency at key boundaries
- Standardize required fields at critical failure handoffs (`error_id`, `category`, `operation`, identifiers when available).
- Prioritize worker/API/service boundaries first.
### B3) Retry policy refinement (only if needed)
- Keep current bounded retry baseline.
- Revisit richer backoff strategy only if observed behavior justifies added complexity.
### Expected Outcome
Step 2 reliability behavior stays stable, diagnosable, and right-sized for personal-scale operation.
---
## Workstream C — Integrate Step 1/2 outputs into ongoing V1 governance
### C1) Traceability integration
- Link completed Step 1/2 outcomes and deferred follow-ups to the V1 traceability matrix.
- Ensure open follow-ups have owning step and validation method.
### C2) Execution alignment with revised V1 plan
- Route architecture follow-ups primarily through Steps 3/4/9.
- Route reliability/diagnostics follow-ups primarily through Steps 6/7.
### Expected Outcome
Step 1/2 work is fully carried forward without revising historical documents.
## Deliverables
1. This document (`docs/ver1/ver1-step1-2_revised.md`) as the carry-forward implementation plan.
2. A compact Step 1/2 carry-forward checklist linked to V1 steps and validation methods.
3. Traceability updates showing where each open Step 1/2 follow-up will be closed.
4. Optional new artifact for error-path inventory (if created during Step 6/7 execution).
---
## Acceptance Criteria
- Historical Step 1/Step 2 documents remain unchanged.
- Open Step 1/2 follow-ups are explicitly mapped to later V1 steps with validation expectations.
- No loss of core technical intent (REQ-7, REQ-10, error taxonomy, retry safety, traceability).
- No conflicts introduced with `docs/architecture.md`, `docs/error_handling.md`, or `docs/ver1/ver1.md`.
- Carry-forward tasks remain right-sized for personal-scale operation.
---
## Implementation Order
1. Keep existing Step 1/Step 2 docs unchanged as historical records.
2. Define carry-forward tasks and owning V1 steps in this document.
3. Create and maintain carry-forward traceability artifacts:
- `docs/ver1/ver1-step1-2-carry-forward-checklist.md`
- `docs/ver1/ver1-step2-error-path-inventory.md`
4. Execute carry-forward tasks during Steps 3+ and capture evidence in step results docs.
5. Perform final consistency pass across `docs/ver1/*` references.
---
## Risks and Mitigations
1. **Risk:** Open Step 1/2 items are forgotten as Step 3+ work proceeds.
**Mitigation:** Track each follow-up in the V1 traceability matrix with owning step and evidence expectation.
2. **Risk:** Carry-forward work expands beyond personal-scale needs.
**Mitigation:** Apply simplicity guardrails from `docs/architecture.md` before accepting additional hardening tasks.
3. **Risk:** Reliability follow-ups become fragmented across multiple steps.
**Mitigation:** Keep one consolidated carry-forward checklist and update it at milestone check-ins.
---
## Notes
This revision effort is scope-alignment and implementation-follow-through focused.
Historical Step 1/Step 2 documents are intentionally preserved as-is.
@@ -0,0 +1,42 @@
# Ver1 Step 2 Error-Path Inventory (Carry-Forward)
## Purpose
Provide a compact inventory of major failure paths with taxonomy mapping and retry behavior, aligned with:
- `docs/error_handling.md`
- `docs/ver1/ver1-step2-results.md`
- `docs/ver1/ver1-step1-2-carry-forward-checklist.md` (CF-B1)
This is a lightweight operational artifact for Step 6/7 follow-through.
---
## Inventory Table
| Path ID | Boundary/Operation | Typical Failure Source | Category | Retriable | Surface Behavior | Current Coverage | Notes |
| --- | --- | --- | --- | --- | --- | --- | --- |
| EP-API-001 | API upload request validation | invalid payload / empty file metadata | `validation_error` | no | structured API error envelope (400) | partial | confirm all upload variants |
| EP-API-002 | API resource lookup | missing job/document | `not_found_error` | no | structured API error envelope (404) | partial | verify consistency for all lookup routes |
| EP-SVC-001 | Service provider-call mapping | provider SDK/HTTP failure | `external_provider_error` | sometimes | normalized AppError and safe message | partial | ensure consistent mapping in service boundary tests |
| EP-WKR-001 | Worker provider timeout | timeout/unavailable upstream | `external_provider_error` or `infrastructure_transient_error` | yes | retry or terminal failed with persisted reason | partial | validate category mapping remains deterministic |
| EP-WKR-002 | Worker non-retriable domain/input failure | deterministic invalid input/state | `user_input_error` or `conflict_error` | no | immediate terminal failed with persisted reason | partial | ensure no retry on non-retriable categories |
| EP-WKR-003 | Worker retry exhaustion | repeated retriable failure | category from source; terminal state | capped then no | explicit failed status + error detail | met | implemented in Step 2; keep regression coverage |
| EP-UI-001 | UI upload action failure | surfaced AppError or fallback exception | category-based safe user message | category-driven | title + message + suggestion + error id | partial | verify consistency on all primary UI actions |
| EP-LOG-001 | Cross-boundary error logging | missing/uneven fields | n/a | n/a | logs include `error_id`, `category`, `operation`, ids when available | partial | complete in Step 6 (CF-B2) |
---
## Verification Targets (Step 6/7)
1. Every critical path has category + retriable policy defined.
2. API/UI behavior remains safe and actionable.
3. Worker terminal failures are explicit and persisted.
4. Logging fields are consistent at critical handoffs.
---
## Evidence Links
- Step 2 implementation results: `docs/ver1/ver1-step2-results.md`
- Carry-forward tracking: `docs/ver1/ver1-step1-2-carry-forward-checklist.md`
- Canonical contract: `docs/error_handling.md`
+160
View File
@@ -0,0 +1,160 @@
# Ver1 Step 3 Results: Functional Completion by Requirement Domain
## Summary
Step 3 implementation has been completed for the planned functional-completion scope in a practical personal-scale form.
Implemented in this step:
1. Revision history and acceptance workflows for transcripts.
2. Search over accepted transcript revisions.
3. Export of accepted transcript data.
4. API routes for jobs, revisions, search, and export.
5. UI pathways for revision management, search, and export.
6. Carry-forward integration updates for Step 1/2 follow-ups owned by Step 3.
---
## Implemented Changes
### 1) Data model expansion (functional domain)
Updated `src/transcription/models.py`:
- Added `JobStatus.COMPLETED`.
- Added `TranscriptRevision` table/model:
- `job_id`
- `revision_number`
- `text`
- `source`
- `accepted`
- `created_at`
- Added `Job.revisions` relationship.
This supports immutable revision history and accepted-transcript semantics for search/export.
### 2) Step 3 service layer
Created `src/transcription/services/library.py` with service-backed functional operations:
- `list_jobs(...)`
- `get_job_detail(...)`
- `add_revision(...)`
- `accept_revision(...)`
- `list_revisions(...)`
- `search_accepted_transcripts(...)`
- `export_transcripts(...)`
Key behavior:
- revisions are append-only and incrementing
- accepted revision is unique per job
- accepting a revision syncs canonical transcript and sets job to `completed`
- search scope is accepted revisions only
- export emits deterministic record payloads for archive workflows
### 3) Worker integration for revision provenance
Updated `src/transcription/worker.py`:
- Success path now calls `add_revision(..., source="worker", accepted=False)`.
- Worker still persists canonical transcript and `transcribed` job state.
- Initial machine transcription now appears in revision history.
### 4) API functional completion
Created `src/transcription/api/routes.py` and wired in `src/transcription/app.py`.
New endpoints:
- `GET /api/jobs`
- `GET /api/jobs/{job_id}`
- `GET /api/jobs/{job_id}/revisions`
- `POST /api/jobs/{job_id}/revisions`
- `POST /api/revisions/{revision_id}/accept`
- `GET /api/search?query=...`
- `GET /api/export?accepted_only=true|false`
### 5) UI functional completion
Updated `src/transcription/ui/jobs_page.py`:
- Job detail now includes revision history panel.
- Added user revision submission.
- Added revision accept action.
- Added `/search` page for accepted transcript search.
- Added `/export` page for accepted transcript export preview.
---
## Test Evidence
### Added/Updated Tests
1. `tests/services/test_library.py`
- revision append/accept behavior
- accepted-only search behavior
- export payload behavior
2. `tests/api/test_routes.py`
- jobs/revisions/search/export API serialization and contract behavior
3. `tests/test_models.py`
- `completed` status transition coverage
- `TranscriptRevision` persistence and relationship coverage
4. `tests/services/test_worker.py`
- success-path now verifies initial worker-generated revision persistence
### Full Validation Run
Executed and passing:
- `uv run pytest -q`
---
## Requirement Slice Coverage (Step 3)
| Slice | REQ Coverage | Status | Evidence |
| --- | --- | --- | --- |
| Core lifecycle completion and visibility | REQ-0, REQ-2, REQ-3, REQ-5, REQ-6 | met | worker integration + API/UI jobs routes + tests |
| Revision history and acceptance | REQ-3, REQ-4, REQ-5, REQ-11 | met | `TranscriptRevision`, `services/library.py`, UI revision panel, tests |
| Search over accepted transcripts | REQ-5, REQ-11 | met | `search_accepted_transcripts`, `/api/search`, `/ui/search`, tests |
| Export transcript data | REQ-4, REQ-5, REQ-11 | met | `export_transcripts`, `/api/export`, `/ui/export`, tests |
| Prompt and verbatim flow continuity | REQ-12 | met (continued) | worker transcription flow unchanged in prompt-loading contract |
---
## Carry-Forward Integration Updates
Updated:
- `docs/ver1/ver1-step1-2-carry-forward-checklist.md`
Step 3 updates recorded for:
- CF-A1: in progress with Step 3 inspection evidence
- CF-A3: in progress with boundary-discipline evidence
- CF-C1: done (Step 3 traceability artifacts integrated)
- CF-C2: in progress (routing preserved for later steps)
---
## Residual Follow-ups
1. Step 4: migration rehearsal and rollback runbook execution for schema changes.
2. Step 6/7: broader error-path inventory closure and logging field normalization.
3. Step 9: release readiness reconfirmation for runtime ownership and migration behavior.
---
## Step 3 Exit Assessment
- Requirement-domain functional completion: **met**
- Data integrity and state consistency for new flows: **met**
- API/UI parity for new Step 3 features: **met**
- Test and regression safety: **met**
- Carry-forward integration obligations (Step 3-owned): **met/in progress as routed**
Step 3 is complete and ready to hand off to Step 4.
+433
View File
@@ -0,0 +1,433 @@
# Step 3 Implementation Plan: Functional Completion by Requirement Domain
## Purpose
Implement **Ver1 Step 3** from `docs/ver1/ver1.md` by completing all in-scope V1 functional requirements in a practical, user-first order while preserving:
- personal-scale operation
- single-operator workflow
- private-network deployment assumptions
- low operational overhead
- clean architecture boundaries
Primary governing docs:
- `docs/ver1/ver1.md` (Step 3 objective and sequencing)
- `docs/architecture.md` (module boundaries, workflow, simplicity guardrails)
- `docs/requirements.md` (REQ-0 through REQ-12 traceability)
- `docs/error_handling.md` (error contract across boundaries)
- `docs/intent.md` (verbatim transcription policy and prompt curation)
- `docs/ver1/ver1-step1-2-carry-forward-checklist.md` (Step 1/2 carry-forward integration)
- `docs/ver1/ver1-step2-error-path-inventory.md` (failure-path coverage visibility)
---
## MCP Resources Reviewed and Applied
All resources on `john-stream-mcp` were reviewed. Step 3 applies the following guidance directly:
1. `resource://skills/nicegui/document`
- modular page registration
- one-way dependency flow (`ui/api -> services -> infra`)
- async-first UI responsiveness expectations
2. `resource://skills/nicegui-ui-customization/document`
- reusable UI component extraction for repeated patterns
- in-flight guards and explicit success/failure user feedback
- event-driven updates over ad-hoc polling
3. `resource://skills/fastapi-async-sqlalchemy-modernization/document`
- explicit transaction/session boundaries
- deterministic resource ownership and cleanup continuity from Step 1
- incremental migration strategy with rollback-aware checkpoints
4. `resource://skills/pydantic-settings/document`
- typed configuration as single source of runtime truth
- explicit source precedence and environment-safe defaults
5. `resource://skills/python-logging-dictconfig/document`
- centralized startup-only logging configuration
- named logger discipline and boundary-level structured fields
6. `resource://skills/pytesting/document`
- deterministic test structure and marker discipline
- behavior-first tests with clear fast-path and full-suite validation
7. `resource://skills/fastapi-uv-docker/document`
- health endpoint and runtime startup/shutdown hygiene
- compose/deployment readiness constraints relevant to functional completion
8. `resource://skills/python-typing/document`
- modern typing updates where touched by Step 3 work
9. `resource://skills/ruff-linting-formating/document`
- maintain lint/format consistency in all modified modules
10. `resource://prompts/greenfield-architecture/document`
- explicit staged delivery with tradeoff-aware sequencing and test strategy
11. `resource://prompts/pytest-scaffold/document`
12. `resource://prompts/pytest-fill-scaffold/document`
- structure-first test planning, then deterministic implementation fill-in
Resources reviewed but not directly in Step 3 execution scope (no changes required now):
- `copilot-customization`, `mcp-details`, `vscode-configuration`, `zensical-docs`
- prompts: `authoring`, `mcp-consumer-repo-shim`
---
## Step 3 Success Criteria
Step 3 is complete when:
1. All Step 3-targeted requirement slices are implemented and verified.
2. Functional behavior is available through UI/API where required.
3. Core data integrity and state transitions are deterministic.
4. Error behavior follows `docs/error_handling.md` contracts.
5. Carry-forward Step 1/2 items mapped to Step 3 are updated with evidence.
---
## Requirement-Slice Execution Model (Applied to Every Slice)
For each slice, execute this sequence:
1. Confirm contract/schema and boundary ownership.
2. Implement service/domain logic.
3. Implement persistence/state transitions.
4. Integrate API and/or UI behavior.
5. Add/update unit + integration + targeted end-to-end tests.
6. Update docs and traceability artifacts.
Definition of done per slice:
- behavior is functional
- tests pass in intended marker lanes
- error pathways are classified and surfaced correctly
- requirement traceability is updated with evidence
---
## Detailed Workstreams
## Workstream A — Functional Baseline Audit and Slice Backlog Lock
### Goals
- establish exact Step 3 functional delta from current implementation
- lock a practical slice backlog before coding
### Tasks
1. Build Step 3 requirement matrix (REQ -> current status -> gap -> target slice).
2. Map each gap to one of these domains:
- Upload and lifecycle integrity
- Review and revision history
- Search over accepted transcripts
- Export workflows
- Prompt asset management behavior
- API/UI parity and status visibility
3. Align each slice with architecture boundary ownership and persistence strategy.
4. Link open carry-forward items from checklist:
- CF-A1, CF-A3 (architecture continuity in Step 3)
- CF-C1, CF-C2 (traceability/execution continuity)
### Deliverables
- Step 3 requirement-slice matrix (appendix in this doc or separate artifact)
- prioritized slice backlog with owner and validation method
### Exit Criteria
- every Step 3 slice maps to REQ IDs and a validation method
- no ambiguous ownership remains for in-scope slices
---
## Workstream B — Core End-User Flows (Upload -> Transcribe -> Review)
### Related Requirements
- REQ-0, REQ-1, REQ-2, REQ-3, REQ-4, REQ-5, REQ-6, REQ-12
### Goals
- guarantee end-to-end reliability and usability of the primary user flow
- ensure review experience supports transcript acceptance and correction
### Tasks
1. Validate and close any lifecycle-state gaps:
- enforce valid transitions (`queued -> processing -> transcribed/failed/completed`)
- ensure transition visibility in UI/API
2. Review experience completion:
- transcript detail display stability
- failure detail readability and actionability
- acceptance/edit path for human review
3. Ensure prompt-asset integration remains file-based and auditable:
- one prompt per Markdown file
- prompt selection/usage traceability in job outcomes (if available in model)
4. Confirm worker/UI interactions remain responsive under long-running jobs:
- in-flight guards
- clear status refresh behavior
### Deliverables
- complete end-user flow behavior with stable lifecycle visibility
- test coverage for happy path and failure path
### Exit Criteria
- user can run upload -> process -> review reliably
- failed and successful outcomes are both actionable and traceable
---
## Workstream C — Revision History and Provenance Completion
### Related Requirements
- REQ-3, REQ-4, REQ-5, REQ-11
### Goals
- finalize immutable transcript revision behavior and provenance consistency
### Tasks
1. Define/confirm revision invariants:
- append-only revision history
- clear current/accepted revision indicator
2. Persist revision events consistently through service layer boundaries.
3. Ensure UI/API expose revision timeline and selected revision details.
4. Align error handling for revision conflicts and missing resources.
### Deliverables
- revision-history feature completeness
- provenance and history read-path coverage
### Exit Criteria
- transcript edits produce deterministic revision records
- previous revisions remain inspectable
---
## Workstream D — Search Completion (Accepted Transcript Scope)
### Related Requirements
- REQ-0, REQ-5, REQ-11
### Goals
- provide practical search over accepted transcripts for personal corpus usage
### Tasks
1. Finalize searchable scope and indexing rules (accepted/current text only).
2. Implement service-backed search query behavior.
3. Expose search in UI/API with clear result metadata (document/job/revision context).
4. Add guardrails for empty/no-result/error scenarios with actionable messaging.
### Deliverables
- functional search pathway with deterministic results for accepted text
### Exit Criteria
- operator can find transcripts reliably by text queries
- no-result and error states are clear and non-silent
---
## Workstream E — Export Completion
### Related Requirements
- REQ-0, REQ-4, REQ-5, REQ-11
### Goals
- deliver practical export of transcript data for personal archive use
### Tasks
1. Finalize export contract (format, included fields, scope filters).
2. Implement export service with deterministic data mapping.
3. Add UI/API trigger path and user-visible completion/failure feedback.
4. Validate export integrity against persisted source-of-record entities.
### Deliverables
- end-to-end export capability with operator-visible outcomes
### Exit Criteria
- export output is complete, consistent, and usable for downstream personal archive workflows
---
## Workstream F — API/UI Parity and Interaction Hardening
### Related Requirements
- REQ-5 plus cross-cutting REQ-2/3/4
### Goals
- ensure UI and API expose coherent feature behavior and error contracts
### Tasks
1. Verify API/UI parity matrix for each Step 3 slice.
2. Standardize interaction behavior:
- loading and in-flight states
- success/failure notifications
- stable error_id visibility where user-facing
3. Ensure route/page modules remain composition-focused (business logic in services).
### Deliverables
- API/UI parity checklist with resolved gaps
### Exit Criteria
- no major flow exists in one interface with conflicting semantics in the other
---
## Workstream G — Carry-Forward Integration During Step 3
### Goals
- close Step 1/2 follow-ups that are Step 3-owned
### Tasks
1. Update checklist item CF-A1 as Step 3 slices touch runtime resources.
2. Update checklist item CF-A3 with lightweight boundary enforcement evidence.
3. Update CF-C1/CF-C2 traceability mapping with Step 3 outcomes.
### Deliverables
- updated `docs/ver1/ver1-step1-2-carry-forward-checklist.md` evidence entries
### Exit Criteria
- Step 3-owned carry-forward items are either completed or explicitly routed with evidence
---
## Test and Validation Plan
Apply `pytesting` guidance with deterministic, behavior-focused coverage.
### Validation Lanes
1. Structure/collection:
- `uv run pytest --collect-only -q`
2. Fast feedback lane:
- `uv run pytest -m unit -q`
3. Main verification lane:
- `uv run pytest -m "not external" -q`
4. Full suite:
- `uv run pytest -q`
### Required Coverage Areas
- lifecycle transition invariants
- revision history invariants
- search query behavior and result mapping
- export integrity and failure handling
- UI interaction guards and actionable failure feedback
- API envelope and status consistency for new/changed flows
### Test Design Rules
- one behavior target per test
- minimize heavy mocking; prefer real-path behavior checks where practical
- keep markers explicit and strict
---
## Logging, Error, and Config Guardrails for Step 3 Changes
1. Logging
- keep centralized startup logging config (`dictConfig`) as canonical
- include required error fields at boundary failures (`error_id`, `category`, `operation`, identifiers where available)
2. Error handling
- preserve taxonomy stability from `docs/error_handling.md`
- map any new failure pathways into existing categories
- surface actionable suggestions in UI/API
3. Configuration
- use typed settings and avoid ad-hoc env reads in business modules
- keep environment behavior explicit and documented
---
## Implementation Order (Detailed)
1. Workstream A: audit and backlog lock
2. Workstream B: core flow completion
3. Workstream C: revision/provenance completion
4. Workstream D: search completion
5. Workstream E: export completion
6. Workstream F: API/UI parity hardening
7. Workstream G: carry-forward integration updates
8. Full validation pass + docs/traceability updates
---
## Deliverables
1. Step 3 requirement-slice matrix with REQ mapping and evidence links
2. implemented Step 3 functional slices across service/persistence/API/UI
3. updated tests and passing validation lanes
4. updated carry-forward checklist entries (`CF-A1`, `CF-A3`, `CF-C1`, `CF-C2` as applicable)
5. Step 3 results document (`docs/ver1/ver1-step3-results.md`)
---
## Risks and Mitigations
1. **Risk:** Scope creep from optional enhancements during feature completion
- **Mitigation:** enforce REQ-mapped slice backlog and defer non-REQ enhancements
2. **Risk:** Functional parity drift between UI and API
- **Mitigation:** maintain parity matrix and verify both surfaces per slice
3. **Risk:** Data-model changes introduce migration surprises
- **Mitigation:** coordinate with Step 4 runbook expectations early and test on representative data
4. **Risk:** Reliability regressions while adding functionality
- **Mitigation:** run full error-path regression checks and keep Step 2 contracts intact
---
## Step 3 Completion Checklist
- [ ] Step 3 requirement-slice matrix completed and linked to REQ IDs.
- [ ] Core end-user flow is functionally complete and verified.
- [ ] Revision history/provenance behavior is complete and test-covered.
- [ ] Search over accepted transcripts is complete and test-covered.
- [ ] Export flow is complete and test-covered.
- [ ] API/UI parity checklist has no unresolved high-impact gaps.
- [ ] Step 3-owned carry-forward items are updated with evidence.
- [ ] Validation lanes pass (`collect-only`, unit, non-external, full).
- [ ] `docs/ver1/ver1-step3-results.md` is created with evidence and residual follow-ups.
---
## Handoff to Step 4
Step 3 completion enables Step 4 (Data Model and Migration Safety) with:
- finalized functional domain behavior
- stable persistence expectations
- traceable requirement evidence
- clarified migration-impact surface
+133 -131
View File
@@ -1,40 +1,41 @@
# Version 1 Implementation Plan # Version 1 Implementation Plan
This plan defines the path from MVP to **Version 1 complete**. This plan defines the path from MVP to **Version 1 complete**.
The objective is to deliver the full scoped product with production readiness, while explicitly separating refinements/enhancements into a future document. The objective is to deliver the full scoped product with readiness for reliable personal-scale operation, while explicitly separating refinements/enhancements into a future document.
--- ---
## 0) Plan Governance & Scope Control (Foundation) ## 0) Plan Governance & Scope Control (Foundation)
**Goal:** Keep execution focused on V1 completion, not optimization/perfection. **Goal:** Keep execution focused on V1 completion and avoid unnecessary process overhead.
### Implementation Steps ### Implementation Steps
1. Create and maintain a **V1 Traceability Matrix**: 1. Create and maintain a **V1 Traceability Matrix**:
- Requirement ID - Requirement ID
- Current status (`done`, `partial`, `not started`) - Current status (`done`, `partial`, `not started`)
- Owner
- Validation method - Validation method
2. Define V1 completion gates: 2. Define V1 completion gates:
- Functional complete - Functional complete
- Operationally complete - Operationally complete
- Production-ready complete - Personal-deployment ready
3. Snapshot the MVP baseline (tag/changelog reference). 3. Snapshot the MVP baseline (tag/changelog reference).
4. Create a standing rule: any non-V1 idea is logged to a separate enhancements backlog document (to be named later), not added to active V1 scope unless explicitly approved. 4. Keep a standing rule: non-V1 ideas go to a separate enhancements backlog, and enter V1 only by explicit approval.
### Deliverables ### Deliverables
- `docs/ver1/ver1.md` (this plan) - `docs/ver1/ver1.md` (this plan)
- V1 traceability artifact (linked from here when created) - V1 traceability artifact:
- `docs/ver1/ver1-step1-2-carry-forward-checklist.md`
- `docs/ver1/ver1-step2-error-path-inventory.md` (supporting artifact)
### Exit Criteria ### Exit Criteria
- Every in-scope requirement has explicit ownership and status. - Every in-scope requirement has explicit status and validation evidence.
- Scope-change process is agreed and followed. - Scope-change discipline is followed consistently.
--- ---
## 1) Architecture Consolidation ## 1) Architecture Consolidation
**Goal:** Align implementation with the intended architecture and reduce MVP shortcuts. **Goal:** Align implementation with intended architecture while preserving simplicity.
### Implementation Steps ### Implementation Steps
1. Compare implemented modules/components with architecture documentation. 1. Compare implemented modules/components with architecture documentation.
@@ -42,201 +43,202 @@ The objective is to deliver the full scoped product with production readiness, w
- Temporary coupling - Temporary coupling
- Missing interfaces - Missing interfaces
- Placeholder services/components - Placeholder services/components
3. Resolve high-risk architectural gaps first. 3. Resolve architecture gaps that threaten reliability, maintainability, or clear boundaries.
4. Record key decisions and tradeoffs in ADRs. 4. Record material decisions and tradeoffs in ADRs.
### Deliverables ### Deliverables
- Updated architecture diagrams and boundaries - Updated architecture diagrams and boundaries
- ADR entries for major decisions - ADR entries for material decisions
### Exit Criteria ### Exit Criteria
- Architecture documentation reflects system reality. - Architecture documentation reflects system reality.
- Critical architecture risks are addressed or scheduled with owners/dates. - High-impact architecture risks are addressed or explicitly scheduled.
--- ---
## 2) Error Handling & Reliability Hardening ## 2) Error Handling & Reliability Hardening
**Goal:** Ensure predictable, safe behavior under failure conditions. **Goal:** Ensure predictable, diagnosable behavior under expected failure conditions.
### Implementation Steps ### Implementation Steps
1. Standardize error taxonomy and envelope format across all layers. 1. Apply the canonical taxonomy and response model from `docs/error_handling.md` across UI/API/service/worker boundaries.
2. Ensure clear distinction between: 2. Ensure clear distinction between:
- User-facing errors - User-facing safe messages
- Internal/system errors - Internal diagnostic detail
- Retryable vs non-retryable failures - Retryable vs non-retryable failures
3. Add resilience controls where needed: 3. Implement practical resilience controls where needed:
- Timeouts - Timeouts
- Retries with backoff - Bounded retries with backoff
- Circuit breaking / fallback logic - Explicit terminal failure states
4. Add failure-path tests for critical workflows. 4. Add failure-path tests for critical workflows.
### Deliverables ### Deliverables
- Error code catalog/reference - Error handling reference aligned with `docs/error_handling.md`
- Failure mode test coverage for critical paths - Failure-mode test coverage for critical paths
### Exit Criteria ### Exit Criteria
- Error behavior is consistent across major flows. - Error behavior is consistent across major flows.
- Known failure scenarios are tested and pass. - Known failure scenarios are tested and pass.
- Failed jobs include actionable, traceable failure detail.
--- ---
## 3) Functional Completion by Requirement Domain ## 3) Functional Completion by Requirement Domain
**Goal:** Complete all V1 functional requirements in a risk-aware order. **Goal:** Complete all V1 requirements in a practical, user-first order.
### Recommended Order ### Recommended Order
1. Business-critical end-user flows 1. End-user core flows (upload → transcribe → review)
2. Data integrity and consistency capabilities 2. Data integrity and persistence behavior
3. Admin/operational controls 3. Minimal operator controls needed for personal use
4. Lower-priority UX and quality-of-life items that are in V1 scope 4. In-scope UX quality improvements
### Implementation Steps ### Implementation Steps
For each requirement slice: For each requirement slice:
1. Finalize contract/schema 1. Confirm contract/schema
2. Implement domain logic 2. Implement service/domain logic
3. Implement persistence/state changes 3. Implement persistence/state transitions
4. Integrate API/UI 4. Integrate API/UI behavior
5. Add automated tests 5. Add or update automated tests
6. Update docs 6. Update relevant docs
### Deliverables ### Deliverables
- Requirement completion report with validation evidence - Requirement completion report with validation evidence linked to REQ IDs
### Exit Criteria ### Exit Criteria
- All V1 must-have requirements are complete and validated. - All V1 must-have requirements are complete and verified.
--- ---
## 4) Data Model, Migration, and Backfill Safety ## 4) Data Model and Migration Safety
**Goal:** Ensure data model and migrations are production-safe. **Goal:** Keep schema evolution safe and simple for personal-scale deployment.
### Implementation Steps ### Implementation Steps
1. Validate schema against final V1 domain needs. 1. Validate schema against finalized V1 domain needs.
2. Implement forward-safe migrations. 2. Implement forward-safe migrations for expected upgrades.
3. Define rollback/mitigation plans for migration failures. 3. Define a simple rollback/mitigation path for migration failures.
4. Build and verify backfill scripts (if needed). 4. Add backfill scripts only where truly required.
5. Add migration rehearsal in staging with representative data. 5. Rehearse migration + rollback locally using representative sample data.
### Deliverables ### Deliverables
- Migration runbook - Migration and rollback runbook
- Backfill verification checklist - Backfill checklist (if applicable)
### Exit Criteria ### Exit Criteria
- Migration plan validated in staging. - Migration path is tested and documented.
- No unresolved data-loss risk for V1 rollout. - No unresolved data-loss risk for V1 upgrade.
--- ---
## 5) Security, Access Control, and Compliance Baseline ## 5) Private-Network Safety Baseline
**Goal:** Close MVP security gaps and establish V1 baseline controls. **Goal:** Apply right-sized security controls for a single-user system on a trusted private network.
### Implementation Steps ### Implementation Steps
1. Complete authn/authz coverage for all routes/actions. 1. Enforce private-network deployment assumptions in docs and configuration.
2. Enforce input validation and output sanitization. 2. Ensure basic single-operator access control for UI/API actions.
3. Verify secret management and credential rotation process. 3. Enforce input validation and safe error output behavior.
4. Add audit logging for sensitive operations. 4. Keep secrets out of source control; document local secret handling.
5. Run dependency/security scanning in CI and remediate findings. 5. Run lightweight dependency/security scanning and resolve high-risk findings.
### Deliverables ### Deliverables
- Security checklist with status - Security assumptions checklist (private network, single operator)
- Threat/risk update for V1 scope - Basic risk update for V1 scope
### Exit Criteria ### Exit Criteria
- No unresolved critical/high vulnerabilities for V1 launch. - No unresolved critical vulnerabilities.
- Access control behavior verified by tests. - Access behavior and validation rules are verified for intended operating model.
--- ---
## 6) Observability & Operability ## 6) Minimal Observability & Operability
**Goal:** Make system behavior observable and supportable in production. **Goal:** Keep operation and troubleshooting simple, clear, and reliable.
### Implementation Steps ### Implementation Steps
1. Standardize structured logging and correlation IDs. 1. Standardize structured logging across UI/API/service/worker boundaries.
2. Add core metrics: 2. Ensure logged errors include category and error reference IDs per `error_handling.md`.
- Latency 3. Add lightweight health/startup checks.
- Throughput 4. Document a concise operator runbook:
- Error rates - start/stop
- Resource saturation - log locations
3. Add tracing for critical request/workflow paths. - common failure patterns and recovery steps
4. Define SLOs/SLIs and alert thresholds. 5. Add minimal counters/timings only where they clearly improve diagnosis.
5. Prepare incident response and rollback runbooks.
### Deliverables ### Deliverables
- Dashboards and alerts - Logging and error-traceability baseline
- Operations runbooks - Operator runbook
### Exit Criteria ### Exit Criteria
- Team can detect, triage, and remediate incidents quickly. - Operator can diagnose common failures using logs + runbook.
- Core production signals are available and reliable. - System recovery procedures are documented and repeatable.
--- ---
## 7) Test Strategy Expansion & Quality Gates ## 7) Test Coverage and Practical Quality Gates
**Goal:** Raise confidence for repeatable, low-risk releases. **Goal:** Prevent regressions in critical flows without overbuilding test infrastructure.
### Implementation Steps ### Implementation Steps
1. Expand unit and integration tests across V1 features. 1. Expand unit and integration tests for all V1 requirement slices.
2. Add contract tests between key components/services. 2. Add end-to-end tests for critical journeys:
3. Add end-to-end tests for critical user journeys. - upload
4. Add non-functional tests where relevant: - process/transcribe
- Performance/load - view result
- Soak - failure visibility
- Failure-injection scenarios 3. Add targeted contract tests where adapter boundaries are error-prone.
5. Enforce CI quality gates (tests, lint, type checks, security scans). 4. Keep CI gates focused on high-value checks (tests, lint, type checks, dependency scan).
### Deliverables ### Deliverables
- Test matrix with ownership - V1 test matrix mapped to requirements and critical flows
- CI gate definition and thresholds - CI quality-gate checklist
### Exit Criteria ### Exit Criteria
- Critical-path regressions are blocked automatically. - Critical-path regressions are automatically detected.
- Test coverage and reliability thresholds meet V1 targets. - Test suite gives consistent release confidence for personal-scale operation.
--- ---
## 8) Performance & Scalability Validation ## 8) Performance Validation for Personal Scale
**Goal:** Meet expected V1 performance at projected load. **Goal:** Confirm acceptable responsiveness for expected personal-use workload.
### Implementation Steps ### Implementation Steps
1. Define performance budgets per key flow. 1. Define practical performance expectations for key flows.
2. Benchmark current behavior in staging. 2. Run representative tests using real document samples.
3. Optimize bottlenecks (queries, caching, concurrency, etc.). 3. Address obvious bottlenecks in queries, file handling, or worker concurrency.
4. Re-test after each optimization and compare against budget. 4. Document known limits and expected operating bounds.
5. Document known limits and safe operating bounds.
### Deliverables ### Deliverables
- Performance benchmark report - Short performance validation note
- Optimization log - Known-limits summary
### Exit Criteria ### Exit Criteria
- V1 performance targets met for expected usage profile. - Core flows remain responsive for expected corpus size and usage patterns.
--- ---
## 9) Release Engineering & Environment Readiness ## 9) Release Readiness and Environment Simplicity
**Goal:** Make deployment repeatable, controlled, and reversible. **Goal:** Make deployment and rollback repeatable for a single-operator Docker Compose setup.
### Implementation Steps ### Implementation Steps
1. Harden CI/CD pipeline with clear promotion gates. 1. Define a simple release checklist:
2. Ensure config parity and consistency across environments. - run tests
3. Define rollout strategy (phased/canary/limited release as applicable). - run one end-to-end transcription check
4. Validate rollback procedures in staging. - verify migration compatibility
5. Produce release checklist and ownership model. 2. Document environment configuration requirements clearly.
3. Validate deployment and rollback steps in a local rehearsal.
4. Add backup/restore verification for core persisted data.
### Deliverables ### Deliverables
- Release playbook - Release checklist
- Environment readiness checklist - Environment and rollback guide
### Exit Criteria ### Exit Criteria
- Deployment and rollback are rehearsed and reliable. - Deployment/rollback is rehearsed and documented.
- Release process is executable without tribal knowledge. - Operator can release safely without hidden steps.
--- ---
@@ -252,7 +254,7 @@ For each requirement slice:
- Index/navigation - Index/navigation
- Intent alignment summary - Intent alignment summary
2. Add operator troubleshooting guides. 2. Add operator troubleshooting guides.
3. Add integration/API examples for consumers. 3. Add integration/API examples for the operator and future maintainers.
4. Publish changelog/version notes for V1. 4. Publish changelog/version notes for V1.
### Deliverables ### Deliverables
@@ -260,55 +262,55 @@ For each requirement slice:
- V1 release notes - V1 release notes
### Exit Criteria ### Exit Criteria
- A new team member can run/support the system using docs alone. - A future maintainer can run and support the system using docs alone.
--- ---
## 11) Final Validation, UAT, and Launch ## 11) Final Validation and Launch
**Goal:** Confirm readiness and launch V1 safely. **Goal:** Confirm V1 readiness and launch with low operational risk.
### Implementation Steps ### Implementation Steps
1. Run full-system acceptance validation against the V1 traceability matrix. 1. Run end-to-end acceptance validation against the V1 traceability matrix.
2. Conduct stakeholder UAT and capture sign-off. 2. Complete operator acceptance checks on representative real documents.
3. Execute production readiness review. 3. Execute launch checklist (including backup, migration, and rollback readiness).
4. Launch in controlled phases and monitor key signals. 4. Launch and monitor logs/status closely during initial use.
### Deliverables ### Deliverables
- UAT/PRR sign-off records - Acceptance validation record
- Launch checklist and monitoring plan - Launch checklist completion record
### Exit Criteria ### Exit Criteria
- Stakeholder approval achieved. - V1 requirements are validated.
- Launch metrics are stable within defined thresholds. - Initial launch behavior is stable and recoverable.
--- ---
## 12) Post-Launch Stabilization (3060 Days) ## 12) Post-Launch Stabilization
**Goal:** Consolidate V1 in production before major expansion. **Goal:** Address early issues quickly and lock in a reliable V1 baseline.
### Implementation Steps ### Implementation Steps
1. Track incidents, defects, and user feedback. 1. Track defects and operational pain points observed after launch.
2. Prioritize stabilization fixes with short cycle times. 2. Prioritize short-cycle stabilization fixes.
3. Remove temporary flags/mitigations introduced during launch. 3. Remove temporary launch-only workarounds when safe.
4. Produce post-launch retrospective and handoff to standard roadmap cadence. 4. Capture a brief retrospective and update the next-phase backlog.
### Deliverables ### Deliverables
- Stabilization report - Stabilization summary
- Prioritized backlog update - Updated backlog for post-V1 enhancements
### Exit Criteria ### Exit Criteria
- Incident/error rates converge to steady-state targets. - Major launch issues are resolved.
- V1 transitions from launch mode to normal operations. - System transitions to steady personal-use operation.
--- ---
## Recommended Execution Rhythm ## Recommended Execution Rhythm
- **Weekly:** Requirement closure + risk review - **Weekly:** Requirement closure + risk review
- **Biweekly:** Release train with quality gates - **As needed (small batch releases):** Run release checklist and deploy
- **Milestone reviews:** After phases 2, 6, 9, and 11 - **Milestone check-ins:** After phases 2, 6, 9, and 11
--- ---
+162
View File
@@ -0,0 +1,162 @@
"""Functional API routes for jobs, revisions, search, and export."""
from __future__ import annotations
from uuid import UUID
from fastapi import APIRouter
from pydantic import BaseModel, Field
from transcription.services.library import (
accept_revision,
add_revision,
export_transcripts,
get_job_detail,
list_jobs,
list_revisions,
search_accepted_transcripts,
)
router = APIRouter(prefix="/api", tags=["transcription"])
class CreateRevisionRequest(BaseModel):
text: str = Field(min_length=1)
source: str = "user"
accepted: bool = False
@router.get("/jobs")
def get_jobs() -> list[dict[str, str]]:
jobs = list_jobs()
return [
{
"id": str(job.id),
"document_id": str(job.document_id),
"status": job.status.value,
"created_at": job.created_at.isoformat(),
"updated_at": job.updated_at.isoformat(),
}
for job in jobs
]
@router.get("/jobs/{job_id}")
def get_job(job_id: UUID) -> dict[str, object | None]:
detail = get_job_detail(job_id=job_id)
return {
"job": {
"id": str(detail.job.id),
"document_id": str(detail.job.document_id),
"status": detail.job.status.value,
"created_at": detail.job.created_at.isoformat(),
"updated_at": detail.job.updated_at.isoformat(),
},
"document": (
{
"id": str(detail.document.id),
"filename": detail.document.filename,
"file_path": detail.document.file_path,
}
if detail.document is not None
else None
),
"transcript": (
{
"id": str(detail.transcript.id),
"text": detail.transcript.text,
"error_detail": detail.transcript.error_detail,
"created_at": detail.transcript.created_at.isoformat(),
}
if detail.transcript is not None
else None
),
"accepted_revision": (
{
"id": str(detail.accepted_revision.id),
"revision_number": detail.accepted_revision.revision_number,
"text": detail.accepted_revision.text,
"source": detail.accepted_revision.source,
"created_at": detail.accepted_revision.created_at.isoformat(),
}
if detail.accepted_revision is not None
else None
),
}
@router.get("/jobs/{job_id}/revisions")
def get_job_revisions(job_id: UUID) -> list[dict[str, object]]:
revisions = list_revisions(job_id=job_id)
return [
{
"id": str(revision.id),
"job_id": str(revision.job_id),
"revision_number": revision.revision_number,
"text": revision.text,
"source": revision.source,
"accepted": revision.accepted,
"created_at": revision.created_at.isoformat(),
}
for revision in revisions
]
@router.post("/jobs/{job_id}/revisions")
def create_job_revision(job_id: UUID, payload: CreateRevisionRequest) -> dict[str, object]:
revision = add_revision(
job_id=job_id,
text=payload.text,
source=payload.source,
accepted=payload.accepted,
)
return {
"id": str(revision.id),
"job_id": str(revision.job_id),
"revision_number": revision.revision_number,
"text": revision.text,
"source": revision.source,
"accepted": revision.accepted,
"created_at": revision.created_at.isoformat(),
}
@router.post("/revisions/{revision_id}/accept")
def accept_job_revision(revision_id: UUID) -> dict[str, object]:
revision = accept_revision(revision_id=revision_id)
return {
"id": str(revision.id),
"job_id": str(revision.job_id),
"revision_number": revision.revision_number,
"text": revision.text,
"source": revision.source,
"accepted": revision.accepted,
"created_at": revision.created_at.isoformat(),
}
@router.get("/search")
def search(query: str) -> list[dict[str, object]]:
results = search_accepted_transcripts(query=query)
return [
{
"revision_id": str(revision.id),
"job_id": str(revision.job_id),
"revision_number": revision.revision_number,
"text": revision.text,
"source": revision.source,
"accepted": revision.accepted,
"created_at": revision.created_at.isoformat(),
}
for revision in results
]
@router.get("/export")
def export(accepted_only: bool = True) -> dict[str, object]:
records = export_transcripts(accepted_only=accepted_only)
return {
"count": len(records),
"accepted_only": accepted_only,
"records": records,
}
+2
View File
@@ -9,6 +9,7 @@ from fastapi import FastAPI
from transcription.api.errors import register_error_handlers from transcription.api.errors import register_error_handlers
from transcription.api.health import router as health_router from transcription.api.health import router as health_router
from transcription.api.routes import router as transcription_router
from transcription.config import get_settings, setup_logging from transcription.config import get_settings, setup_logging
from transcription.db import ( from transcription.db import (
create_all, create_all,
@@ -74,5 +75,6 @@ def create_app() -> FastAPI:
register_error_handlers(app) register_error_handlers(app)
register_pages(app) register_pages(app)
app.include_router(health_router) app.include_router(health_router)
app.include_router(transcription_router)
return app return app
+23 -18
View File
@@ -1,8 +1,4 @@
"""SQLModel domain models for the transcription system. """SQLModel domain models for the transcription system."""
Three models capture the MVP lifecycle:
Document -> one-to-many -> Job -> one-to-one -> Transcript
"""
from datetime import datetime, timezone from datetime import datetime, timezone
from enum import StrEnum from enum import StrEnum
@@ -16,6 +12,7 @@ class JobStatus(StrEnum):
QUEUED = "queued" QUEUED = "queued"
PROCESSING = "processing" PROCESSING = "processing"
TRANSCRIBED = "transcribed" TRANSCRIBED = "transcribed"
COMPLETED = "completed"
FAILED = "failed" FAILED = "failed"
@@ -25,9 +22,7 @@ class Document(SQLModel, table=True):
id: UUID = Field(default_factory=uuid4, primary_key=True) id: UUID = Field(default_factory=uuid4, primary_key=True)
filename: str filename: str
file_path: str file_path: str
uploaded_at: datetime = Field( uploaded_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
default_factory=lambda: datetime.now(timezone.utc),
)
# --- relationships --- # --- relationships ---
jobs: list["Job"] = Relationship(back_populates="document") jobs: list["Job"] = Relationship(back_populates="document")
@@ -40,28 +35,38 @@ class Job(SQLModel, table=True):
document_id: UUID = Field(foreign_key="document.id") document_id: UUID = Field(foreign_key="document.id")
status: JobStatus = Field(default=JobStatus.QUEUED) status: JobStatus = Field(default=JobStatus.QUEUED)
retry_count: int = Field(default=0, ge=0) retry_count: int = Field(default=0, ge=0)
created_at: datetime = Field( created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
default_factory=lambda: datetime.now(timezone.utc), updated_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
)
updated_at: datetime = Field(
default_factory=lambda: datetime.now(timezone.utc),
)
# --- relationships --- # --- relationships ---
document: Document = Relationship(back_populates="jobs") document: Document = Relationship(back_populates="jobs")
transcript: Optional["Transcript"] = Relationship(back_populates="job") transcript: Optional["Transcript"] = Relationship(back_populates="job")
revisions: list["TranscriptRevision"] = Relationship(back_populates="job")
class Transcript(SQLModel, table=True): class Transcript(SQLModel, table=True):
"""The output of a transcription job.""" """Canonical transcript state for a job (latest text or failure detail)."""
id: UUID = Field(default_factory=uuid4, primary_key=True) id: UUID = Field(default_factory=uuid4, primary_key=True)
job_id: UUID = Field(foreign_key="job.id", unique=True) job_id: UUID = Field(foreign_key="job.id", unique=True)
text: str | None = None text: str | None = None
error_detail: str | None = None error_detail: str | None = None
created_at: datetime = Field( created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
default_factory=lambda: datetime.now(timezone.utc),
)
# --- relationships --- # --- relationships ---
job: Job = Relationship(back_populates="transcript") job: Job = Relationship(back_populates="transcript")
class TranscriptRevision(SQLModel, table=True):
"""Immutable transcript revision history for review/acceptance workflows."""
id: UUID = Field(default_factory=uuid4, primary_key=True)
job_id: UUID = Field(foreign_key="job.id", index=True)
revision_number: int = Field(ge=1)
text: str
source: str = Field(default="worker")
accepted: bool = Field(default=False)
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
# --- relationships ---
job: Job = Relationship(back_populates="revisions")
+257
View File
@@ -0,0 +1,257 @@
"""Step 3 functional services: job detail, revisions, search, and export."""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
from uuid import UUID
from sqlmodel import Session, select
from transcription.db import get_session
from transcription.errors import AppError, ErrorCategory
from transcription.models import Document, Job, JobStatus, Transcript, TranscriptRevision
class LibraryError(AppError):
"""Base error for review/search/export service pathways."""
@dataclass(frozen=True)
class JobDetail:
"""Job detail read model including latest transcript and accepted revision."""
job: Job
document: Document | None
transcript: Transcript | None
accepted_revision: TranscriptRevision | None
def list_jobs(*, session: Session | None = None) -> list[Job]:
"""Return jobs in most-recent-first order."""
if session is None:
with get_session() as local_session:
return list_jobs(session=local_session)
return list(session.exec(select(Job).order_by(Job.created_at.desc())).all())
def get_job_detail(*, job_id: UUID, session: Session | None = None) -> JobDetail:
"""Fetch job detail with related document/transcript and accepted revision."""
if session is None:
with get_session() as local_session:
return get_job_detail(job_id=job_id, session=local_session)
job = session.get(Job, job_id)
if job is None:
raise LibraryError(
f"Job not found: {job_id}",
category=ErrorCategory.NOT_FOUND,
suggestion="Refresh jobs list and open a valid job id.",
)
document = session.get(Document, job.document_id)
transcript = session.exec(select(Transcript).where(Transcript.job_id == job.id)).first()
accepted_revision = session.exec(
select(TranscriptRevision)
.where(TranscriptRevision.job_id == job.id, TranscriptRevision.accepted.is_(True))
.order_by(TranscriptRevision.revision_number.desc())
).first()
return JobDetail(
job=job,
document=document,
transcript=transcript,
accepted_revision=accepted_revision,
)
def add_revision(
*,
job_id: UUID,
text: str,
source: str = "user",
accepted: bool = False,
session: Session | None = None,
) -> TranscriptRevision:
"""Append a transcript revision and optionally mark it as accepted."""
if not text.strip():
raise LibraryError(
"Revision text cannot be empty",
category=ErrorCategory.VALIDATION,
suggestion="Provide non-empty transcript text and retry.",
)
if session is None:
with get_session() as local_session:
return add_revision(
job_id=job_id,
text=text,
source=source,
accepted=accepted,
session=local_session,
)
job = session.get(Job, job_id)
if job is None:
raise LibraryError(
f"Job not found: {job_id}",
category=ErrorCategory.NOT_FOUND,
suggestion="Refresh jobs list and retry with a valid job id.",
)
revisions = list(
session.exec(
select(TranscriptRevision)
.where(TranscriptRevision.job_id == job_id)
.order_by(TranscriptRevision.revision_number)
).all()
)
next_revision_number = (revisions[-1].revision_number + 1) if revisions else 1
if accepted:
for existing in revisions:
if existing.accepted:
existing.accepted = False
session.add(existing)
revision = TranscriptRevision(
job_id=job_id,
revision_number=next_revision_number,
text=text,
source=source,
accepted=accepted,
)
session.add(revision)
transcript = session.exec(select(Transcript).where(Transcript.job_id == job_id)).first()
if transcript is None:
transcript = Transcript(job_id=job_id)
transcript.text = text
transcript.error_detail = None
session.add(transcript)
job.updated_at = datetime.now(timezone.utc)
if accepted:
job.status = JobStatus.COMPLETED
elif job.status == JobStatus.QUEUED:
job.status = JobStatus.TRANSCRIBED
session.add(job)
session.commit()
session.refresh(revision)
return revision
def accept_revision(*, revision_id: UUID, session: Session | None = None) -> TranscriptRevision:
"""Mark one revision as accepted and synchronize canonical transcript/job state."""
if session is None:
with get_session() as local_session:
return accept_revision(revision_id=revision_id, session=local_session)
revision = session.get(TranscriptRevision, revision_id)
if revision is None:
raise LibraryError(
f"Revision not found: {revision_id}",
category=ErrorCategory.NOT_FOUND,
suggestion="Refresh job detail and select a valid revision.",
)
all_revisions = list(session.exec(select(TranscriptRevision).where(TranscriptRevision.job_id == revision.job_id)).all())
for item in all_revisions:
item.accepted = item.id == revision.id
session.add(item)
transcript = session.exec(select(Transcript).where(Transcript.job_id == revision.job_id)).first()
if transcript is None:
transcript = Transcript(job_id=revision.job_id)
transcript.text = revision.text
transcript.error_detail = None
session.add(transcript)
job = session.get(Job, revision.job_id)
if job is not None:
job.status = JobStatus.COMPLETED
job.updated_at = datetime.now(timezone.utc)
session.add(job)
session.commit()
session.refresh(revision)
return revision
def list_revisions(*, job_id: UUID, session: Session | None = None) -> list[TranscriptRevision]:
"""Return revision history for a job in ascending revision order."""
if session is None:
with get_session() as local_session:
return list_revisions(job_id=job_id, session=local_session)
if session.get(Job, job_id) is None:
raise LibraryError(
f"Job not found: {job_id}",
category=ErrorCategory.NOT_FOUND,
suggestion="Refresh jobs list and open a valid job id.",
)
return list(
session.exec(
select(TranscriptRevision)
.where(TranscriptRevision.job_id == job_id)
.order_by(TranscriptRevision.revision_number)
).all()
)
def search_accepted_transcripts(*, query: str, session: Session | None = None) -> list[TranscriptRevision]:
"""Search accepted transcript revisions using case-insensitive text containment."""
if not query.strip():
raise LibraryError(
"Search query cannot be empty",
category=ErrorCategory.VALIDATION,
suggestion="Enter a non-empty search query and retry.",
)
if session is None:
with get_session() as local_session:
return search_accepted_transcripts(query=query, session=local_session)
pattern = f"%{query.strip()}%"
return list(
session.exec(
select(TranscriptRevision)
.where(TranscriptRevision.accepted.is_(True), TranscriptRevision.text.ilike(pattern))
.order_by(TranscriptRevision.created_at.desc())
).all()
)
def export_transcripts(*, accepted_only: bool = True, session: Session | None = None) -> list[dict[str, str | int | None]]:
"""Export transcript data as serializable records for archive workflows."""
if session is None:
with get_session() as local_session:
return export_transcripts(accepted_only=accepted_only, session=local_session)
statement = select(TranscriptRevision).order_by(TranscriptRevision.created_at)
if accepted_only:
statement = statement.where(TranscriptRevision.accepted.is_(True))
revisions = list(session.exec(statement).all())
payload: list[dict[str, str | int | None]] = []
for revision in revisions:
detail = get_job_detail(job_id=revision.job_id, session=session)
payload.append(
{
"job_id": str(revision.job_id),
"document_id": str(detail.job.document_id),
"filename": detail.document.filename if detail.document else None,
"revision_id": str(revision.id),
"revision_number": revision.revision_number,
"accepted": revision.accepted,
"source": revision.source,
"text": revision.text,
"created_at": revision.created_at.isoformat(),
}
)
return payload
+107
View File
@@ -10,6 +10,13 @@ from sqlmodel import select
from transcription.db import get_session from transcription.db import get_session
from transcription.models import Document, Job, Transcript from transcription.models import Document, Job, Transcript
from transcription.services.library import (
accept_revision,
add_revision,
export_transcripts,
list_revisions,
search_accepted_transcripts,
)
from transcription.ui.error_presenter import show_error, summarize_error from transcription.ui.error_presenter import show_error, summarize_error
@@ -134,4 +141,104 @@ def register_page() -> None:
ui.label("Failure detail:") ui.label("Failure detail:")
ui.label(transcript.error_detail) ui.label(transcript.error_detail)
ui.separator()
ui.label("Revision History")
revisions_container = ui.column()
def render_revisions() -> None:
revisions_container.clear()
with revisions_container:
revisions = list_revisions(job_id=parsed_id)
if not revisions:
ui.label("No revisions yet.")
return
for revision in revisions:
with ui.card().classes("w-full"):
ui.label(
f"Revision {revision.revision_number} | source={revision.source} | accepted={revision.accepted}"
)
ui.markdown(revision.text)
if not revision.accepted:
ui.button(
"Accept revision",
on_click=lambda rev_id=revision.id: _accept_revision(rev_id),
)
def _accept_revision(revision_id):
try:
accept_revision(revision_id=revision_id)
ui.notify("Revision accepted", type="positive")
render_revisions()
except Exception as exc: # noqa: BLE001
show_error(exc, title="Accept revision failed", operation="revisions.accept")
new_revision_text = ui.textarea("Add revision text").props("rows=6")
def _submit_revision() -> None:
try:
add_revision(job_id=parsed_id, text=new_revision_text.value or "", source="user", accepted=False)
new_revision_text.value = ""
ui.notify("Revision added", type="positive")
render_revisions()
except Exception as exc: # noqa: BLE001
show_error(exc, title="Add revision failed", operation="revisions.create")
ui.button("Add revision", on_click=_submit_revision)
render_revisions()
ui.link("Search transcripts", "/search")
ui.link("Export transcripts", "/export")
ui.link("Back to jobs", "/jobs")
@ui.page("/search")
def search_page() -> None:
ui.label("Search Accepted Transcripts")
query_input = ui.input("Search query")
results_container = ui.column()
def run_search() -> None:
results_container.clear()
try:
results = search_accepted_transcripts(query=query_input.value or "")
except Exception as exc: # noqa: BLE001
show_error(exc, title="Search failed", operation="search.run")
return
with results_container:
if not results:
ui.label("No results.")
return
for result in results:
with ui.card().classes("w-full"):
ui.label(f"Job {result.job_id} | Revision {result.revision_number}")
ui.markdown(result.text)
ui.button("Search", on_click=run_search)
ui.link("Back to jobs", "/jobs")
@ui.page("/export")
def export_page() -> None:
ui.label("Export Accepted Transcripts")
results_container = ui.column()
def run_export() -> None:
results_container.clear()
try:
records = export_transcripts(accepted_only=True)
except Exception as exc: # noqa: BLE001
show_error(exc, title="Export failed", operation="export.run")
return
with results_container:
ui.label(f"Exported records: {len(records)}")
for record in records:
with ui.card().classes("w-full"):
ui.label(f"{record['filename']} | Revision {record['revision_number']}")
ui.markdown(str(record["text"]))
ui.button("Run export", on_click=run_export)
ui.link("Back to jobs", "/jobs") ui.link("Back to jobs", "/jobs")
+10 -2
View File
@@ -15,6 +15,7 @@ from transcription.config import Settings, get_settings
from transcription.db import get_session from transcription.db import get_session
from transcription.errors import AppError, ErrorCategory, classify_unexpected_error, format_error_detail from transcription.errors import AppError, ErrorCategory, classify_unexpected_error, format_error_detail
from transcription.models import Document, Job, JobStatus, Transcript from transcription.models import Document, Job, JobStatus, Transcript
from transcription.services.library import add_revision
from transcription.services.transcription import transcribe_document_image from transcription.services.transcription import transcribe_document_image
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -66,16 +67,23 @@ def _process_next_queued_job(*, session: Session) -> bool:
try: try:
result = transcribe_document_image(document.file_path) result = transcribe_document_image(document.file_path)
_upsert_transcript(session=session, job_id=job.id, text=result.text, error_detail=None) revision = add_revision(
job_id=job.id,
text=result.text,
source="worker",
accepted=False,
session=session,
)
job.status = JobStatus.TRANSCRIBED job.status = JobStatus.TRANSCRIBED
job.updated_at = datetime.now(timezone.utc) job.updated_at = datetime.now(timezone.utc)
session.add(job) session.add(job)
session.commit() session.commit()
logger.info( logger.info(
"Job transcribed operation=worker.process_job job_id=%s document_id=%s provider=%s", "Job transcribed operation=worker.process_job job_id=%s document_id=%s provider=%s revision_number=%s",
job.id, job.id,
document.id, document.id,
result.provider, result.provider,
revision.revision_number,
) )
except Exception as exc: # noqa: BLE001 except Exception as exc: # noqa: BLE001
error = exc if isinstance(exc, AppError) else classify_unexpected_error(exc, operation="worker.process_job") error = exc if isinstance(exc, AppError) else classify_unexpected_error(exc, operation="worker.process_job")
+118
View File
@@ -0,0 +1,118 @@
"""Tests for Step 3 functional API routes."""
from datetime import datetime, timezone
from types import SimpleNamespace
from uuid import uuid4
from fastapi import FastAPI
from fastapi.testclient import TestClient
import pytest
from transcription.api.errors import register_error_handlers
from transcription.api.routes import router
def _build_app() -> FastAPI:
app = FastAPI()
register_error_handlers(app)
app.include_router(router)
return app
@pytest.mark.integration
class TestFunctionalRoutes:
"""Verify jobs/revisions/search/export route behavior."""
def test_get_jobs_returns_serialized_rows(self, monkeypatch):
"""GET /api/jobs returns normalized job rows."""
now = datetime.now(timezone.utc)
job = SimpleNamespace(
id=uuid4(),
document_id=uuid4(),
status=SimpleNamespace(value="queued"),
created_at=now,
updated_at=now,
)
monkeypatch.setattr("transcription.api.routes.list_jobs", lambda: [job])
client = TestClient(_build_app())
response = client.get("/api/jobs")
assert response.status_code == 200
payload = response.json()
assert len(payload) == 1
assert payload[0]["id"] == str(job.id)
assert payload[0]["status"] == "queued"
def test_create_revision_returns_revision_payload(self, monkeypatch):
"""POST /api/jobs/{job_id}/revisions returns created revision fields."""
revision = SimpleNamespace(
id=uuid4(),
job_id=uuid4(),
revision_number=2,
text="edited text",
source="user",
accepted=False,
created_at=datetime.now(timezone.utc),
)
monkeypatch.setattr("transcription.api.routes.add_revision", lambda **_kwargs: revision)
client = TestClient(_build_app())
response = client.post(
f"/api/jobs/{revision.job_id}/revisions",
json={"text": "edited text", "source": "user", "accepted": False},
)
assert response.status_code == 200
payload = response.json()
assert payload["id"] == str(revision.id)
assert payload["revision_number"] == 2
assert payload["text"] == "edited text"
def test_search_returns_results(self, monkeypatch):
"""GET /api/search returns accepted transcript matches."""
result = SimpleNamespace(
id=uuid4(),
job_id=uuid4(),
revision_number=1,
text="family archive",
source="user",
accepted=True,
created_at=datetime.now(timezone.utc),
)
monkeypatch.setattr("transcription.api.routes.search_accepted_transcripts", lambda query: [result])
client = TestClient(_build_app())
response = client.get("/api/search", params={"query": "archive"})
assert response.status_code == 200
payload = response.json()
assert len(payload) == 1
assert payload[0]["revision_id"] == str(result.id)
assert payload[0]["accepted"] is True
def test_export_returns_count_and_records(self, monkeypatch):
"""GET /api/export returns record count and payload list."""
records = [
{
"job_id": str(uuid4()),
"document_id": str(uuid4()),
"filename": "letter.jpg",
"revision_id": str(uuid4()),
"revision_number": 1,
"accepted": True,
"source": "user",
"text": "exported",
"created_at": datetime.now(timezone.utc).isoformat(),
}
]
monkeypatch.setattr("transcription.api.routes.export_transcripts", lambda accepted_only=True: records)
client = TestClient(_build_app())
response = client.get("/api/export")
assert response.status_code == 200
payload = response.json()
assert payload["count"] == 1
assert payload["accepted_only"] is True
assert payload["records"] == records
@@ -11,11 +11,11 @@ BOOK 1 had 54 pages; 14 chapters. BOOK 2 has 70 pages; 18 chapters. BOOK 1 con-
sisted largely of first generation family history. BOOK 2 throws more light on sisted largely of first generation family history. BOOK 2 throws more light on
the second generation. Sidney promises a BOOK 3 and that may begin to do justice the second generation. Sidney promises a BOOK 3 and that may begin to do justice
to the third generation. We suggest that Sidney get the help of Louis Shinn to the third generation. We suggest that Sidney get the help of Louis Shinn
who has a chapter in this book (Chapter 16 - The Last 25 Years on the Doumeeq who has a chapter in this book (Chapter 16 - The Last 25 Years on the Doumecq
Plains. Louis has the gift of seeing, recalling and telling. One sentence in Plains. Louis has the gift of seeing, recalling and telling. One sentence in
his chapter gives a great tribute to the Doumeeqers - so far as he knows no one his chapter gives a great tribute to the Doumecqers--so far as he knows no one
on the Doumeeq Plains went on relief during the depression. That in a nutshell on the Doumecq Plains went on relief during the depression. That in a nutshell
shows the sturdy character of the residents of the Doumeeq Plains. shows the sturdy character of the residents of the Doumecq Plains.
We promised in BOOK 1 that in BOOK 2 we would give the story of the trip of We promised in BOOK 1 that in BOOK 2 we would give the story of the trip of
John E. Cochran and wife to Tennessee, Cuba and the Panama Canal. You will see John E. Cochran and wife to Tennessee, Cuba and the Panama Canal. You will see
@@ -32,7 +32,7 @@ enough pictures but we had to take only part of them. We think there are great
possibilities in reproducing old pictures. We wish we had a Pickard group. Some possibilities in reproducing old pictures. We wish we had a Pickard group. Some
Pickard descendant may wish to make a collection. Pickard descendant may wish to make a collection.
We are much impressed with the future possibilities of getting a complete geneol- We are much impressed with the future possibilities of getting a complete geneal-
ogy of the Pickard family. Mr. Cochran has a fine chapter on the Pickards but ogy of the Pickard family. Mr. Cochran has a fine chapter on the Pickards but
to date we have not had the pleasure of finding all of the family dates. We had to date we have not had the pleasure of finding all of the family dates. We had
intended to give more family data in this book but it takes time to get the intended to give more family data in this book but it takes time to get the
@@ -18,7 +18,6 @@ the family newsletter two years ago.
Nome Alaska August 26, 1923 Nome Alaska August 26, 1923
My Dear Ethel et al. My Dear Ethel et al.
I don't know when I did write or when you did I don't know when I did write or when you did
but I am going to write now however and never but I am going to write now however and never
the less. But I wish I could talk (I can yet but I the less. But I wish I could talk (I can yet but I
@@ -27,7 +26,7 @@ and Polly sit up and listen and that little black
rascal of yours would fairly sparkle with rascal of yours would fairly sparkle with
listening. Can't I see him listening now to all the listening. Can't I see him listening now to all the
yarns we told last summer? yarns we told last summer?
[photo of people on ice with ship in background]
You see, we-Miss Saville and I, took a trip north You see, we-Miss Saville and I, took a trip north
on the Buford and it was very interesting. We on the Buford and it was very interesting. We
went north thru the Bering Strait into the Arctic and as far as the Ice Pack. There the captain went north thru the Bering Strait into the Arctic and as far as the Ice Pack. There the captain
@@ -43,14 +42,14 @@ all around it similar to a currycomb in coarseness; no ears but huge tusks of iv
the most repulsive looking animals imaginable and tho I have always read about them I never the most repulsive looking animals imaginable and tho I have always read about them I never
expect such disagreeable looking creatures. They had a rough brown hairy skin and some of expect such disagreeable looking creatures. They had a rough brown hairy skin and some of
them looked warty. They must have weighed two ton at least. Ere we got them back to Nome them looked warty. They must have weighed two ton at least. Ere we got them back to Nome
to the natives they were getting extremely odiferousin fact, you could scarcely stay on the to the natives they were getting extremely odiferousin fact, you could scarcely stay on the
ship with any degree of comfort unless you had per chance lost your sense of smell. ship with any degree of comfort unless you had per chance lost your sense of smell.
Then we went north to a few minutes beyond the 70th degree of latitude and thot [sic] for awhile Then we went north to a few minutes beyond the 70th degree of latitude and thot for awhile
we would go to Wrangell Island where some men from Stefflonsons [sic] ship were supposed to be we would go to Wrangell Island where some men from Steffonsons ship were supposed to be
stranded but we didn't get there and instead stopped at a small native village at Cape Serdz [sic] in stranded but we didn't get there and instead stopped at a small native village at Cape Serdz in
Siberia. These Eskimo were very primitive. One white squaw man lived there and had for 23 Siberia. These Eskimo were very primitive. One white squaw man lived there and had for 23
years. He was a Swedewho else could. Their houses were circular and built up with dirt 2 or years. He was a Swede--who else could. Their houses were circular and built up with dirt 2 or
3 feet and then skins were stretched over it and weighted down with rocks. Inside, the room 3 feet and then skins were stretched over it and weighted down with rocks. Inside, the room
was partitioned off at the sides with skins for sleeping quarters. In the main part they had the was partitioned off at the sides with skins for sleeping quarters. In the main part they had the
fire on the ground and the fish drying on lines and the skins hanging around and the dogs and fire on the ground and the fish drying on lines and the skins hanging around and the dogs and
@@ -65,22 +64,22 @@ The other place we stopped was at Whalen, a trading post in Siberia. There these
went wild. They rushed helter-skelter, hither and thither, here and there, trying to find went wild. They rushed helter-skelter, hither and thither, here and there, trying to find
something to buy. Prices raised right before your eyes. One would but something for $1.00 something to buy. Prices raised right before your eyes. One would but something for $1.00
and the next might have to pay $2.00, $4.00 or $10.00. That made no difference. They had to and the next might have to pay $2.00, $4.00 or $10.00. That made no difference. They had to
have it. One man I was sort of taking care of, tho [sic] he had his son along for the purpose, have it. One man I was sort of taking care of, tho he had his son along for the purpose,
bought 2 ivory tusks, 1 pup, 2 moccasins, 3 or 4 billi[illegible]s, 6 or 8 ivory and silver rings, one bought 2 ivory tusks, 1 pup, 2 moccasins, 3 or 4 billikens, 6 or 8 ivory and silver rings, one
fishing line, hooks, floats, etc. and two bird slings. The slings have rocks at the end and the fishing line, hooks, floats, etc. and two bird slings. The slings have rocks at the end and the
little natives throw them at the flocks of geese and ducks which fly close over the village and little natives throw them at the flocks of geese and ducks which fly close over the village and
the slings entangle their wings and legs, sometimes more than one, and they can't fly. They the slings entangle their wings and legs, sometimes more than one, and they can't fly. They
come down and the natives capture them. There was more junk brot [sic] aboard than baggage, I come down and the natives capture them. There was more junk brot aboard than baggage, I
do believe. And they say that at the first stop it was worse than here. The red flag was flying do believe. And they say that at the first stop it was worse than here. The red flag was flying
over Whalen and the Russian soldiers were therea few, one or two or three, I forget the over Whalen and the Russian soldiers were therea few, one or two or three, I forget the
number. number.
We got home yesterday morning at 5 a.m. but missed the first lighter in so had to stay out We got home yesterday morning at 5 a.m. but missed the first lighter in so had to stay out
until 2:30. The girls had prepared a big meal for us and invited up the Hartfords and then let until 2:30. The girls had prepared a big meal for us and invited up the Hartfords and then let
us talk. Miss Saville talked quite a bit. Any how if you folks don't like this I don't care, it is us talk. Miss Saville talked quite a bit. Any how if you folks don't like this I don't care, it is
all I had to write about and I know Buster'd [sic] listen anyway and I'd soak ole Peter's head if he all I had to write about and I know Buster'ud listen anyway and I'd soak ole Peter's head if he
didn't and Polly would in my lap and I don't know much about the youngest one of yours so didn't and Polly would in my lap and I don't know much about the youngest one of yours so
likely he would be squawling. But we did surely enjoy our trip and were gone just long enuf [sic]. likely he would be squawling. But we did surely enjoy our trip and were gone just long enuf.
I expect there were 150 passengers on board and almost or more of the crew and helpers. We I expect there were 150 passengers on board and almost or more of the crew and helpers. We
had a stateroom down next to the kitchen and 'twas pretty fierce for odor at times. had a stateroom down next to the kitchen and 'twas pretty fierce for odor at times.
@@ -109,7 +108,7 @@ Ome
Reprinted from Cochran Chronicles, Volume 9, Number 1, November 1986 Reprinted from Cochran Chronicles, Volume 9, Number 1, November 1986
© [inserted: JECFA] 1986 © JECFA 1986
Up Up
@@ -4,28 +4,28 @@ model: google/gemini-2.5-flash
--- ---
JOHN ISBILL JOHN ISBILL
R. T. MOSER R. T. MOSER
ISBILL & MOSER ISBILL & MOSER
DEALERS IN DEALERS IN
GENERAL MERCHANDISE GENERAL MERCHANDISE
Vonore, Tenn., Jany 27- 1913 Vonore, Tenn. January 27 1913
Dear Much Aunt Louie Dear Uncle Aunt Adeline
How are you a Was at home a
few nights ago I sewed a few nights ago & saw a
letter from your folks, so letter from your folks, So
I decided to write you I decided to write you
a few lines myself ok a few lines myself ok
I am contemplateing a I am contemplate a
trip out west next summer trip out west next summer
& I want Some Olders to go & [inserted: I] would like of adders [sic] to go
where I and them. where I [inserted: am] them.
Am getting
I am getting
up in years & unmarried up in years & unmarried
so you see the object of so you see the object of
my trip, is to get a bunch my trip is to get a wife
of Young & old maids & if there is any old maid
& widows out there. I or widows out there, I
want you to kiss them want you to kiss them
at my fans [sic] mug as they at my [inserted: mind] for me at there [sic]
as soon as I get there as soon as I get there.
+90
View File
@@ -0,0 +1,90 @@
"""Tests for Step 3 library services (revisions, search, export)."""
from sqlmodel import select
import pytest
from transcription.models import Document, Job, JobStatus, Transcript, TranscriptRevision
from transcription.services.library import accept_revision, add_revision, export_transcripts, list_revisions, search_accepted_transcripts
def _create_job(session) -> Job:
document = Document(filename="letter.jpg", file_path="uploads/letter.jpg")
session.add(document)
session.commit()
session.refresh(document)
job = Job(document_id=document.id, status=JobStatus.TRANSCRIBED)
session.add(job)
session.commit()
session.refresh(job)
return job
@pytest.mark.integration
class TestRevisionHistoryBehavior:
"""Verify revision append/accept behavior."""
def test_add_revision_appends_incrementing_revision_numbers(self, session):
"""add_revision creates immutable incrementing revisions per job."""
job = _create_job(session)
r1 = add_revision(job_id=job.id, text="first", session=session)
r2 = add_revision(job_id=job.id, text="second", session=session)
revisions = list_revisions(job_id=job.id, session=session)
assert r1.revision_number == 1
assert r2.revision_number == 2
assert [revision.revision_number for revision in revisions] == [1, 2]
def test_accept_revision_marks_selected_revision_and_sets_job_completed(self, session):
"""accept_revision marks one revision accepted and transitions job to completed."""
job = _create_job(session)
r1 = add_revision(job_id=job.id, text="v1", session=session)
r2 = add_revision(job_id=job.id, text="v2", session=session)
accepted = accept_revision(revision_id=r2.id, session=session)
session.refresh(job)
all_revisions = list(session.exec(select(TranscriptRevision).where(TranscriptRevision.job_id == job.id)).all())
accepted_flags = {revision.id: revision.accepted for revision in all_revisions}
transcript = session.exec(select(Transcript).where(Transcript.job_id == job.id)).first()
assert accepted.id == r2.id
assert accepted_flags[r1.id] is False
assert accepted_flags[r2.id] is True
assert job.status == JobStatus.COMPLETED
assert transcript is not None
assert transcript.text == "v2"
@pytest.mark.integration
class TestSearchAndExportBehavior:
"""Verify accepted-only search and export semantics."""
def test_search_returns_only_accepted_revisions(self, session):
"""search_accepted_transcripts filters out non-accepted revisions."""
job = _create_job(session)
draft = add_revision(job_id=job.id, text="family archive draft", session=session)
accepted = add_revision(job_id=job.id, text="family archive final", accepted=True, session=session)
results = search_accepted_transcripts(query="archive", session=session)
assert results
result_ids = {result.id for result in results}
assert accepted.id in result_ids
assert draft.id not in result_ids
def test_export_returns_serializable_records_for_accepted_revisions(self, session):
"""export_transcripts returns expected fields for accepted-only export."""
job = _create_job(session)
accepted = add_revision(job_id=job.id, text="export me", accepted=True, session=session)
records = export_transcripts(accepted_only=True, session=session)
assert len(records) == 1
record = records[0]
assert record["job_id"] == str(job.id)
assert record["revision_id"] == str(accepted.id)
assert record["accepted"] is True
assert record["text"] == "export me"
+12 -3
View File
@@ -8,7 +8,7 @@ from sqlmodel import select
from transcription.config import Settings from transcription.config import Settings
from transcription.errors import AppError, ErrorCategory from transcription.errors import AppError, ErrorCategory
from transcription.models import Document, Job, JobStatus, Transcript from transcription.models import Document, Job, JobStatus, Transcript, TranscriptRevision
from transcription.providers.base import TranscriptionResult from transcription.providers.base import TranscriptionResult
from transcription.worker import process_next_queued_job, run_worker_loop from transcription.worker import process_next_queued_job, run_worker_loop
@@ -74,12 +74,21 @@ class TestWorkerSuccessPath:
process_next_queued_job(session=session) process_next_queued_job(session=session)
transcript = session.exec( transcript = session.exec(select(Transcript).where(Transcript.job_id == job.id)).first()
select(Transcript).where(Transcript.job_id == job.id) revision = session.exec(
select(TranscriptRevision)
.where(TranscriptRevision.job_id == job.id)
.order_by(TranscriptRevision.revision_number)
).first() ).first()
assert transcript is not None assert transcript is not None
assert transcript.text == "Transcript body" assert transcript.text == "Transcript body"
assert transcript.error_detail is None assert transcript.error_detail is None
assert revision is not None
assert revision.revision_number == 1
assert revision.text == "Transcript body"
assert revision.source == "worker"
assert revision.accepted is False
@pytest.mark.integration @pytest.mark.integration
+40 -1
View File
@@ -5,7 +5,7 @@ from uuid import UUID
import pytest import pytest
from sqlalchemy.exc import IntegrityError from sqlalchemy.exc import IntegrityError
from transcription.models import Document, Job, JobStatus, Transcript from transcription.models import Document, Job, JobStatus, Transcript, TranscriptRevision
def _make_document(**overrides) -> Document: def _make_document(**overrides) -> Document:
@@ -89,6 +89,12 @@ class TestJobModel:
session.refresh(job) session.refresh(job)
assert job.status == JobStatus.TRANSCRIBED assert job.status == JobStatus.TRANSCRIBED
job.status = JobStatus.COMPLETED
session.add(job)
session.commit()
session.refresh(job)
assert job.status == JobStatus.COMPLETED
def test_transitions_to_failed(self, session): def test_transitions_to_failed(self, session):
"""Status updates from processing to failed.""" """Status updates from processing to failed."""
doc = _persist_document(session) doc = _persist_document(session)
@@ -152,6 +158,27 @@ class TestTranscriptModel:
session.commit() session.commit()
class TestTranscriptRevisionModel:
"""Verify transcript revision persistence and defaults."""
def test_revision_defaults_and_persistence(self, session):
"""Revision records persist with revision metadata and defaults."""
doc = _persist_document(session)
job = _persist_job(session, doc)
revision = TranscriptRevision(job_id=job.id, revision_number=1, text="Rev text")
session.add(revision)
session.commit()
session.refresh(revision)
fetched = session.get(TranscriptRevision, revision.id)
assert fetched is not None
assert fetched.revision_number == 1
assert fetched.text == "Rev text"
assert fetched.source == "worker"
assert fetched.accepted is False
class TestRelationships: class TestRelationships:
"""Verify SQLModel relationship navigation between models.""" """Verify SQLModel relationship navigation between models."""
@@ -177,3 +204,15 @@ class TestRelationships:
assert job.transcript is not None assert job.transcript is not None
assert isinstance(job.transcript, Transcript) assert isinstance(job.transcript, Transcript)
assert job.transcript.text == "Transcribed text" assert job.transcript.text == "Transcribed text"
def test_job_exposes_revisions(self, session):
"""job.revisions returns revision history linked to the Job."""
doc = _persist_document(session)
job = _persist_job(session, doc)
session.add(TranscriptRevision(job_id=job.id, revision_number=1, text="v1"))
session.add(TranscriptRevision(job_id=job.id, revision_number=2, text="v2", accepted=True))
session.commit()
session.refresh(job)
assert len(job.revisions) == 2
assert all(isinstance(revision, TranscriptRevision) for revision in job.revisions)
+6
View File
@@ -19,14 +19,17 @@ MVP_REQUIREMENT_TEST_MAP: dict[str, list[str]] = {
"REQ-3": [ "REQ-3": [
"tests/services/test_worker.py", "tests/services/test_worker.py",
"tests/ui/test_jobs_page.py", "tests/ui/test_jobs_page.py",
"tests/services/test_library.py",
], ],
"REQ-4": [ "REQ-4": [
"tests/services/test_worker.py", "tests/services/test_worker.py",
"tests/integration/test_pipeline_flow.py", "tests/integration/test_pipeline_flow.py",
"tests/services/test_library.py",
], ],
"REQ-5": [ "REQ-5": [
"tests/ui/test_jobs_page.py", "tests/ui/test_jobs_page.py",
"tests/ui/test_pages_registration.py", "tests/ui/test_pages_registration.py",
"tests/api/test_routes.py",
], ],
"REQ-6": [ "REQ-6": [
"tests/test_app.py", "tests/test_app.py",
@@ -36,6 +39,9 @@ MVP_REQUIREMENT_TEST_MAP: dict[str, list[str]] = {
"tests/test_app.py", "tests/test_app.py",
"tests/test_config.py", "tests/test_config.py",
], ],
"REQ-11": [
"tests/services/test_library.py",
],
"REQ-12": [ "REQ-12": [
"tests/test_prompts.py", "tests/test_prompts.py",
"tests/services/test_transcription.py", "tests/services/test_transcription.py",