bbchops/transcription
1
0
Fork 0
generated from john/python-template
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

Revised and simplified V4 Plan and core documents.

Browse Source
This commit is contained in:
Jim Lancaster
2026-08-10 10:53:13 -05:00
parent 9b4d6f0340
commit 4b3baf5a3e
7 changed files with 495 additions and 578 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ver4/requirements_v4.md
+36 -55
View File
@@ -1,70 +1,51 @@
# Relationship and Document Type Governance Requirements (Version 4)
# Document Transcription System Requirements (Version 4)
This document defines Version 4 baseline requirements for expanding relationships between `Document` and `Person` and introducing governed document type classification.
V4 preserves all applicable V3 capabilities and adds role extensibility, assisted suggestion workflows, explicit relationship policy enforcement, and registry-driven `Document` type governance.
This document defines the baseline requirements for the document transcription system.
## Requirements Model
| ID | Category | Requirement | Verify Method |
| --- | --- | --- | --- |
| REQ-0 | System | Provide end-to-end, reviewable, policy-enforced document-person relationship management supporting asserted and suggested links. | demonstration |
| REQ-1 | Functional | Preserve many-to-many `Document` ↔ `Person` linking, and allow multiple people per role on a single document. | test |
| REQ-2 | Functional | Support an extensible role taxonomy for document-person relationships beyond fixed `author`/`recipient`. | inspection |
| REQ-3 | Functional | Represent relationship assertion state explicitly (`asserted` and `suggested`) and keep machine suggestions non-canonical until human acceptance. | test |
| REQ-4 | Functional | Provide assisted suggestion lifecycle operations: create/list/filter suggestions, accept suggestion, reject suggestion, and promote accepted suggestions to asserted links. | test |
| REQ-5 | Policy Constraint | Enforce a role exclusivity matrix for a single `(document_id, person_id)` pair on asserted links; initial rule set must block `author` + `recipient` coexistence while allowing `mentioned` to coexist with other roles. | test |
| REQ-6 | Data Constraint | Store canonical asserted links in `document_person` and lifecycle-managed suggestions in a separate `document_person_suggestion` table. | test |
| REQ-7 | Functional | Provide set-based synchronization behavior for relationship mutations (add/remove delta), replacing single-value replacement patterns that can drop unrelated links. | test |
| REQ-8 | Interface | Render grouped relationship metadata by role and assertion state on document detail and person detail views. | demonstration |
| REQ-9 | Interface | Document create/edit UI must support selecting multiple people per role and reviewing pending suggestions with explicit accept/reject controls. | demonstration |
| REQ-10 | API Constraint | Expose additive, role-aware and state-aware retrieval/filtering in API contracts for documents, people, and relationship records. | test |
| REQ-11 | Data Provenance | Capture suggestion provenance metadata sufficient for operator review (for example source mechanism, confidence, and evidence reference) without mutating canonical asserted links implicitly. | inspection |
| REQ-12 | Operations | Provide migration/backfill validation that identifies and resolves historical records violating newly enforced exclusivity policies before hard enforcement. | test |
| REQ-13 | Revision Upgrade | Support deterministic in-place revision upgrade behavior by mapping existing author/recipient links into V4 asserted semantics without requiring long-lived runtime legacy compatibility layers. | test |
| REQ-14 | Reliability | Ensure document/person deletion and cleanup workflows remain safe and deterministic with expanded relationship semantics, including suggestion records. | test |
| REQ-15 | Quality | Add automated test coverage for role extensibility, exclusivity enforcement, suggestion lifecycle transitions, and regression scenarios across service/API/UI flows. | test |
| REQ-16 | Functional | Support a registry-driven document type taxonomy (`document_type`) with stable machine-readable codes, mutable display labels, and active/inactive lifecycle control. | test |
| REQ-17 | Data Constraint | Replace unconstrained free-text document type assignment with controlled type references or deterministic code mapping governed by the document type registry. | test |
| REQ-18 | Interface | Document create/edit UI must present type selection from active registry entries and prevent invalid type assignment. | demonstration |
| REQ-19 | API Constraint | Provide additive API contracts for document type catalog retrieval, including active-only filtering and stable code-based selection for document writes. | test |
| REQ-20 | Operations | Provide deterministic one-time manual normalization/backfill for existing document type strings in the current small corpus, assigning each document to a canonical registry type before strict write enforcement. | test |
| REQ-0 | System | Provide end-to-end multi-page document transcription with persistent, inspectable async job states. | demonstration |
| REQ-1 | Functional | Allow users to upload one or more images as ordered `Source` pages under a `Document`. | test |
| REQ-2 | Functional | Process page transcription asynchronously using an `asyncio` worker pool bounded by rate limits. | test |
| REQ-3 | Functional | Persist submission-time prompt configuration and full page-level provider response evidence for every job execution. | test |
| REQ-4 | Functional | Support job states `queued`, `processing`, `completed`, `partial_success`, and `failed`, plus page states `pending`, `transcribed`, and `failed`. | inspection |
| REQ-5 | Functional | Allow users to manage historical `Person` records and link multiple people per role to a `Document`. | test |
| REQ-6 | Functional | Support an extensible role taxonomy for document-person relationships. | inspection |
| REQ-7 | Policy Constraint | Enforce a role exclusivity matrix for a single `(document_id, person_id)` pair; initial rules must block `author` + `recipient` coexistence while allowing `mentioned` to coexist with other roles. | test |
| REQ-8 | Functional | Use set-based synchronization for document-person mutations so updates add and remove only the intended links. | test |
| REQ-9 | Functional | Maintain immutable machine output on `Source.raw_transcription` while permitting inline human edits on `Source.revised_text`. | test |
| REQ-10 | Functional | Support a registry-driven `DocumentType` taxonomy with stable codes, mutable labels, and active/inactive lifecycle control. | test |
| REQ-11 | Data Constraint | Store `Document` type as a controlled reference to `DocumentType`. | test |
| REQ-12 | Interface | Render multi-page transcriptions sequentially by `page_number` with document, people, and document-type metadata. | demonstration |
| REQ-13 | Interface | Document create/edit UI must support selecting multiple people per role and selecting an active document type from the registry. | demonstration |
| REQ-14 | API Constraint | Expose additive, role-aware retrieval and write behavior for document-person links and code-based selection for document types. | test |
| REQ-15 | Data Constraint | Calculate and store cryptographic file hashes (SHA-256) and file sizes for uploaded source images. | test |
| REQ-16 | Data Constraint | Preserve a portable relational model across supported backends using SQLModel, SQLAlchemy, SQLite, and PostgreSQL. | inspection |
| REQ-17 | Reliability | Ensure delete and update flows for documents, people, and relationship links remain deterministic and safe. | test |
| REQ-18 | Operations Constraint | Keep canonical development, testing, restore, and recovery workflows OS-independent. | inspection |
| REQ-19 | Quality | Provide automated coverage for async transcription workflows, relationship-role enforcement, document-type selection, and regression behavior. | test |
## Clarifying Constraints
1. Suggestions are advisory only until accepted by a human operator.
2. Relationship acceptance/rejection must be explicit and auditable.
3. Role-policy enforcement must occur consistently across service and API boundaries.
4. Many-per-role behavior is required for both asserted and suggested states where applicable.
5. Suggestions that would violate asserted exclusivity may exist as `pending`, but acceptance must fail until the exclusivity conflict is resolved.
6. Document type codes are stable identifiers; display labels may evolve without changing canonical type identity.
## Assumptions
1. V4 scope is limited to document-person relationship expansion and does not redesign the core transcription job execution model.
2. Existing V3 data remains the starting corpus and is transformed via deterministic backfill/validation rules.
3. V4 uses a role registry model and separate suggestion storage; enum-first role expansion is out of scope for this revision.
4. V4 introduces document type registry governance and deprecates unconstrained free-text typing as an authoring-time default.
1. `DocumentType.code` and `PersonRole.code` are stable machine identifiers.
2. `DocumentType.label` and `PersonRole.label` may evolve without changing canonical identity.
3. Role-policy enforcement must be consistent across UI, API, services, and persistence.
4. Many-per-role behavior is required for document-person links.
5. Relationship conflicts must fail deterministically without partial mutation.
## Element Satisfaction Mapping
- **UI (NiceGUI):** Satisfies REQ-0, REQ-1, REQ-3, REQ-4, REQ-8, REQ-9, REQ-18.
- **API (FastAPI):** Satisfies REQ-0, REQ-3, REQ-4, REQ-5, REQ-7, REQ-10, REQ-19.
- **PERSISTENCE (SQLModel/SQLAlchemy):** Satisfies REQ-1, REQ-2, REQ-5, REQ-6, REQ-11, REQ-12, REQ-13, REQ-14, REQ-16, REQ-17, REQ-20.
- **SERVICES (Domain Layer):** Satisfies REQ-4, REQ-5, REQ-7, REQ-10, REQ-14, REQ-17, REQ-20.
- **TEST SUITE:** Satisfies REQ-15 and verifies all test-marked requirements.
## Change Classification vs V3
- **Semantic expansion:** role model and assertion state model.
- **Policy expansion:** exclusivity matrix enforcement.
- **Workflow expansion:** suggestion review and promotion lifecycle.
- **Contract expansion:** role/state-aware read and write behavior.
- **Governance expansion:** registry-managed document type taxonomy and normalization workflow.
- UI (NiceGUI): Satisfies REQ-0, REQ-1, REQ-5, REQ-9, REQ-12, REQ-13.
- API (FastAPI): Satisfies REQ-1, REQ-4, REQ-5, REQ-7, REQ-8, REQ-14.
- Worker (`asyncio`): Satisfies REQ-2, REQ-3, REQ-4.
- Persistence (SQLModel / SQLAlchemy): Satisfies REQ-3, REQ-9, REQ-10, REQ-11, REQ-15, REQ-16, REQ-17.
- Test Suite: Verifies all test-marked requirements and satisfies REQ-19.
## Related Local References
- [V4 Scope Boundary](scope_boundary_v4.md)
- [System Overview V3](../index_v3.md)
- [System Requirements V3](../requirements_v3.md)
- [Data Model V3](../schema_v3.md)
- [System Overview](index_v4.md)
- [System Architecture](architecture_v4.md)
- [Data Model](schema_v4.md)
- [Error Handling Policy](error_handling_v4.md)