bbchops/transcription
1
0
Fork 0
generated from john/python-template
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

V4 plan created: Adding many-to-many links between Documents & People in the UI. Also adding some new tables for Document Type, Person role.

Browse Source
This commit is contained in:
Jim Lancaster
2026-08-09 13:29:50 -05:00
parent e6549277c6
commit 5753eb0135
13 changed files with 1356 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ver4/architecture_v4.md
+137
View File
@@ -0,0 +1,137 @@
# System Architecture (Version 4)
This document describes the V4 architecture changes for expanded `Document`-`Person` relationship management.
V4 extends V3 with role extensibility, assisted suggestion review, and role-policy enforcement while preserving the existing transcription execution topology.
## Architecture Objectives
- Keep V3 transcription pipeline behavior stable unless relationship evidence extraction requires additive integration.
- Support many people per role for each document.
- Support extensible relationship roles without hardcoding UI and API behavior to two values.
- Support non-canonical suggestion intake with explicit human promotion to canonical asserted links.
- Enforce role exclusivity policy consistently at service and persistence boundaries.
- Preserve auditability for suggestion provenance and review actions.
## Runtime Topology
V4 keeps the existing runtime shape:
- FastAPI + NiceGUI web app process.
- Async service layer with SQLModel/SQLAlchemy persistence.
- Existing worker execution path for transcription jobs.
Additive V4 components:
- Relationship policy evaluator (exclusivity checks).
- Suggestion lifecycle service (propose/list/accept/reject).
- Optional suggestion extraction adapter (rule/NLP or LLM-backed).
^^^mermaid
flowchart LR
U[Browser User] --> UI[NiceGUI Pages]
UI --> API[FastAPI Routes]
API --> DS[Document Service]
API --> SS[Suggestion Service]
DS --> PE[Policy Evaluator]
SS --> PE
DS --> DB[(Relational DB)]
SS --> DB
W[Async Worker Engine] --> DB
W --> E[Optional Suggestion Extraction]
E --> SS
^^^
## Layer Responsibilities
### UI Layer (`src/transcription/ui/**`)
- Render per-role grouped relationships for document and person views.
- Provide multi-select role assignment controls in document create/edit flows.
- Provide suggestion review controls (accept/reject) for pending suggestions.
- Surface policy conflict errors from API/service layer clearly.
### API Layer (`src/transcription/api/**`)
- Expose role-aware and state-aware read contracts.
- Expose suggestion lifecycle write contracts.
- Return deterministic validation/conflict errors for exclusivity and duplicate semantics.
- Evolve endpoints additively, with explicit deprecations and short-lived transition windows.
### Service Layer (`src/transcription/services/**`)
- Implement set-based relationship sync (delta add/remove) to avoid destructive replacement behavior.
- Apply policy evaluator before persistence commits.
- Orchestrate suggestion acceptance/rejection transitions.
- Ensure accepted suggestions result in asserted link creation/confirmation.
### Policy Evaluator (New logical component)
- Evaluate role exclusivity matrix for `(document_id, person_id)` writes.
- Provide deterministic conflict reason payloads suitable for UI/API display.
- Stay stateless and reusable across create/update/sync code paths.
### Persistence Layer (`src/transcription/db/**`)
- Store asserted links and suggestion records according to selected schema option.
- Persist provenance metadata for suggestion records.
- Enforce uniqueness and support performant role/state filtering.
## Core V4 Workflows
### 1) Manual Relationship Management
1. User opens document edit view.
2. UI loads asserted links grouped by role.
3. User adds/removes people per role.
4. Service computes delta and runs policy checks.
5. Persistence applies adds/removes atomically.
### 2) Assisted Suggestion Review
1. Suggestion records are created (`pending`) by extraction logic or manual propose action.
2. UI displays pending suggestions with evidence metadata.
3. User accepts or rejects each suggestion.
4. Accept path creates/confirms asserted relationship and marks suggestion `accepted`.
5. Reject path marks suggestion `rejected`.
### 3) Exclusivity Conflict Handling
1. Incoming write attempts role assignment.
2. Policy evaluator checks role pair conflicts for same `(document_id, person_id)`.
3. If conflict exists, write is rejected with structured conflict details.
4. UI presents actionable message without partial updates.
## Invariants
- Suggested links are never canonical until accepted.
- Asserted links must satisfy exclusivity rules.
- Relationship mutations are set-based and deterministic.
- Relationship views remain consistent between document detail and person detail pages.
- Existing V3 links remain valid under asserted semantics after migration.
## Backward Compatibility
- V3 author/recipient links are interpreted as asserted links in V4.
- Existing document and person flows continue to function where unaffected by new role/state dimensions.
- During development revisions, the current API contract is authoritative; long-lived legacy runtime compatibility layers are not required.
## Observability and Auditability
- Persist suggestion provenance fields sufficient for operator review.
- Record review decision outcomes (`accepted`/`rejected`) with timestamps and reviewer identity where available.
- Emit structured service-level logs for exclusivity conflicts and review actions.
## Test Strategy Additions
- Service tests for delta sync and exclusivity matrix enforcement.
- API tests for role/state filtering and suggestion transitions.
- UI tests for multi-role selection and suggestion review interactions.
- Migration tests for V3-to-V4 asserted mapping and conflict scans.
## Related Local References
- [V4 Scope Boundary](scope_boundary_v4.md)
- [V4 Requirements](requirements_v4.md)
- [V4 Schema](schema_v4.md)
- [V3 Architecture](../architecture_v3.md)
docs/ver4/error_handling_v4.md
+104
View File
@@ -0,0 +1,104 @@
# Error Handling Policy (Version 4)
This document defines canonical error-handling behavior for V4 document-person relationship expansion.
V4 keeps V3 transcription error behavior and adds policy/conflict handling for role extensibility, suggestion lifecycle transitions, and exclusivity enforcement.
## Error Handling Objectives
- Provide clear, actionable conflict and validation feedback for relationship write operations.
- Prevent partial, silent, or destructive relationship mutations when policy checks fail.
- Preserve suggestion review auditability with deterministic accept/reject outcomes.
- Keep consistent API/UI/service error envelopes across relationship workflows.
## Scope and Authority
Governs relationship-related error behavior in:
- NiceGUI document/person relationship views,
- FastAPI relationship and suggestion endpoints,
- domain services for relationship sync and suggestion review,
- persistence constraints for role, state, and exclusivity invariants.
## Relationship Error Taxonomy
| Category | Definition | Retriable |
| --- | --- | --- |
| `validation_error` | Payload shape/type invalid, unknown role/state, malformed IDs | no |
| `not_found_error` | Target `Document`, `Person`, role, or suggestion record does not exist | no |
| `conflict_error` | Write violates uniqueness or exclusivity policy | no |
| `suggestion_state_error` | Invalid suggestion transition (for example accept after reject) | no |
| `policy_violation_error` | Action blocked by configured role matrix or governance rule | no |
| `infrastructure_transient_error` | Temporary DB or network instability during relationship operation | yes |
| `infrastructure_persistent_error` | Persistent DB/configuration failure | no |
| `internal_unexpected_error` | Unhandled exception/logic defect | no |
## Deterministic Conflict Behavior
When relationship writes fail policy checks:
1. Reject the full write operation (no partial apply).
2. Return structured conflict details including conflicting role pair and target identifiers.
3. Preserve existing canonical relationships unchanged.
When suggestion transitions fail:
1. Reject invalid state transition.
2. Return current state and allowed next actions.
3. Preserve suggestion record integrity.
## API Error Response Contract
Relationship endpoints return a structured envelope:
^^^json
{
"error_id": "err_uuid_12345",
"category": "conflict_error",
"message": "Role assignment violates exclusivity policy.",
"suggestion": "Remove recipient role before assigning author for this person on this document.",
"details": {
"document_id": "...",
"person_id": "...",
"attempted_role": "author",
"conflicting_role": "recipient",
"policy_rule": "author+recipient exclusive"
},
"timestamp": "2026-08-09T15:00:00Z"
}
^^^
HTTP status mappings:
- `validation_error` -> `400`
- `not_found_error` -> `404`
- `conflict_error`, `suggestion_state_error`, `policy_violation_error` -> `409`
- `infrastructure_transient_error` -> `503`
- `infrastructure_persistent_error`, `internal_unexpected_error` -> `500`
## UI Error Presentation Rules
- Display concise conflict summary with actionable next step.
- Keep user edits in context (do not discard form state when feasible).
- Differentiate between validation issues, policy conflicts, and infrastructure failures.
- For bulk role sync operations, show per-item conflict context when multiple failures occur.
## Logging and Audit Expectations
- Log relationship write failures with correlation IDs.
- Log suggestion acceptance/rejection outcomes with actor and timestamp where available.
- Log policy matrix violations with deterministic machine-readable context.
## Relationship-Specific Retry Guidance
- Do not auto-retry policy or conflict failures.
- Permit user-driven retry only after input changes.
- Retry infrastructure transient failures with bounded policy in service layer if operation is idempotent.
## Related Local References
- [V4 Scope Boundary](scope_boundary_v4.md)
- [V4 Requirements](requirements_v4.md)
- [V4 Schema](schema_v4.md)
- [V4 Architecture](architecture_v4.md)
- [V3 Error Handling](../error_handling_v3.md)
docs/ver4/implementation_plan_v4.md
+107
View File
@@ -0,0 +1,107 @@
# Implementation Plan (Version 4)
## Goal
Implement V4 document-person relationship expansion and document type governance with extensible registries, assisted suggestion lifecycle, and policy-enforced exclusivity while preserving V3 transcription behavior.
## Current Project Impact
- `src/transcription/db/models.py` will require relationship schema evolution for role extensibility and suggestion lifecycle support.
- `src/transcription/db/models.py` will require document type registry entities and document type reference updates.
- `src/transcription/services/documents.py` will require set-based relationship sync and policy checks.
- `src/transcription/services/documents.py` will require registry-based document type lookup, validation, and normalization helpers.
- API modules under `src/transcription/api/**` will require role/state-aware contracts and suggestion lifecycle endpoints.
- API modules under `src/transcription/api/**` will require additive document type catalog and code-based selection contracts.
- UI pages under `src/transcription/ui/pages/**` will require multi-role, multi-person editing, suggestion review controls, and registry-backed type selectors.
- Existing tests under `tests/services`, `tests/api`, and `tests/ui` need expanded coverage for V4 behavior and regression safety.
## Implementation Phases
### 1. Finalize V4 Schema Decisions
- Role extensibility mechanism: use role registry tables.
- Document type extensibility mechanism: use `document_type` registry tables.
- Suggestion storage model: use a separate `document_person_suggestion` table.
- Exclusivity baseline: `author` vs `recipient` exclusive, `mentioned` non-exclusive.
- Enforce exclusivity on asserted links; evaluate conflicts on suggestion acceptance.
### 2. Evolve Persistence Layer
- Implement selected schema model in SQLModel.
- Add constraints and indexes for dedupe, filtering, and policy support.
- Add migration/backfill logic for V3 links to V4 asserted semantics.
- Add conflict scan tooling for historical records violating exclusivity rules.
- Perform a one-time manual mapping of existing document type values to registry-backed type references (`document_type_id`) for the current small corpus.
### 3. Implement Service-Layer Policy and Sync Semantics
- Add relationship delta sync operations (set-based add/remove).
- Implement centralized policy evaluator for exclusivity checks.
- Add suggestion lifecycle operations (propose/list/accept/reject).
- Ensure accepted suggestions create/confirm asserted links atomically.
- Add document type resolution operations by stable `code` and active/inactive state handling.
### 4. Implement API Contract Changes
- Add role-aware and state-aware query parameters/filters.
- Add suggestion lifecycle endpoints and response models.
- Add deterministic conflict/error payloads aligned to V4 error policy.
- Use additive endpoint evolution for V4; during development mode, deprecate then remove without maintaining long-lived legacy runtime compatibility layers.
- Add document type catalog endpoints with active-only filtering and code-based selection for document writes.
### 5. Update UI Workflows
- Replace single-author controls with grouped multi-role selectors.
- Replace unconstrained document type free-text entry with registry-backed type selection.
- Add pending suggestion review panel and accept/reject actions.
- Update document/person detail cards to group links by role and state.
- Preserve edit-state ergonomics on validation/conflict failures.
### 6. Verification and Hardening
- Add service tests for:
- many-per-role behavior,
- exclusivity enforcement,
- set-based sync correctness,
- suggestion transition validity,
- document type code resolution and inactive-type handling.
- Add API tests for role/state filtering and conflict response shapes.
- Add API tests for document type catalog retrieval and code-based write validation.
- Add UI tests/manual walkthroughs for create/edit/review workflows.
- Add UI tests/manual walkthroughs for registry-backed document type selection and validation messaging.
- Add regression tests for document/person delete cleanup semantics.
## Done When
- V4 relationship schema and contracts are implemented and validated.
- V4 document type registry schema and contracts are implemented and validated.
- Suggestions remain non-canonical until explicit acceptance.
- Exclusivity policy is enforced deterministically across service/API boundaries.
- Existing V3 links are migrated to V4 asserted semantics without data loss.
- Existing document type strings are manually normalized to registry references for the current corpus.
- Test suite includes V4-specific coverage and passes on supported backends.
## Out of Scope
- Automatic acceptance of suggestions.
- Global person entity-resolution/merge engine.
- Core transcription execution redesign unrelated to relationship expansion.
- Automated semantic document type classification.
## Delivery Order Recommendation
1. Requirements freeze (`requirements_v4.md`).
2. Schema decision freeze (`schema_v4.md`).
3. Error policy freeze (`error_handling_v4.md`).
4. Implementation of persistence and service layer.
5. API and UI changes.
6. Final integration and regression validation.
## Related Local References
- [V4 Scope Boundary](scope_boundary_v4.md)
- [V4 Requirements](requirements_v4.md)
- [V4 Schema](schema_v4.md)
- [V4 Architecture](architecture_v4.md)
- [V4 Error Handling](error_handling_v4.md)
- [V3 Implementation Plan](../implementation_plan_v3.md)
docs/ver4/index_v4.md
+49
View File
@@ -0,0 +1,49 @@
# Document-Person Expansion Overview (Version 4)
Version 4 defines the relationship-model evolution track for linking `Document` and `Person` entities with extensible roles, suggestion review, and policy enforcement.
## Start Here
Read [scope_boundary_v4.md](scope_boundary_v4.md) first to confirm scope and non-scope before implementation work begins.
## Core V4 Capabilities
- Extensible relationship role taxonomy.
- Many-people-per-role linking for documents.
- Explicit distinction between canonical asserted links and pending suggested links.
- Human-in-the-loop suggestion accept/reject workflow.
- Role exclusivity policy enforcement for configured role pairs.
- Role/state-aware API and UI retrieval/presentation behavior.
- Minimal document type governance rollout for the current corpus, with one-time manual mapping and no alias helper table.
## V4 Documentation Index
- [Scope Boundary](scope_boundary_v4.md)
- [System Requirements](requirements_v4.md)
- [Data Model](schema_v4.md)
- [System Architecture](architecture_v4.md)
- [Error Handling Policy](error_handling_v4.md)
- [Implementation Plan](implementation_plan_v4.md)
## Relationship To V3
V3 remains the baseline production architecture and requirements set for transcription pipeline behavior. V4 is an additive evolution track focused on document-person relationship semantics and workflows.
## Decision Status
Locked decisions:
1. Role extensibility uses registry tables.
2. Suggestion storage uses a separate suggestion table.
3. Exclusivity baseline is `author` vs `recipient` exclusive, with `mentioned` non-exclusive.
4. API evolution is additive in development mode with explicit deprecate-then-remove behavior.
Remaining decision:
1. Suggestion generation strategy (deterministic rules/NLP vs LLM extraction).
## Related Local References
- [V3 System Overview](../index_v3.md)
- [V3 Requirements](../requirements_v3.md)
- [V3 Schema](../schema_v3.md)
docs/ver4/requirements_v4.md
+70
View File
@@ -0,0 +1,70 @@
# Relationship and Document Type Governance Requirements (Version 4)
This document defines Version 4 baseline requirements for expanding relationships between `Document` and `Person` and introducing governed document type classification.
V4 preserves all applicable V3 capabilities and adds role extensibility, assisted suggestion workflows, explicit relationship policy enforcement, and registry-driven `Document` type governance.
## Requirements Model
| ID | Category | Requirement | Verify Method |
| --- | --- | --- | --- |
| REQ-0 | System | Provide end-to-end, reviewable, policy-enforced document-person relationship management supporting asserted and suggested links. | demonstration |
| REQ-1 | Functional | Preserve many-to-many `Document` ↔ `Person` linking, and allow multiple people per role on a single document. | test |
| REQ-2 | Functional | Support an extensible role taxonomy for document-person relationships beyond fixed `author`/`recipient`. | inspection |
| REQ-3 | Functional | Represent relationship assertion state explicitly (`asserted` and `suggested`) and keep machine suggestions non-canonical until human acceptance. | test |
| REQ-4 | Functional | Provide assisted suggestion lifecycle operations: create/list/filter suggestions, accept suggestion, reject suggestion, and promote accepted suggestions to asserted links. | test |
| REQ-5 | Policy Constraint | Enforce a role exclusivity matrix for a single `(document_id, person_id)` pair on asserted links; initial rule set must block `author` + `recipient` coexistence while allowing `mentioned` to coexist with other roles. | test |
| REQ-6 | Data Constraint | Store canonical asserted links in `document_person` and lifecycle-managed suggestions in a separate `document_person_suggestion` table. | test |
| REQ-7 | Functional | Provide set-based synchronization behavior for relationship mutations (add/remove delta), replacing single-value replacement patterns that can drop unrelated links. | test |
| REQ-8 | Interface | Render grouped relationship metadata by role and assertion state on document detail and person detail views. | demonstration |
| REQ-9 | Interface | Document create/edit UI must support selecting multiple people per role and reviewing pending suggestions with explicit accept/reject controls. | demonstration |
| REQ-10 | API Constraint | Expose additive, role-aware and state-aware retrieval/filtering in API contracts for documents, people, and relationship records. | test |
| REQ-11 | Data Provenance | Capture suggestion provenance metadata sufficient for operator review (for example source mechanism, confidence, and evidence reference) without mutating canonical asserted links implicitly. | inspection |
| REQ-12 | Operations | Provide migration/backfill validation that identifies and resolves historical records violating newly enforced exclusivity policies before hard enforcement. | test |
| REQ-13 | Revision Upgrade | Support deterministic in-place revision upgrade behavior by mapping existing author/recipient links into V4 asserted semantics without requiring long-lived runtime legacy compatibility layers. | test |
| REQ-14 | Reliability | Ensure document/person deletion and cleanup workflows remain safe and deterministic with expanded relationship semantics, including suggestion records. | test |
| REQ-15 | Quality | Add automated test coverage for role extensibility, exclusivity enforcement, suggestion lifecycle transitions, and regression scenarios across service/API/UI flows. | test |
| REQ-16 | Functional | Support a registry-driven document type taxonomy (`document_type`) with stable machine-readable codes, mutable display labels, and active/inactive lifecycle control. | test |
| REQ-17 | Data Constraint | Replace unconstrained free-text document type assignment with controlled type references or deterministic code mapping governed by the document type registry. | test |
| REQ-18 | Interface | Document create/edit UI must present type selection from active registry entries and prevent invalid type assignment. | demonstration |
| REQ-19 | API Constraint | Provide additive API contracts for document type catalog retrieval, including active-only filtering and stable code-based selection for document writes. | test |
| REQ-20 | Operations | Provide deterministic one-time manual normalization/backfill for existing document type strings in the current small corpus, assigning each document to a canonical registry type before strict write enforcement. | test |
## Clarifying Constraints
1. Suggestions are advisory only until accepted by a human operator.
2. Relationship acceptance/rejection must be explicit and auditable.
3. Role-policy enforcement must occur consistently across service and API boundaries.
4. Many-per-role behavior is required for both asserted and suggested states where applicable.
5. Suggestions that would violate asserted exclusivity may exist as `pending`, but acceptance must fail until the exclusivity conflict is resolved.
6. Document type codes are stable identifiers; display labels may evolve without changing canonical type identity.
## Assumptions
1. V4 scope is limited to document-person relationship expansion and does not redesign the core transcription job execution model.
2. Existing V3 data remains the starting corpus and is transformed via deterministic backfill/validation rules.
3. V4 uses a role registry model and separate suggestion storage; enum-first role expansion is out of scope for this revision.
4. V4 introduces document type registry governance and deprecates unconstrained free-text typing as an authoring-time default.
## Element Satisfaction Mapping
- **UI (NiceGUI):** Satisfies REQ-0, REQ-1, REQ-3, REQ-4, REQ-8, REQ-9, REQ-18.
- **API (FastAPI):** Satisfies REQ-0, REQ-3, REQ-4, REQ-5, REQ-7, REQ-10, REQ-19.
- **PERSISTENCE (SQLModel/SQLAlchemy):** Satisfies REQ-1, REQ-2, REQ-5, REQ-6, REQ-11, REQ-12, REQ-13, REQ-14, REQ-16, REQ-17, REQ-20.
- **SERVICES (Domain Layer):** Satisfies REQ-4, REQ-5, REQ-7, REQ-10, REQ-14, REQ-17, REQ-20.
- **TEST SUITE:** Satisfies REQ-15 and verifies all test-marked requirements.
## Change Classification vs V3
- **Semantic expansion:** role model and assertion state model.
- **Policy expansion:** exclusivity matrix enforcement.
- **Workflow expansion:** suggestion review and promotion lifecycle.
- **Contract expansion:** role/state-aware read and write behavior.
- **Governance expansion:** registry-managed document type taxonomy and normalization workflow.
## Related Local References
- [V4 Scope Boundary](scope_boundary_v4.md)
- [System Overview V3](../index_v3.md)
- [System Requirements V3](../requirements_v3.md)
- [Data Model V3](../schema_v3.md)
docs/ver4/schema_v4.md
+214
View File
@@ -0,0 +1,214 @@
# Database Schema (Version 4)
This document defines the selected schema direction for V4 document-person relationship expansion.
V4 goals are:
- extensible role taxonomy,
- extensible document type taxonomy,
- explicit assertion state (`asserted`, `suggested`),
- policy-driven exclusivity,
- deterministic migration from V3 links.
## Scope
This specification focuses on relationship and document-type governance persistence changes. Existing `Person`, `Source`, `Job`, and `JobSource` core structures remain as in V3 unless explicitly noted.
## New/Expanded Concepts
- **Relationship role:** semantic label such as `author`, `recipient`, `mentioned`.
- **Assertion state:** whether the link is canonical (`asserted`) or pending review (`suggested`).
- **Exclusivity matrix:** configurable role-pair conflicts for same `(document_id, person_id)`.
- **Suggestion provenance:** evidence fields enabling review decisions.
- **Document type registry:** controlled taxonomy for `Document` classification with stable code identity.
## Selected Model: Role Registry + Separate Suggestion Table
This option cleanly separates canonical links from pending suggestions and enables fully data-driven role expansion.
### Tables
#### `person_role`
| Column | Type | Notes |
| --- | --- | --- |
| `id` | UUID PK | Stable key |
| `code` | TEXT UNIQUE | Canonical role code, for example `author`, `recipient`, `mentioned` |
| `label` | TEXT | UI label |
| `is_active` | BOOLEAN | Soft-enable/disable role |
| `created_at` | TIMESTAMPTZ | Audit timestamp |
| `updated_at` | TIMESTAMPTZ | Audit timestamp |
#### `document_person` (asserted links only)
| Column | Type | Notes |
| --- | --- | --- |
| `id` | UUID PK | Stable key |
| `document_id` | UUID FK | -> `document.id` |
| `person_id` | UUID FK | -> `person.id` |
| `role_id` | UUID FK | -> `person_role.id` |
| `created_at` | TIMESTAMPTZ | Audit timestamp |
| `updated_at` | TIMESTAMPTZ | Audit timestamp |
Constraints:
- `UNIQUE(document_id, person_id, role_id)`
#### `document_person_suggestion`
| Column | Type | Notes |
| --- | --- | --- |
| `id` | UUID PK | Stable key |
| `document_id` | UUID FK | -> `document.id` |
| `person_id` | UUID FK | -> `person.id` |
| `role_id` | UUID FK | -> `person_role.id` |
| `status` | TEXT | `pending`, `accepted`, `rejected` |
| `confidence` | FLOAT NULL | Optional score |
| `source_mechanism` | TEXT NULL | For example `rule`, `llm` |
| `evidence_ref` | TEXT NULL | Pointer or excerpt ID |
| `evidence_span` | JSON NULL | Optional text span payload |
| `note` | TEXT NULL | Reviewer note |
| `created_at` | TIMESTAMPTZ | Suggestion creation time |
| `reviewed_at` | TIMESTAMPTZ NULL | Decision time |
| `reviewed_by` | TEXT NULL | Operator identifier |
Constraints:
- `UNIQUE(document_id, person_id, role_id, status)` with policy for multiple pending rows defined in service layer.
- Optional stricter rule: one active pending suggestion per `(document_id, person_id, role_id)`.
#### `role_exclusivity`
| Column | Type | Notes |
| --- | --- | --- |
| `id` | UUID PK | Stable key |
| `left_role_id` | UUID FK | -> `person_role.id` |
| `right_role_id` | UUID FK | -> `person_role.id` |
| `created_at` | TIMESTAMPTZ | Audit timestamp |
Constraints:
- Canonical ordering rule to avoid duplicate pairs (`left_role_id < right_role_id` enforced in service/DB).
- `UNIQUE(left_role_id, right_role_id)`
Initial seed:
- Exclusivity pair: (`author`, `recipient`)
## Deferred Alternative (Not Selected for V4)
An enum-based shared table model was considered but is intentionally not selected for V4 because it couples canonical and provisional states in one table and increases invariant complexity.
## Assertion-State Semantics
- `asserted`: canonical relationship used for document/person metadata and business logic.
- `suggested`: non-canonical proposal requiring explicit review.
- Accept action:
- creates asserted link (or confirms existing),
- marks suggestion `accepted`.
- Reject action:
- marks suggestion `rejected`.
## Exclusivity Enforcement
Policy target:
- For a single `(document_id, person_id)`, disallow coexistence of role pairs configured as exclusive.
Enforcement layers:
1. Service-level pre-check for clear API errors.
2. Database-level guard where feasible (constraints/triggers or deterministic write path).
Initial configured rule:
- `author` and `recipient` are exclusive.
Enforcement semantics:
- Exclusivity is enforced for asserted links.
- Pending suggestions may exist even if they would conflict when asserted.
- Accepting a suggestion must run exclusivity checks and fail deterministically on conflict.
## Migration From V3
### Data Mapping
- Existing V3 `document_person` rows map to V4 `asserted` semantics.
- Existing V3 role values:
- `author` -> role `author`
- `recipient` -> role `recipient`
### Backfill Steps
1. Seed role rows (`author`, `recipient`, `mentioned`) if using Option A.
2. Migrate current links into asserted table/state.
3. Run conflict scan for exclusivity violations.
4. Apply deterministic conflict policy for any violations.
5. Enable hard enforcement after data passes validation.
## Indexing Guidance
Recommended indexes:
- `document_person(document_id)`
- `document_person(person_id)`
- `document_person(role_id)` (Option A) or `document_person(role)` (Option B)
- `document_person_suggestion(document_id, status)` (Option A)
- `document_person_suggestion(person_id, status)` (Option A)
## Selection Rationale
V4 selects role registry plus separate suggestion storage for clearer provenance boundaries, cleaner lifecycle transitions, and long-term extensibility.
## Document Type Registry Model (Selected for V4)
V4 applies the same registry governance pattern to document classification.
### Tables
#### `document_type`
| Column | Type | Notes |
| --- | --- | --- |
| `id` | UUID PK | Stable key |
| `code` | TEXT UNIQUE | Canonical type code, for example `letter`, `diary`, `book`, `postcard` |
| `label` | TEXT | UI display label |
| `is_active` | BOOLEAN | Soft-enable/disable type |
| `sort_order` | INTEGER NULL | Optional UI ordering |
| `created_at` | TIMESTAMPTZ | Audit timestamp |
| `updated_at` | TIMESTAMPTZ | Audit timestamp |
#### `document` update
| Column | Type | Notes |
| --- | --- | --- |
| `document_type_id` | UUID FK NULL | -> `document_type.id` |
### Constraints and Governance
- `document_type.code` must be stable and unique.
- `document_type.label` may change without changing canonical type identity.
- Inactive types remain valid for historical records but are excluded from default create/edit selectors.
### Initial Seeds
- Seed baseline type codes from current V3 usage set (for example `letter`, `diary`, `book`, `postcard`, `record`, `memo`) and refine labels as needed.
### Migration and Normalization (Small Corpus)
1. Seed canonical `document_type` rows.
2. Manually assign each existing document (10 total) to a canonical type via `document_type_id`.
3. Resolve any outlier values directly during this one-time pass.
4. Enforce registry-backed write validation after manual assignment is complete.
### Indexing Guidance (Document Type)
- `document_type(code)` unique index.
- `document(document_type_id)` index.
## Related Local References
- [V4 Scope Boundary](scope_boundary_v4.md)
- [V4 Requirements](requirements_v4.md)
- [V3 Schema](../schema_v3.md)
docs/ver4/scope_boundary_v4.md
+134
View File
@@ -0,0 +1,134 @@
# V4 Scope Boundary (Version 4)
This document defines what is and is not included in Version 4 for expanding `Document`-`Person` relationships.
## Purpose
Create a clear implementation boundary before updating full V4 architecture, requirements, schema, and plan documents.
## Why This Is V4 (Not V3.1)
V4 is required because the change is semantic and cross-cutting:
- Expands relationship meaning beyond fixed `author`/`recipient`.
- Introduces a suggestion lifecycle (`suggested` vs `asserted`).
- Introduces policy constraints (role exclusivity matrix).
- Impacts persistence, service contracts, API behavior, UI workflows, and test strategy.
A V3.1 patch would only be appropriate for non-semantic quality-of-life fixes inside existing role semantics.
## In Scope for V4
### 1) Relationship Semantics
- Role taxonomy becomes extensible (not hardcoded to only two role values).
- A `Document` can link to many `Person` records per role.
- Relationship states are explicit:
- `asserted`: human-confirmed canonical link.
- `suggested`: machine- or heuristic-proposed link pending review.
### 2) Policy Rules
- Enforce role exclusivity for the same `(document, person)` pair where configured.
- Initial policy decision:
- `author` + `recipient` are mutually exclusive.
- `mentioned` may coexist with other roles.
### 3) Persistence and Contracts
- Evolve persistence model to support:
- extensible roles,
- suggestion lifecycle metadata,
- deterministic conflict handling.
- Replace single-value relationship mutation patterns with set-based sync behavior.
### 4) Human-in-the-Loop Workflow
- Add assisted suggestion review flow:
- list suggestions,
- accept,
- reject,
- promote accepted suggestion to asserted link.
- No silent auto-promotion from suggestion to asserted.
### 5) UI and API Behavior
- Document create/edit/detail workflows support multi-person per role.
- UI surfaces grouped role links and suggestion status.
- API supports role-aware retrieval and suggestion lifecycle operations.
### 6) Verification
- Add tests for:
- many-per-role behavior,
- exclusivity enforcement,
- suggestion lifecycle,
- migration/conflict detection,
- regression on delete/link cleanup behavior.
## Out of Scope for V4
- Automatic acceptance of suggested links without human review.
- Full entity resolution/identity merge pipeline across all `Person` records.
- Historical provenance graph redesign beyond relationship-level evidence fields.
- Large-scale NLP research features unrelated to document-person linking.
- Changes to core transcription execution model (`Job`, `JobSource`) except where needed to expose suggestion evidence inputs.
## Locked Design Decisions
### A) Role Extensibility Mechanism
- Adopt role registry tables (data-driven roles).
- Do not use enum-first role expansion for V4.
### B) Suggestion Storage Model
- Adopt a separate `document_person_suggestion` table.
- Keep `document_person` focused on canonical asserted links.
- Revisit only if operational complexity proves materially higher than expected.
### C) Exclusivity Matrix Baseline
- `author` + `recipient` are mutually exclusive for the same `(document, person)` pair.
- `mentioned` remains non-exclusive.
- Hard exclusivity enforcement applies to asserted links.
- Suggestions may be stored even if they would conflict at assert time; promotion to asserted must enforce exclusivity.
### D) API Compatibility Strategy
- Use additive API evolution in V4.
- In development mode, the current revision is authoritative; long-lived legacy compatibility layers are not required.
- Deprecations should be explicit and short-lived, with removals performed in subsequent revisions.
### E) Document Type Rollout Strategy
- Use a minimal registry rollout for the current corpus: no `document_type_alias` helper table.
- Perform a one-time manual mapping of existing document types to canonical registry types.
### F) Suggestion Generation Strategy (Still Open)
- Option A: deterministic rules/NLP over transcribed text.
- Option B: LLM extraction with confidence/evidence spans.
## Compatibility and Rollout
- Existing V3 author links migrate to `asserted` behavior.
- Backfill validation identifies policy conflicts before constraints are enforced.
- Preserve existing V3 core behavior where unaffected by role/suggestion evolution.
## Exit Criteria for Scope Freeze
V4 scope is considered frozen when:
- Suggestion generation strategy is chosen.
- Suggestion acceptance workflow details are approved.
- Additive API change list and deprecation schedule are approved.
## Core V4 Documents (Current Set)
1. `docs/ver4/index_v4.md`
2. `docs/ver4/requirements_v4.md`
3. `docs/ver4/schema_v4.md`
4. `docs/ver4/architecture_v4.md`
5. `docs/ver4/error_handling_v4.md`
6. `docs/ver4/implementation_plan_v4.md`