generated from john/python-template
V4.4 Complete
This commit is contained in:
@@ -11,10 +11,11 @@ Documents manages the archival record for each historical artifact independently
|
||||
| `/documents` | Searchable archival Document list. |
|
||||
| `/documents/new` | Create a Document. |
|
||||
| `/documents/{document_id}` | View one Document and its related records. |
|
||||
| `/documents/{document_id}/edit` | Edit metadata and people-by-role links. |
|
||||
| `/documents/{document_id}/edit` | Edit metadata and the complete Linked People set. |
|
||||
| `/documents/{document_id}/delete` | Confirm or block deletion. |
|
||||
| `/documents/{document_id}/jobs` | Show Jobs belonging to the Document. |
|
||||
| `/documents/{document_id}/sources` | Redirect to the Document-filtered Sources list. |
|
||||
| `/documents/{document_id}/print` | Preview and browser-print the persisted Document. |
|
||||
|
||||
## List Behavior
|
||||
|
||||
@@ -42,7 +43,7 @@ Optional:
|
||||
- Document location.
|
||||
- Archive identifier.
|
||||
- Notes.
|
||||
- Multiple people for every configured Person Role.
|
||||
- Linked People, with exactly one Person Role per linked Person.
|
||||
|
||||
Rules:
|
||||
|
||||
@@ -53,6 +54,11 @@ Rules:
|
||||
- An invalid requested Person produces a warning rather than a broken form.
|
||||
- `return_to=jobs_new` returns a successful create to Job creation with the new Document selected.
|
||||
- Edit includes active and inactive Document Types so historical values remain maintainable.
|
||||
- One Linked People table contains Select, Person, and Role columns.
|
||||
- Add and Edit use an inline Person/Role editor; Save, Cancel, and Delete change staged UI state only.
|
||||
- A Person may appear once per Document regardless of role.
|
||||
- Existing inactive-role links remain visible; only active roles may be newly assigned.
|
||||
- Document fields and the complete staged link set commit atomically on the main save.
|
||||
- Save success returns to Document Detail.
|
||||
|
||||
## Detail Behavior
|
||||
@@ -63,9 +69,20 @@ Rules:
|
||||
- System Logistics shows created and updated timestamps.
|
||||
- Related People are grouped by role and link to Person Detail.
|
||||
- **Sources & Pipeline Jobs** shows counts and actions for filtered Sources, Document Jobs, and adding a Job.
|
||||
- **Edit Document** and **Delete** are available from the header.
|
||||
- **Edit Document**, **Print**, and **Delete** are available from the header.
|
||||
- Invalid IDs and missing Documents produce explicit states without rendering a partial page.
|
||||
|
||||
## Print Behavior
|
||||
|
||||
- Print opens a dedicated preview for persisted Document data.
|
||||
- **Facsimile** places each Source image beside its current transcription and starts every Source on a new printed sheet.
|
||||
- **Text only** omits images, joins single line breaks inside paragraphs, and preserves blank-line paragraph boundaries.
|
||||
- Non-null revised text takes precedence over raw transcription, including an intentionally empty revision.
|
||||
- Archival metadata resolves Author through the hidden built-in semantic identity, not its mutable label.
|
||||
- Job metadata uses one oldest-to-newest column per Job and ends with Status.
|
||||
- Stored text is escaped and Source media uses record-validated application URLs rather than local file paths.
|
||||
- Printing uses the browser print dialog; server-generated PDFs are not provided.
|
||||
|
||||
## Document Jobs Behavior
|
||||
|
||||
- The page lists the Document's Jobs newest first with status and Job ID.
|
||||
@@ -84,10 +101,12 @@ Rules:
|
||||
|
||||
- List columns, alignment, search, sorting, date fallback, and row navigation match this contract.
|
||||
- Create/edit enforce name, registered type, and valid exact-date input.
|
||||
- Multiple people can be selected independently for each configured role.
|
||||
- Linked People staging enforces one role and one row per Person.
|
||||
- Document and Linked People writes never partially commit.
|
||||
- Person-first Document creation preselects the requested Person as author.
|
||||
- Detail links people, Sources, and Jobs to the correct records.
|
||||
- Delete never removes a Document with Source or Job dependencies.
|
||||
- Both print formats preserve the frozen content, ordering, text-precedence, and safety contracts.
|
||||
- Service failures use the shared error presenter and never report false success.
|
||||
|
||||
## Implementation Anchors
|
||||
@@ -96,10 +115,14 @@ Rules:
|
||||
- `src/transcription/ui/components/table/documents.py`
|
||||
- `src/transcription/services/documents.py`
|
||||
- `src/transcription/services/people.py`
|
||||
- `src/transcription/services/workflows.py`
|
||||
- `src/transcription/ui/components/linked_people.py`
|
||||
- `src/transcription/ui/pages/print_preview_page.py`
|
||||
- `src/transcription/api/v4_print.py`
|
||||
- `tests/ui/test_documents_page.py`
|
||||
- `tests/services/test_document_service.py`
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- Document creation persists the Document before adding relationship links; a later link failure is surfaced but is not currently one atomic write.
|
||||
- Source ordering controls are deferred to the [draft V4.3 scope](../../ver4.3/scope_boundary_v4_3.md).
|
||||
- Source page ordering remains read-only in V4.4.
|
||||
- Printing other entities, batch printing, and server-side export formats are deferred.
|
||||
|
||||
@@ -10,8 +10,8 @@ This document describes the production architecture of the document transcriptio
|
||||
- Execute page transcription concurrently with bounded `asyncio` workers.
|
||||
- Maintain relational portability across SQLite and PostgreSQL.
|
||||
- Keep operator workflows cross-platform and Python-driven.
|
||||
- Support many-to-many document-person relationships with extensible roles.
|
||||
- Support registry-driven document type classification.
|
||||
- Support one role-bearing link per Person and Document through an extensible role registry.
|
||||
- Support registry-driven document classification with protected semantic built-ins.
|
||||
|
||||
## Core Capabilities
|
||||
|
||||
@@ -20,8 +20,8 @@ This document describes the production architecture of the document transcriptio
|
||||
- Preserve original source files with SHA-256 digests and byte sizes.
|
||||
- Freeze prompt text, prompt hash, model, and explicitly configured sampling parameters on each `Job`.
|
||||
- Preserve page-level machine output, normalized metadata, and an SDK-serialized OpenRouter response snapshot on `JobSource`.
|
||||
- Organize historical `Person` records through many-to-many Document relationships and extensible roles.
|
||||
- Classify Documents through a UUID-identified registry with unique labels.
|
||||
- Organize historical `Person` records through UUID-identified Document links and extensible roles.
|
||||
- Classify Documents through a UUID-identified registry with hidden semantic built-ins and unique labels.
|
||||
- Maintain human revision separately from machine-generated text.
|
||||
- Isolate page failures so multi-page jobs can complete with partial success.
|
||||
- Operate across supported platforms through Python-based application and maintenance tooling.
|
||||
@@ -110,8 +110,8 @@ Responsibilities:
|
||||
- Jobs own job lifecycle state and transitions.
|
||||
- People own person records, relationship roles, document-person links, and portrait media.
|
||||
- Apply deterministic conflict handling for relationship-role writes.
|
||||
- Use set-based synchronization for many-to-many relationship updates.
|
||||
- Resolve and validate registry-backed document types by UUID.
|
||||
- Synchronize each Document's complete Person link set in the same transaction as Document fields.
|
||||
- Resolve and validate registry records by UUID; use hidden semantic keys only for application-owned built-in behavior.
|
||||
|
||||
### Source Media Policy
|
||||
|
||||
@@ -146,11 +146,11 @@ Responsibilities:
|
||||
|
||||
### 2. Document-Person Relationship Management
|
||||
|
||||
1. User opens a document or person edit flow.
|
||||
2. UI loads existing links grouped by role.
|
||||
3. User adds or removes people within one or more roles.
|
||||
4. Service computes add/remove deltas rather than replacing all links blindly.
|
||||
5. Conflict checks enforce uniqueness and deterministic write semantics before persistence commits.
|
||||
1. User opens Document Create or Edit.
|
||||
2. UI loads one Linked People table containing Person and Role.
|
||||
3. Add, Edit, and Delete operations change staged UI state only.
|
||||
4. Service validates the complete desired set and computes deterministic add, update, and remove deltas.
|
||||
5. Document fields and links commit once in one transaction; any failure leaves both unchanged.
|
||||
|
||||
### 3. Document Type Management
|
||||
|
||||
@@ -159,6 +159,14 @@ Responsibilities:
|
||||
3. Persistence stores the `document_type_id` reference.
|
||||
4. Inactive types remain valid for historical rows but are excluded from default selectors.
|
||||
|
||||
### 4. Document Printing
|
||||
|
||||
1. User opens Print from persisted Document Detail.
|
||||
2. Service builds a safe projection containing archival metadata, semantic Author links, ordered Sources, current text,
|
||||
and oldest-to-newest Job metadata.
|
||||
3. The preview renders Facsimile or Text-only HTML without exposing local file paths.
|
||||
4. An explicit action opens the browser print dialog; browser Save as PDF remains available.
|
||||
|
||||
## V4 Domain Rules
|
||||
|
||||
- `JobSource.raw_transcription` preserves page output for its Job execution.
|
||||
@@ -169,23 +177,25 @@ Responsibilities:
|
||||
- Every V4.2 provider call appends a distinct `ExecutionAttempt`; retries never rewrite earlier attempts.
|
||||
- Exact response bytes identify the OpenRouter HTTP boundary and are not labeled as native upstream-provider JSON.
|
||||
- Generic `ProcessingArtifact` records use versioned schemas, digests, and one inline or external content location.
|
||||
- `DocumentPerson` links are unique for `(document_id, person_id, role_id)`.
|
||||
- Relationship mutations are deterministic and set-based.
|
||||
- `DocumentType.id` is canonical identity; its unique label may evolve.
|
||||
- `DocumentPerson` links are unique for `(document_id, person_id)` and require one `role_id`.
|
||||
- Relationship mutations are deterministic, set-based, and atomic with Document writes.
|
||||
- `DocumentType.id` and `PersonRole.id` are canonical relationship identities; unique labels may evolve.
|
||||
- Nullable immutable `semantic_key` values identify protected application-defined built-ins and are never public selectors.
|
||||
- Current printable text uses non-null `Source.revised_text`; otherwise it uses `Source.raw_transcription`.
|
||||
|
||||
## Data Model Summary
|
||||
|
||||
- `Document` has one `DocumentType`, many `Source` pages, many `Job` runs, and many `Person` records through `DocumentPerson`.
|
||||
- `Source` belongs to one `Document` and may participate in many `JobSource` executions.
|
||||
- `Job` has many `JobSource` rows.
|
||||
- `PersonRole` defines available relationship roles.
|
||||
- `PersonRole` defines available relationship roles; `DocumentType` and `PersonRole` may carry hidden semantic identity.
|
||||
|
||||
## Test Strategy
|
||||
|
||||
- Unit tests for models, validation, hashing, and registry resolution.
|
||||
- Service tests for CRUD, set-based sync, uniqueness conflicts, and deterministic relationship writes.
|
||||
- Service tests for registry protection, atomic link synchronization, uniqueness conflicts, and print projections.
|
||||
- Async workflow tests for page isolation, partial failure handling, and stored evidence.
|
||||
- UI integration tests for multi-page rendering, role grouping, and document type selection.
|
||||
- UI integration tests for Linked People staging, registry selection, and safe print rendering.
|
||||
|
||||
## Related Local References
|
||||
|
||||
|
||||
@@ -11,36 +11,42 @@ This document defines the baseline requirements for the document transcription s
|
||||
| REQ-2 | Functional | Process page transcription asynchronously using an `asyncio` worker pool bounded by rate limits. | test |
|
||||
| REQ-3 | Functional | Persist submission-time request provenance and accurately labeled page-level SDK evidence; V4.2 adds exact OpenRouter-boundary transport evidence for new attempts. | test |
|
||||
| REQ-4 | Functional | Support job states `queued`, `processing`, `completed`, `partial_success`, and `failed`, plus page states `pending`, `transcribed`, and `failed`. | inspection |
|
||||
| REQ-5 | Functional | Allow users to manage historical `Person` records and link multiple people per role to a `Document`. | test |
|
||||
| REQ-5 | Functional | Allow users to manage historical `Person` records and link each Person to a Document once with exactly one role. | test |
|
||||
| REQ-6 | Functional | Support an extensible role taxonomy for document-person relationships. | inspection |
|
||||
| REQ-7 | Policy Constraint | Enforce deterministic relationship-role writes with uniqueness on `(document_id, person_id, role_id)` and explicit conflict responses for invalid duplicate link attempts. | test |
|
||||
| REQ-7 | Policy Constraint | Enforce deterministic relationship-role writes with uniqueness on `(document_id, person_id)` and explicit conflict responses for duplicate Person links. | test |
|
||||
| REQ-8 | Functional | Use set-based synchronization for document-person mutations so updates add and remove only the intended links. | test |
|
||||
| REQ-9 | Functional | Maintain immutable machine output on `Source.raw_transcription` while permitting inline human edits on `Source.revised_text`. | test |
|
||||
| REQ-10 | Functional | Support a UUID-identified `DocumentType` taxonomy with unique user-facing labels and active/inactive lifecycle control. | test |
|
||||
| REQ-11 | Data Constraint | Store `Document` type as a controlled reference to `DocumentType`. | test |
|
||||
| REQ-12 | Interface | Render multi-page transcriptions sequentially by `page_number` with document, people, and document-type metadata. | demonstration |
|
||||
| REQ-13 | Interface | Document create/edit UI must support selecting multiple people per role and selecting an active document type from the registry. | demonstration |
|
||||
| REQ-13 | Interface | Document create/edit UI must provide one staged Linked People table and select active registry entries by UUID and label. | demonstration |
|
||||
| REQ-14 | API Constraint | Expose additive, role-aware retrieval and write behavior for document-person links and UUID-based selection for document types. | test |
|
||||
| REQ-15 | Data Constraint | Calculate and store cryptographic file hashes (SHA-256) and file sizes for uploaded source images. | test |
|
||||
| REQ-16 | Data Constraint | Preserve a portable relational model across supported backends using SQLModel, SQLAlchemy, SQLite, and PostgreSQL. | inspection |
|
||||
| REQ-17 | Reliability | Ensure delete and update flows for documents, people, and relationship links remain deterministic and safe. | test |
|
||||
| REQ-18 | Operations Constraint | Keep canonical development, testing, restore, and recovery workflows OS-independent; for AI-run unit tests, require a pre-test backup of `./data` and an always-shown post-success confirmation prompt before any restore action. | inspection |
|
||||
| REQ-19 | Quality | Provide automated coverage for async transcription workflows, relationship-role enforcement, document-type selection, and regression behavior. | test |
|
||||
| REQ-20 | Data Constraint | Permit hidden immutable semantic keys only on protected built-in Document Types and Person Roles while retaining UUID as relationship identity. | test |
|
||||
| REQ-21 | Reliability | Persist Document fields and their complete Linked People set atomically. | test |
|
||||
| REQ-22 | Interface | Provide safe browser-native Facsimile and Text-only print views from persisted Document Detail. | demonstration |
|
||||
| REQ-23 | Security | Escape stored print text and serve Source images through record-validated application routes without disclosing local paths. | test |
|
||||
| REQ-24 | Functional | Print current human-preferred Source text, semantic Author metadata, deterministic Source order, and oldest-to-newest Job metadata. | test |
|
||||
|
||||
## Clarifying Constraints
|
||||
|
||||
1. `DocumentType.id` is its sole identity; labels are unique ignoring case and surrounding whitespace.
|
||||
2. `PersonRole.code` is a stable machine identifier; `PersonRole.label` may evolve.
|
||||
1. `DocumentType.id` and `PersonRole.id` are their public and relationship identities; labels are unique ignoring case and surrounding whitespace.
|
||||
2. Nullable `semantic_key` values identify protected application built-ins, remain internal, and never change.
|
||||
3. Relationship-write policy and conflict handling must be consistent across UI, API, services, and persistence.
|
||||
4. Many-per-role behavior is required for document-person links.
|
||||
5. Relationship conflicts must fail deterministically without partial mutation.
|
||||
4. One Person may appear only once per Document and every link has exactly one role.
|
||||
5. Relationship conflicts must fail deterministically without partial Document or link mutation.
|
||||
6. Source page reordering and server-generated PDF files remain outside this revision.
|
||||
|
||||
## Element Satisfaction Mapping
|
||||
|
||||
- UI (NiceGUI): Satisfies REQ-0, REQ-1, REQ-5, REQ-9, REQ-12, REQ-13.
|
||||
- API (FastAPI): Satisfies REQ-1, REQ-4, REQ-5, REQ-7, REQ-8, REQ-14.
|
||||
- UI (NiceGUI): Satisfies REQ-0, REQ-1, REQ-5, REQ-9, REQ-12, REQ-13, REQ-22, REQ-24.
|
||||
- API (FastAPI): Satisfies REQ-1, REQ-4, REQ-5, REQ-7, REQ-8, REQ-14, REQ-23.
|
||||
- Worker (`asyncio`): Satisfies REQ-2, REQ-3, REQ-4.
|
||||
- Persistence (SQLModel / SQLAlchemy): Satisfies REQ-3, REQ-9, REQ-10, REQ-11, REQ-15, REQ-16, REQ-17.
|
||||
- Persistence (SQLModel / SQLAlchemy): Satisfies REQ-3, REQ-7, REQ-9, REQ-10, REQ-11, REQ-15, REQ-16, REQ-17, REQ-20, REQ-21.
|
||||
- Test Suite: Verifies all test-marked requirements and satisfies REQ-19.
|
||||
|
||||
## Related Local References
|
||||
|
||||
+23
-7
@@ -8,6 +8,7 @@ This document defines the relational schema for the document transcription syste
|
||||
erDiagram
|
||||
DOCUMENT_TYPE {
|
||||
UUID id PK
|
||||
TEXT semantic_key UK
|
||||
TEXT label
|
||||
TEXT normalized_label
|
||||
BOOLEAN is_active
|
||||
@@ -17,8 +18,9 @@ TIMESTAMPTZ updated_at
|
||||
|
||||
PERSON_ROLE {
|
||||
UUID id PK
|
||||
TEXT code
|
||||
TEXT semantic_key UK
|
||||
TEXT label
|
||||
TEXT normalized_label
|
||||
BOOLEAN is_active
|
||||
TIMESTAMPTZ created_at
|
||||
TIMESTAMPTZ updated_at
|
||||
@@ -196,17 +198,29 @@ EXECUTION_ATTEMPT ||--o{ PROCESSING_ARTIFACT : produces
|
||||
- `SOURCE.raw_transcription` remains immutable machine output.
|
||||
- `SOURCE.revised_text` stores human edits and is the preferred display value when present.
|
||||
|
||||
### Semantic Registry Governance
|
||||
|
||||
- `DOCUMENT_TYPE.id` and `PERSON_ROLE.id` are the only relationship and public API identities.
|
||||
- Nullable unique `semantic_key` values identify application-defined built-ins and are immutable after creation.
|
||||
- Semantic keys are internal and are never accepted from Settings or public relationship APIs.
|
||||
- A non-null semantic key marks a protected built-in; built-ins may be relabeled or disabled but not deleted.
|
||||
- Custom entries have null semantic keys and may be deleted only when unreferenced.
|
||||
- Labels are mutable display text and are unique after trimming and case normalization.
|
||||
- Inactive entries remain valid for historical rows but are excluded from new-assignment selectors.
|
||||
|
||||
### Document-Person Role Governance
|
||||
|
||||
- Documents support zero, one, or many people per relationship role.
|
||||
- Documents support zero or one relationship for each Person.
|
||||
- Relationship roles are defined by `PERSON_ROLE` rather than hardcoded columns.
|
||||
- `DOCUMENT_PERSON` must be unique for `(document_id, person_id, role_id)`.
|
||||
- Relationship writes must be deterministic and use explicit add/remove link intent.
|
||||
- `DOCUMENT_PERSON.role_id` is required.
|
||||
- `DOCUMENT_PERSON` must be unique for `(document_id, person_id)`.
|
||||
- Complete link sets and Document fields are validated and persisted in one atomic transaction.
|
||||
- Existing inactive roles may remain unchanged; new or changed assignments require active roles.
|
||||
|
||||
### Document Type Governance
|
||||
|
||||
- Every document type is defined by `DOCUMENT_TYPE`.
|
||||
- `DOCUMENT_TYPE.id` is the sole machine identity.
|
||||
- `DOCUMENT_TYPE.id` is the relationship identity; hidden semantic keys identify protected built-in meaning.
|
||||
- `DOCUMENT_TYPE.label` is mutable display text and is unique after trimming and case normalization.
|
||||
- `DOCUMENT_TYPE.normalized_label` stores the normalized uniqueness key.
|
||||
- Inactive types remain valid for historical rows but should be excluded from default selection UIs.
|
||||
@@ -214,8 +228,10 @@ EXECUTION_ATTEMPT ||--o{ PROCESSING_ARTIFACT : produces
|
||||
## Constraint Summary
|
||||
|
||||
- `DOCUMENT_TYPE.normalized_label` is unique.
|
||||
- `PERSON_ROLE.code` is unique.
|
||||
- `DOCUMENT_PERSON(document_id, person_id, role_id)` is unique.
|
||||
- `DOCUMENT_TYPE.semantic_key` is nullable and unique.
|
||||
- `PERSON_ROLE.normalized_label` is unique.
|
||||
- `PERSON_ROLE.semantic_key` is nullable and unique.
|
||||
- `DOCUMENT_PERSON(document_id, person_id)` is unique.
|
||||
|
||||
## Indexing Guidance
|
||||
|
||||
|
||||
Reference in New Issue
Block a user