transcription/docs/ver4.4/implementation_plan_v4_4.md

Implementation Plan (Version 4.4)

Goal

Deliver hidden semantic identity for built-in registries, a single atomic Linked People workflow, and safe browser-native printing of archival Documents and their current transcriptions.

Planning Constraints

• V4.3 is the completed implementation baseline.
• V4.4 may replace V4/V4.3 registry and document-person contracts only as specified by the V4.4 scope.
• Semantic keys are internal and immutable; UI and public API contracts use UUIDs and labels.
• Document and link edits must not partially commit.
• Print output must not execute stored text or expose machine-local source paths.
• Source page reordering remains excluded.
• Database, integration, and UI tests must use confirmed isolated data and never modify data/transcription.db.
• Potentially destructive tests must run only through tools/run_destructive_tests.py.

Expected Project Impact

Area	Expected impact
Models and schema bootstrap	Add nullable unique semantic keys, simplify document-person identity, and seed frozen built-ins.
Document service	Maintain built-in Document Types, usage summaries, UUID assignment, and atomic Document/link writes.
People service	Maintain built-in Person Roles, link summaries, UUID-only role assignment, and one-person-per-document enforcement.
V4 document API	Remove role-code selectors and compatibility role fields; enforce UUID-only relationship writes.
Settings UI	Use matching table workflows for Document Types and Person Roles.
Document Create/Edit	Replace role-specific multiselects with one staged Linked People table and inline editor.
Document Detail/printing	Add format selection, print preview, safe Source media rendering, print CSS, and job metadata.
Tests and documentation	Replace superseded cardinality/identity assertions and add isolated registry, editor, transaction, and print coverage.

Implementation Phases

1. Align Durable Registry Contracts

• Add nullable, unique semantic_key fields to DocumentType and PersonRole.
• Keep UUIDs as primary and foreign-key identity.
• Add normalized-label storage and uniqueness to Person Roles using the same trim and case-normalization policy as Document Types.
• Remove the user-created Person Role code contract.
• Define built-in detection as semantic_key is not null.
• Centralize the frozen built-in definitions in one domain-owned location.
• Seed six Document Types and three Person Roles idempotently.
• Ensure label edits never change semantic keys.
• Reject deletion of every built-in before checking references.
• Continue blocking deletion of referenced custom entries.
• Return deterministic validation, conflict, dependency, and not-found errors through existing error categories.

2. Establish the Clean Schema

• Remove legacy DocumentPerson.role compatibility storage and the fixed DocumentPersonRole enum.
• Make DocumentPerson.role_id required.
• Replace role-specific uniqueness with a unique (document_id, person_id) constraint.
• Remove obsolete Document Type and Person Role migration paths that exist only for disposable development data.
• Keep fresh schema creation and built-in seeding portable across SQLite and PostgreSQL.
• Make configured development-database recreation a separate operator-confirmed step that displays the resolved target path rather than assuming app.db or data/transcription.db.
• Never invoke recreation from application startup or test setup.
• Add isolated schema tests for fresh creation, seed idempotence, semantic-key uniqueness, normalized-label uniqueness, required roles, and link uniqueness.

3. Refine Registry Services and API Contracts

• Add summary queries for Document counts and Person Role link counts without per-row queries.
• Order both registries by normalized label with UUID as a deterministic tie-breaker.
• Expose built-in status as a derived read value where the Settings UI needs it.
• Keep semantic-key lookup behind service methods for application-owned behavior such as resolving authors.
• Ensure create operations always produce custom entries with null semantic keys.
• Ensure update operations accept only label and active state.
• Remove role_code request alternatives and compatibility role responses from the V4 document API.
• Require role_id for document-person creation and updates.
• Add service and API tests for hidden semantic identity, relabeling, activation, built-in protection, custom deletion, counts, ordering, UUID-only writes, and conflicts.

4. Build Matching Settings Tables

• Retain the existing Document Types table workflow and add the Built-in column.
• Replace the current per-row Person Role controls with the same selection-based table pattern.
• Render the agreed columns and usage counts.
• Keep labels as the only registry text shown in selectors.
• Add creates custom entries only.
• Edit dialogs expose label and active state only.
• Delete reports protected-built-in and referenced-custom conflicts clearly.
• Avoid direct persistence queries from the Settings page.
• Add component-level UI assertions for columns, actions, label-only selectors, and immutable built-in presentation.

5. Add a Staged Linked People Editor

• Introduce a small typed UI-state model for staged (person_id, role_id) rows rather than storing raw widget values.
• Share the editor component between Create Document and Edit Document.
• Render a multi-selection table with Person and Role labels.
• Add an inline editor whose mode is explicitly Add or Edit.
• Disable already-linked People when adding; retain the edited Person as an option during Edit.
• Require exactly one row for Edit and allow one or more rows for Delete.
• Save and Delete mutate only staged UI state.
• Cancel discards only the active inline edit.
• Preserve inactive-role historical rows in Edit while restricting new assignments and changes to active roles.
• Preserve person_id preselection by staging that Person with the active built-in author role, with warning behavior for invalid or unavailable selections.
• Preserve the return_to=jobs_new success path.
• Keep navigation to Person creation separate; V4.4 does not add an embedded Person editor.
• Add UI tests for staging, duplicate prevention, selection rules, inactive roles, cancel behavior, and both Document forms.

6. Persist Document and Links Atomically

• Add service commands for Create Document with complete links and Update Document with complete links.
• Validate Document Type, every Person, every Person Role, active assignment rules, and duplicate People before mutation.
• Compute deterministic add, update, and remove deltas for Edit.
• Apply Document and link mutations in one database transaction and commit once.
• Roll back the complete operation on any validation, conflict, or persistence failure.
• Return the persisted Document detail required by the UI after success.
• Reuse these commands from UI orchestration rather than sequencing independent service commits.
• Add failure-injection tests proving no partial Document or link mutation survives.

7. Define a Print Projection

• Add a read-only service projection containing:
  • Document title and selected archival metadata.
  • Authors resolved by the author semantic key.
  • Notes.
  • Ordered Sources with application media URLs and current transcription text.
  • Ordered Job metadata.
• Load the projection with bounded queries and deterministic ordering.
• Use non-null revised_text, including an intentionally empty revision; otherwise fall back to raw_transcription.
• Map empty or whitespace-only current text to the explicit unavailable state without falling back past an intentional revision.
• Represent unavailable text and optional metadata explicitly.
• Do not expose semantic keys, direct file paths, full prompts, provider evidence, or raw API responses.
• Keep the projection independent of NiceGUI rendering so formatting tests can use plain typed values.

8. Build Print Preview and Styles

• Add a Print action to Document Detail.
• Open a dedicated persisted-Document print route with a Facsimile/Text-only format choice.
• Render the exact content order frozen in the scope.
• Render stored Notes and transcription as escaped text.
• For Text-only mode, normalize whitespace by joining single line breaks inside paragraphs while preserving blank-line paragraph boundaries.
• For Facsimile mode, preserve line breaks and use a two-column Source layout.
• Start each Facsimile Source on a new printed sheet with CSS page breaks.
• Allow long transcription content to continue rather than clipping it.
• Fetch images through an application-controlled Source media route.
• Add print-only CSS that hides navigation, controls, and non-document chrome.
• Invoke the browser print dialog only from an explicit user action.
• Add rendering tests for both modes, missing data, long text, special characters, image URLs, and page ordering.

9. Align Documentation and Verification

• Update V4 architecture, requirements, schema, and Document UI contracts to reflect:
  • UUID plus hidden semantic-key registries.
  • Built-in protection.
  • One Person per Document.
  • UUID-only role API writes.
  • Atomic Document/link synchronization.
  • Browser-native print projection and formats.
• Confirm every database, integration, and UI test target is isolated before execution.
• Run focused registry and service tests first.
• Run schema tests only through the destructive-test wrapper when they are potentially destructive.
• Run Linked People UI and print rendering tests against isolated fixtures.
• Run the broader non-external regression suite after focused coverage passes.
• Verify that data/transcription.db was not changed by test execution.

Delivery Order

1. Registry and clean-schema contracts.
2. Registry services, API changes, and Settings tables.
3. Atomic Document/link service commands.
4. Shared staged Linked People editor.
5. Print projection.
6. Print preview and styles.
7. Documentation alignment and regression verification.

Done Criteria

• All V4.4 acceptance criteria are implemented and testable.
• UI and API contracts use UUID identity and never expose semantic keys.
• Built-in registries retain meaning after relabeling and cannot be deleted.
• Custom registries retain reference-aware deletion.
• Both Document forms use one Linked People table.
• One Person cannot be linked twice to the same Document.
• Main Document saves are atomic across fields and relationships.
• Print preview provides both frozen formats and content sections.
• Print output uses current human-preferred text, deterministic ordering, escaped content, and application media URLs.
• Job metadata lists every Job oldest-to-newest and ends with Status.
• Source page reordering and server-generated PDFs are not introduced.
• Verification uses isolated data and does not modify data/transcription.db.

Related Local References

• V4.4 Scope Boundary
• V4.3 Scope Boundary
• V4.3 Implementation Plan
• V4 Architecture
• V4 Schema
• V4 Requirements
• V4 Error Handling Policy