V4.1 Mostly UI adjustments by GC

docs/Proposed Revisions to V4.md

@@ -0,0 +1,63 @@
+# Proposed Revisions
+
+This document contains a list of proposed fixes, adjustments, and additional features. Consider whether each change provides a real improvement, is practical to implement, and whether or not it introduces unnecessary complexity and make recommendations.
+
+Also, consider whether these changes should be implemented in more than one revision. For example, some might be considered Version 4.1 revisions, some might be Version 5, and some might be saved for a future date after I've had time to use the app for a while.
+
+## UI
+Most of the proposed changes to the UI are cosmetic. The issue numbers refer to issues I have accumulated in the Gitea repository for my own reference.
+
+### Archival Documents page (issue #13)
+* Limit the Document Title column width and wrap long titles. All columns in the table should fit on the screen.
+* Left-align Document Title and Type columns
+* Add Author, Document Date columns to the table
+
+### Document Detail page (issue #15)
+* In Create/Edit mode, When linking people by role it is impossible to distinguish between two people with the exact same name (e.g., there are two Albert Edward Higgins, Albert Edward Higgins (father & son). Possible solutions:
+    * Consider using the Display Name instead, and adding a contraint that the Display Name be unique
+    * Consider concatenating birth year onto the Full Name (e.g., 'Albert Edward Higgins (1885)')
+* In View mode, The person entries in the "Related People" box should be hyper-linked to their People Detail pages
+* Move "View All Sources" box from under image to under Pipeline Jobs. Consider combining both Pipeline Jobs and Sources together in one box.
+* Hide unused fields when not in Edit mode. Example: When editing the Document Detail page there are boxes for Exact Date and Approx. Date. Only one field is ever used (never both). Is it possible to hide the unused field when not in Edit mode?
+
+### Archival Entities: People page (issue #19)
+* Left align Display Name, Maiden Name columns
+* Birth Date column should contain Birth Date if known, else Approx Birth Date if known, else "unknown".
+* Add Death Date column with rule similar to Birth Date
+
+### Person Detail page (issue #17)
+* Add "New Document" button to People page to accommodate an additional workflow: ```Add new person --> Add document --> Add job```. The current workflow is this: ```Add new document --> Link person, add new if one doesn't exist --> Add job``` The user should be able to start from either Document or Person, whichever seems more natural to them.
+* Hide unused fields when not in Edit mode. Example: When editing the People Detail page there are boxes for birth date and approx. birth date. Only one field is ever used (never both). Hide the unused fields on the Person Detail page when not in Edit mode.
+
+### Source Asset Records page (issue #12)
+* A Source with a long Document Name or Upload Title forces columns (e.g., Stored Filename, Status, and Error Detail) off the right edge of the bounding box. Limit the width of the columns and wrap their contents. All columns in the table should fit on the screen.
+* Remove Stored Filename from the table
+* Left align Document Name, Upload Title, and Error Detail
+
+### Source Detail page (issue #10)
+* If a Source is one page of a multi-page document, add left/right navigation arrows on either side of the displayed image or page (which ever is easier to implement) screen to allow the user to page through the Sources without having to exit select another source then go back into the source detail screen.
+
+### Transcription Pipeline Jobs page
+* No changes at this time
+
+### Job Detail Page (issue #7)
+* Is it possible to add an automatic screen refresh while waiting for a job to finish? This could happen on a timed basis, like every 3-5 seconds.
+
+### Home page
+* Is the current location of the home page in the ./data/homepage folder the correct location to store the image & markdown file? Does it belong somewhere else in the repository like the context prompt which is in ./prompts
+
+---
+
+## New Features
+* Consider creating a "Settings" page. My idea is that this page would provide one place for the user to customize the app. This screen may lead to subscreens. For example, the View/Edit prompt button might call up a page where the markdown is displayed in an editable box, and include a brief description of what the prompt is, how it's used, and the document format (markdown).
+    * Maintain Document Types and People Roles
+    * View/edit .env?
+    * View/edit theme?
+    * View/edit prompt?
+* Consider adding field to Person Detail page (and person table) for links to ancestry.com and familysearch.org (and maybe others in the future). Explore using familysearch.org api to automatically populate Person details like Birth date, Birth Place, Death Date, Death Place, etc. (The goal here is not reinvent the wheel and leverage existing data).
+* Consider adding the ability to renumber/reorder pages. (What happens when they get imported out of order?)
+* Consider adding link to location/place fields to Google Maps. (Perhaps this is better left to the genealogy services...)
+
+
+
+

docs/api_qa.md

@@ -1,219 +0,0 @@
-# API Questions & Answers
-
-## Question
-```
-what is v4_documents.py?
-```
-
-## Response
-v4_documents.py is the new additive V4 API router I added in Phase 4.
-
-What it does:
-- Defines V4 endpoints under /api/v4 for registry-driven document/relationship workflows.
-- Exposes read endpoints for catalogs:
-  - GET /api/v4/document-types
-  - GET /api/v4/person-roles
-- Exposes write endpoints for code-or-id updates:
-  - PUT /api/v4/documents/{document_id}/type
-  - POST /api/v4/documents/{document_id}/people
-  - PATCH /api/v4/document-people/{document_person_id}
-  - DELETE /api/v4/document-people/{document_person_id}
-- Uses DocumentService methods as the backend (no direct DB access in the route layer).
-- Returns typed request/response models for backward-compatible, additive API evolution.
-
-It is registered in app.py, so it is live when the app starts.
-
-If you want, I can walk endpoint-by-endpoint with examples of request/response payloads.
-
----
-
-## Question
-```
-Please do
-```
-
-## Response
-Absolutely. Here is a quick endpoint-by-endpoint walkthrough of v4_documents.py, with practical request/response examples.
-
-**What This Router Is**
-- Prefix: /api/v4
-- Purpose: additive V4 contract for:
-1. Registry catalogs (document types, person roles)
-2. Role-aware document-person linking
-3. Code-or-id document type assignment
-- It delegates to DocumentService and does not access DB sessions directly in route handlers.
-
-**Dependency Wiring**
-- Route dependency: get_document_service in v4_documents.py
-- Behavior:
-1. Uses app.state.services.documents when running in the app lifecycle
-2. Falls back to a direct DocumentService() instance if needed (useful in isolated tests)
-
-**1) List Document Types**
-- Endpoint: GET /api/v4/document-types
-- Query param: active_only=true by default
-- Backing service call: list_document_types(active_only=...)
-- Response shape:
-[
-  {
-    "id": "uuid",
-    "code": "letter",
-    "label": "Letter",
-    "is_active": true,
-    "sort_order": 0
-  }
-]
-
-**2) List Person Roles**
-- Endpoint: GET /api/v4/person-roles
-- Query param: active_only=true by default
-- Backing service call: list_person_roles(active_only=...)
-- Response shape:
-[
-  {
-    "id": "uuid",
-    "code": "author",
-    "label": "Author",
-    "is_active": true
-  }
-]
-
-**3) Set Document Type (By Id Or Code)**
-- Endpoint: PUT /api/v4/documents/{document_id}/type
-- Body options:
-1. document_type_id
-2. document_type_code
-- Rule: provide one selector, not both
-- Example request:
-{
-  "document_type_code": "record"
-}
-- Example response:
-{
-  "document_id": "uuid",
-  "document_type_id": "uuid",
-  "document_type_code": "record"
-}
-
-**4) List Document-Person Links For A Document**
-- Endpoint: GET /api/v4/documents/{document_id}/people
-- Backing service call: list_document_people(document_id=...)
-- Example response:
-{
-  "document_id": "uuid",
-  "links": [
-    {
-      "id": "uuid",
-      "document_id": "uuid",
-      "person_id": "uuid",
-      "role_id": "uuid",
-      "role_code": "author",
-      "person_name": "Zenna Cochran"
-    }
-  ]
-}
-
-**5) Add A Document-Person Link**
-- Endpoint: POST /api/v4/documents/{document_id}/people
-- Body fields:
-1. person_id required
-2. role_id optional
-3. role_code optional
-- If duplicate document/person/role link is attempted, returns conflict envelope via shared error handlers.
-- Example request:
-{
-  "person_id": "uuid",
-  "role_code": "mentioned"
-}
-- Example success response:
-{
-  "id": "uuid",
-  "document_id": "uuid",
-  "person_id": "uuid",
-  "role_id": "uuid",
-  "role_code": "mentioned",
-  "person_name": "Ada Lovelace"
-}
-
-**6) Update Link Role**
-- Endpoint: PATCH /api/v4/document-people/{document_person_id}
-- Body options:
-1. role_id
-2. role_code
-- Rule: provide one selector, not both
-- Example request:
-{
-  "role_code": "recipient"
-}
-- Example response:
-{
-  "id": "uuid",
-  "document_id": "uuid",
-  "person_id": "uuid",
-  "role_id": "uuid",
-  "role_code": "recipient",
-  "person_name": "Ada Lovelace"
-}
-
-**7) Delete Link**
-- Endpoint: DELETE /api/v4/document-people/{document_person_id}
-- Response: HTTP 204 No Content
-
-**Error Behavior**
-- Uses shared registration in errors.py
-- So AppError categories map to consistent envelope + status codes:
-1. validation_error -> 400
-2. not_found_error -> 404
-3. conflict_error -> 409
-4. internal_unexpected_error -> 500
-
-**Where It Gets Mounted**
-- Included in app factory at app.py
-- That makes all /api/v4 endpoints available during normal app startup.
-
-If you want, next I can generate a compact curl-style smoke checklist for all 7 endpoints so you can test them in under 2 minutes.
-
----
-
-## Question
-```
-Should there be one of these for people, jobs, and sources too?
-```
-
-## Response
-
-Short answer: yes, but not all at once.
-
-Given your V4 scope, the strongest split is:
-
-1. Keep v4_documents.py for document-centric workflows  
-This file already owns:
-- document type catalog/use
-- document-person role links
-- document-level relationship mutations
-
-2. Add a V4 people router next  
-Recommended new file:
-- src/transcription/api/v4_people.py
-
-Why:
-- You likely need person-centric queries and updates that are not naturally document-scoped
-- It keeps API boundaries clearer as UI grows
-
-3. Do not rush a separate jobs/sources V4 router unless contracts are diverging  
-For now, jobs and sources are still tightly tied to the existing transcription pipeline behavior. A new v4_jobs or v4_sources router is useful only when you need genuinely new contract shape, such as:
-- job lifecycle contract changes
-- source revision/audit contract changes
-- new pagination/filtering semantics incompatible with current endpoints
-
-Practical rule of thumb:
-1. New router if the domain has new V4 semantics and independent lifecycle
-2. Keep existing router if only additive fields are needed
-
-Suggested target structure:
-1. v4_documents.py
-2. src/transcription/api/v4_people.py
-3. optional later: src/transcription/api/v4_jobs.py
-4. optional later: src/transcription/api/v4_sources.py
-
-If you want, I can scaffold v4_people next with additive endpoints that mirror your current service capabilities and keep error envelope behavior consistent.

docs/ver4.1/implementation_plan_v4_1.md

@@ -0,0 +1,146 @@
+# Implementation Plan (Version 4.1)
+
+## Goal
+
+Deliver the V4.1 usability revision as a small, behavior-safe increment over the V4 baseline.
+
+## Implementation Principles
+
+- Keep presentation formatting in UI components and route orchestration in pages.
+- Keep persistence and cross-record queries behind service boundaries.
+- Reuse shared table and date-label helpers instead of duplicating fallback logic.
+- Make the FamilySearch schema change additive and nullable.
+- Add focused tests for changed behavior before broad regression verification.
+
+## Current Project Impact
+
+| Area | Expected impact |
+| --- | --- |
+| Persistence | Add nullable `Person.family_search_id`; provide the repository's supported schema-upgrade path for existing databases. |
+| People service | Normalize and validate FamilySearch IDs at the domain/service boundary if model validation does not fully cover writes. |
+| Documents UI | Add table data, improve relationship labels/links, compact date display, and combine processing navigation. |
+| People UI | Add table date fields, Person-first Document creation, compact date display, and FamilySearch controls. |
+| Sources service/UI | Query adjacent document Sources and add bounded navigation; revise list columns and wrapping. |
+| Jobs UI | Refresh the active detail read model on a timer until terminal status. |
+| Shared UI | Add reusable constrained/wrapped table presentation and compact date formatting where appropriate. |
+| Tests | Update model/service and UI coverage for all affected workflows. |
+
+## Implementation Phases
+
+### 1. Add Shared Presentation Rules
+
+- Review `ui/components/table/common.py` and packaged theme CSS for the narrowest reusable table-width solution.
+- Add reusable styles or column slots for constrained, wrapping, left-aligned text.
+- Add a shared formatter for exact/approximate/unknown dates if it can be reused without coupling components to persistence.
+- Preserve sorting and search behavior for rendered display values.
+
+### 2. Update Archival List Tables
+
+- Extend the Document table read model with author names and the compact document date.
+- Build author display from eagerly loaded document-person links using the `author` role.
+- Apply title/type alignment and constrained title wrapping.
+- Extend the Person table read model with compact birth and death date values.
+- Apply Display Name and Maiden Name alignment.
+- Remove Stored Filename from the Source table read model only if no other list behavior consumes it; always remove its rendered column.
+- Constrain and left-align the requested Source columns.
+- Add or update UI component tests for serialized rows, columns, and fallback formatting.
+
+### 3. Improve Document Relationship Workflows
+
+- Introduce one person-label formatter that combines preferred Display Name, Full Name context, and known birth year without implying uniqueness.
+- Use Person UUIDs as selector values.
+- Apply the formatter to every relationship role selector.
+- Change Related People rows into actions that navigate to `/people/{person_id}`.
+- Replace separate exact/approximate rows in view mode with one conditional Document Date row.
+- Combine Pipeline Jobs and Sources into one related-processing card beneath Related People.
+- Preserve existing job/source counts and navigation actions.
+
+### 4. Add the Person-First Document Workflow
+
+- Add a New Document action on Person Detail.
+- Pass the Person UUID through a narrowly defined query parameter to `/documents/new`.
+- Validate the requested UUID against the loaded people list.
+- Preselect that person in the intended default relationship role. Use `author` unless a different role is explicitly encoded later.
+- Ignore invalid or unavailable preselection values with the application's normal visible error/notification behavior.
+- Confirm ordinary `/documents/new` behavior remains unchanged.
+
+### 5. Add FamilySearch Person References
+
+- Add nullable, unique `family_search_id` to the `Person` model and schema.
+- Implement a non-destructive upgrade for existing SQLite and PostgreSQL databases using the repository's established schema-management approach.
+- Normalize values by trimming and uppercasing.
+- Validate the `XXXX-XXX` alphanumeric identifier shape and return a clear validation error for malformed input.
+- Report duplicate identifiers as a deterministic conflict rather than a generic persistence failure.
+- Add the field to Person create/edit forms and preserve it during updates.
+- Add a URL builder that safely inserts only a validated identifier into the fixed FamilySearch details URL.
+- Render a FamilySearch action on Person Detail only when an identifier is present.
+- Add persistence, normalization, validation, form, and link-generation tests.
+
+### 6. Add Source Page Navigation
+
+- Add a Sources service query that returns previous/current/next context for a Source within its Document.
+- Define ordering by `page_number`, with a stable secondary key such as Source UUID for defensive determinism.
+- Keep navigation bounded to the current `document_id`.
+- Render previous and next actions adjacent to the source viewer or detail header.
+- Disable or omit unavailable boundary actions.
+- Test first, middle, last, single-page, and cross-document cases.
+
+### 7. Add Job Detail Auto-Refresh
+
+- Make Job Detail content refreshable without rebuilding unrelated global navigation.
+- Start a NiceGUI timer only for queued or processing jobs.
+- On each tick, re-read the Job through `JobService` and refresh the detail content.
+- Use a 4-second default interval.
+- Stop or deactivate the timer when status becomes completed, partial success, failed, or cancelled, according to the model's actual terminal states.
+- Prevent overlapping refresh callbacks.
+- Retain existing error presentation if a refresh read fails.
+- Add UI tests for timer creation, refresh, and terminal-state stopping.
+
+### 8. Simplify View-Mode Date Rows
+
+- On Document Detail, show exact date, else approximate date, else one not-set value.
+- On Person Detail, apply the same independent rule to birth and death.
+- Do not hide either input in create/edit mode.
+- Test each exact, approximate, and absent state.
+
+### 9. Verification and Documentation Alignment
+
+- Run the focused model/service/UI tests covering changed surfaces.
+- Run the existing regression suite appropriate to persistence and UI changes.
+- Confirm SQLite and PostgreSQL model compatibility at the schema-definition level.
+- Update V4.1 documentation if implementation reveals a necessary boundary change; do not silently expand scope.
+
+## Recommended Delivery Order
+
+1. Shared formatters and table presentation.
+2. Additive Person schema change and FamilySearch validation.
+3. Document and Person list/detail changes.
+4. Person-first Document workflow.
+5. Source navigation.
+6. Job polling.
+7. Focused and regression verification.
+
+## Done When
+
+- Every V4.1 acceptance criterion is demonstrated or covered by a focused test.
+- Existing Person rows remain valid after the nullable schema addition.
+- Duplicate FamilySearch references cannot be assigned to multiple local Person records.
+- FamilySearch links are generated only from normalized, validated IDs.
+- Auto-refresh performs no polling after a terminal job state.
+- Adjacent Source navigation never crosses Document boundaries.
+- The existing V4 workflows remain operational.
+
+## Out of Scope
+
+- Page reordering.
+- Settings management.
+- External genealogy API integration.
+- Raw `.env` editing.
+- Theme editing.
+
+## Related Local References
+
+- [V4.1 Scope Boundary](scope_boundary_v4_1.md)
+- [V4 Implementation Plan](../ver4/implementation_plan_v4.md)
+- [V4 Requirements](../ver4/requirements_v4.md)
+- [V4 Error Handling Policy](../ver4/error_handling_v4.md)

docs/ver4.1/scope_boundary_v4_1.md

@@ -0,0 +1,142 @@
+# V4.1 Scope Boundary
+
+This document defines the scope of the first incremental revision to Version 4. V4 remains the product and architecture baseline; V4.1 adds focused usability improvements and one additive Person field.
+
+## Purpose
+
+- Improve common archival record workflows without redesigning the application.
+- Resolve table overflow, ambiguous person selection, and unnecessary navigation.
+- Add a manually maintained FamilySearch person reference without introducing external API integration.
+
+## In Scope
+
+### 1. Archival Documents List
+
+- Keep every table column within the available page width.
+- Limit and wrap long Document Title values.
+- Left-align Document Title and Type.
+- Add Author and Document Date columns.
+- Display all people linked through the `author` role in the Author column.
+- Display exact document date when present, otherwise approximate date when present, otherwise `Unknown`.
+
+### 2. Document Detail and Editing
+
+- Use an unambiguous label in person selectors. The label should prefer Display Name, retain Full Name for context, and include the birth year when known.
+- Do not require Display Name to be unique.
+- Link each Related People entry to its Person Detail page.
+- In view mode, display only the populated exact or approximate document date row. Display a single unknown/not-set state when neither exists.
+- Keep both exact and approximate inputs available in create/edit mode.
+- Move source navigation out from beneath the media viewer.
+- Present Pipeline Jobs and Sources together in one related-processing card with counts and actions.
+
+### 3. People List
+
+- Left-align Display Name and Maiden Name.
+- Display exact birth date when present, otherwise approximate birth date when present, otherwise `Unknown`.
+- Add a Death Date column with the same fallback rule.
+
+### 4. Person Detail and Editing
+
+- Add a New Document action that opens Document creation with the current person preselected.
+- Preserve the existing Document-first workflow.
+- In view mode, display only the populated exact or approximate row for each of birth and death date. Display a single unknown/not-set state when neither value exists.
+- Keep both exact and approximate inputs available in create/edit mode.
+
+### 5. FamilySearch Reference
+
+- Add a nullable, unique `family_search_id` field to `Person`.
+- Allow the field to be entered and changed in Person create/edit flows.
+- Trim whitespace, normalize the identifier to uppercase, and validate it against the supported
+  `XXXX-XXX` alphanumeric shape before persistence.
+- When an identifier exists, show a FamilySearch action on Person Detail linking to:
+  `https://www.familysearch.org/tree/person/details/{family_search_id}`
+- Construct the URL in application code; do not persist the full URL.
+
+### 6. Source List and Detail
+
+- Keep every Source Asset Records table column within the available page width.
+- Limit and wrap long Document Name, Upload Title, and Error Detail values.
+- Left-align Document Name, Upload Title, and Error Detail.
+- Remove Stored Filename only from the Source Asset Records table. Continue storing it and showing it on Source Detail.
+- On Source Detail, add previous and next navigation for Sources belonging to the same Document, ordered by `page_number`.
+- Disable or omit the previous/next action at the first/last page.
+
+### 7. Job Detail
+
+- Automatically refresh Job Detail while the job is in a non-terminal state.
+- Use a modest interval in the 3-5 second range.
+- Stop polling when the job reaches a terminal state or the page is no longer active.
+- Preserve manual navigation and existing job actions.
+
+### 8. Homepage Storage Decision
+
+- Continue treating homepage markdown and images as mutable application data, not prompt artifacts or packaged source assets.
+- Keep homepage content separate from `prompts`.
+- Defer relocation to a configurable application-data root unless the existing location prevents normal installed or deployed operation.
+
+## Out of Scope
+
+- Source page renumbering or reordering.
+- A Settings page.
+- Editing `.env` or secrets through the UI.
+- Runtime theme editing.
+- FamilySearch authentication, API calls, search, import, synchronization, or conflict resolution.
+- Ancestry references or other genealogy providers.
+- Google Maps links from place fields.
+- Enforcing unique Display Name values.
+- Changes to transcription execution or provider behavior.
+- Changes to the V4 API solely to expose the V4.1 presentation enhancements.
+
+## Locked Design Decisions
+
+### A. Person Selector Identity
+
+- Selection values remain internal Person UUIDs.
+- Display labels provide disambiguating context but are not identity keys.
+- Duplicate Full Name and Display Name values remain valid.
+
+### B. Date Presentation
+
+- Exact dates take precedence over approximate/raw dates for compact list and view presentation.
+- Create/edit forms retain both fields so either representation can be maintained.
+- V4.1 does not introduce a new mutual-exclusion database constraint.
+
+### C. FamilySearch Storage
+
+- Store only the FamilySearch person identifier.
+- Treat a FamilySearch person identifier as unique across local Person records.
+- Use one dedicated nullable Person field while FamilySearch is the only supported external genealogy reference.
+- Reconsider a generic external-reference model only when a second provider or multiple references per person are required.
+
+### D. Stored Filename
+
+- Stored Filename remains part of the Source model and Source Detail diagnostics.
+- Only the list-table column is removed.
+
+## Data and Compatibility Policy
+
+- The `family_search_id` addition must be nullable and non-destructive for existing Person rows.
+- Existing records, routes, relationships, jobs, Sources, prompt provenance, and uploaded media remain valid.
+- UI changes must preserve both Document-first and Person-first workflows.
+- V4.1 must remain portable across SQLite and PostgreSQL.
+
+## Acceptance Criteria
+
+1. Document, Person, and Source tables fit their page containers at supported desktop widths without losing requested columns.
+2. Long table text wraps or is constrained without forcing important columns outside the table container.
+3. Duplicate-named people can be distinguished in every document relationship selector.
+4. Related People entries navigate to the correct Person Detail page.
+5. Compact date displays consistently use exact, then approximate, then unknown fallback behavior.
+6. Starting from Person Detail can create a Document with that person preselected without breaking normal Document creation.
+7. A valid FamilySearch ID is persisted and produces the correct Person Detail hyperlink; absent IDs produce no action.
+8. Source previous/next actions remain within the same Document and follow `page_number`.
+9. Active Job Detail pages update without manual refresh and stop polling after terminal status.
+10. Stored Filename is absent from the Source list table but remains available on Source Detail.
+11. Focused automated tests pass and unaffected V4 behavior remains intact.
+
+## Related Local References
+
+- [V4.1 Implementation Plan](implementation_plan_v4_1.md)
+- [V4 Scope Boundary](../ver4/scope_boundary_v4.md)
+- [V4 Architecture](../ver4/architecture_v4.md)
+- [V4 Schema](../ver4/schema_v4.md)

docs/ver4.2/implementation_plan_v4_2.md

@@ -0,0 +1,137 @@
+# Draft Implementation Plan (Version 4.2)
+
+## Goal
+
+Prepare a safe implementation path for Source page reordering and constrained application settings. This plan remains provisional until the V4.2 scope-freeze decisions are resolved.
+
+## Planning Constraints
+
+- V4 and V4.1 remain the behavioral baseline.
+- Reordering must be atomic and service-owned.
+- Settings must use explicit domain operations rather than direct database, environment-file, or arbitrary filesystem access from UI pages.
+- Prompt changes must preserve historical Job provenance and use a defined safe-write policy.
+
+## Expected Project Impact
+
+| Area | Expected impact |
+| --- | --- |
+| Sources service | Add validated, transactional set-based page reordering. |
+| Documents/Sources UI | Add a reorder entry point and interaction for one Document. |
+| Documents service | Expand controlled Document Type maintenance operations. |
+| People service | Expand controlled Person Role maintenance operations. |
+| Prompt adapter/service | Add constrained listing, reading, validation, and safe writing of prompt artifacts. |
+| UI composition/navigation | Register Settings routes and navigation without moving persistence into UI code. |
+| Tests | Add transaction, conflict, registry lifecycle, prompt safety, and UI workflow coverage. |
+
+## Proposed Implementation Phases
+
+### 1. Resolve Scope-Freeze Decisions
+
+- Select and document the reorder interaction.
+- Define whether active Jobs block reorder.
+- Define prompt atomic-write, backup, and recovery policy.
+- Decide whether prompt creation/deletion is excluded.
+- Finalize registry ordering requirements.
+- Remove the Draft designation only after these decisions are reflected in scope and acceptance criteria.
+
+### 2. Define Service Contracts
+
+- Define a Source reorder command containing `document_id`, the complete ordered Source ID list, and a concurrency token or equivalent stale-write guard if supported by the current model.
+- Define Document Type maintenance commands for create, relabel, sort, activate, and deactivate.
+- Define Person Role maintenance commands for create, relabel, activate, and deactivate.
+- Define a Prompt Store interface for constrained list/read/write behavior.
+- Map validation, conflict, not-found, dependency, and filesystem failures to existing `AppError` categories.
+
+### 3. Implement Transactional Source Reordering
+
+- Load all Sources for the target Document in the same transaction.
+- Reject missing, extra, duplicate, or foreign Source IDs.
+- Reject stale writes using the selected concurrency policy.
+- Apply a collision-safe renumbering strategy suitable for both SQLite and PostgreSQL.
+- Finish with contiguous `page_number` values beginning at 1.
+- Roll back the entire operation on any failure.
+- Add service tests for valid reorder, no-op, reverse order, invalid membership, duplicates, stale submissions, rollback, and backend-compatible SQL behavior.
+
+### 4. Implement the Reorder UI
+
+- Add a Reorder Pages action from a Document-scoped Source view or Document Detail.
+- Render Source labels/previews sufficient to identify each page.
+- Capture the complete intended order.
+- Require explicit Save and provide Cancel without mutation.
+- Surface validation and conflict errors through the shared error presenter.
+- Return to a Document-scoped ordered view after success.
+- Verify keyboard-accessible controls for any drag-and-drop interaction.
+
+### 5. Expand Registry Maintenance Services
+
+- Reuse existing Document and People service ownership.
+- Add explicit write methods rather than passing UI-mutated ORM objects directly where practical.
+- Normalize and validate new stable codes.
+- Reject duplicate codes deterministically.
+- Block deletion or omit deletion entirely; use activation state for lifecycle management.
+- Preserve inactive entries for historical reads.
+- Add service tests for create, relabel, activation, deactivation, duplicates, immutable codes, and referenced records.
+
+### 6. Add Constrained Prompt Storage
+
+- Place filesystem access behind a dedicated Prompt Store/service boundary.
+- Resolve all filenames directly beneath the configured prompt root and reject traversal.
+- Permit only the agreed markdown extension and reject empty content.
+- Implement the approved safe-write strategy, including flush/replace behavior and backup/recovery if selected.
+- Preserve file encoding and provide explicit failures for read-only or unavailable storage.
+- Do not modify any Job row when prompt defaults change.
+- Add unit tests for valid reads/writes, traversal, invalid names, empty content, filesystem failures, and unchanged Job provenance.
+
+### 7. Build the Settings UI
+
+- Register a Settings landing page and navigation entry.
+- Add separate pages or panels for Document Types, Person Roles, and Prompts.
+- Keep pages responsible for orchestration and notifications only.
+- Use service callbacks for all mutations.
+- Explain stable codes, inactive historical entries, and future-only prompt effects in the UI.
+- Do not render raw environment values or secrets.
+
+### 8. Verification and Rollout
+
+- Run focused service tests before UI integration tests.
+- Verify reorder behavior against Documents with one and many Sources.
+- Verify ordered transcription rendering and V4.1 previous/next navigation after reorder.
+- Verify inactive registry behavior in both historical display and create/edit selectors.
+- Verify prompt changes are picked up by newly created Jobs while historical Jobs retain frozen content/hash.
+- Run the relevant regression suite.
+
+## Migration and Compatibility Notes
+
+- No new table is expected solely for reordering; `Source.page_number` remains authoritative.
+- A uniqueness constraint on `(document_id, page_number)` should be evaluated before scope freeze. If added, migration and collision-safe update behavior must be designed for both supported databases.
+- Existing registry records remain valid.
+- Prompt editing changes mutable application files, not database provenance already captured on Jobs.
+- V4.2 must not require users to recreate existing Sources, Documents, People, roles, or types.
+
+## Proposed Delivery Order
+
+1. Freeze the remaining decisions.
+2. Implement and verify Source reorder service semantics.
+3. Build the reorder UI.
+4. Implement registry maintenance service operations.
+5. Implement the Prompt Store and safety policy.
+6. Build Settings pages.
+7. Run integration and regression verification.
+
+## Draft Done Criteria
+
+- All V4.2 acceptance criteria are testable and satisfied.
+- Reordering is atomic, conflict-aware, contiguous, and cross-database compatible.
+- Settings mutations cross explicit service or adapter boundaries.
+- Registry codes cannot be accidentally changed.
+- Prompt writes cannot escape the configured directory or rewrite historical provenance.
+- No secret or raw environment editor exists.
+- V4.1 workflows remain intact.
+
+## Related Local References
+
+- [Draft V4.2 Scope Boundary](scope_boundary_v4_2.md)
+- [V4.1 Implementation Plan](../ver4.1/implementation_plan_v4_1.md)
+- [V4 Implementation Plan](../ver4/implementation_plan_v4.md)
+- [V4 Error Handling Policy](../ver4/error_handling_v4.md)
+

docs/ver4.2/scope_boundary_v4_2.md

@@ -0,0 +1,130 @@
+# Draft V4.2 Scope Boundary
+
+This document defines the proposed boundary for the second incremental revision to Version 4. It is intentionally a draft until V4.1 has been used and the remaining workflows have been validated.
+
+## Purpose
+
+- Allow correction of Source page order after import.
+- Provide a constrained Settings area for safe maintenance of selected application-managed configuration.
+- Avoid exposing secrets, restart-sensitive settings, or unrestricted filesystem editing through the UI.
+
+## Proposed In Scope
+
+### 1. Source Page Reordering
+
+- Allow Sources within one Document to be reordered after import.
+- Present the current order using page number and a recognizable source label or preview.
+- Persist the complete intended order atomically.
+- Renumber the affected Document's Sources to a contiguous sequence beginning at 1.
+- Keep all Sources attached to their existing Document.
+- Ensure transcription rendering and previous/next navigation use the updated order.
+- Detect stale or invalid reorder submissions and fail without partial mutation.
+
+### 2. Settings Navigation
+
+- Add a Settings entry to application navigation.
+- Provide separate, clearly described settings areas rather than a raw configuration editor.
+- Restrict V4.2 settings to application-managed values that can be validated and safely changed at runtime.
+
+### 3. Document Type Maintenance
+
+- List active and inactive Document Types.
+- Add new types with a stable unique code and user-facing label.
+- Edit mutable labels and sort order.
+- Activate or deactivate types without invalidating historical Documents.
+- Do not allow changing a stable code after creation.
+- Do not delete types that are referenced by Documents.
+
+### 4. Person Role Maintenance
+
+- List active and inactive Person Roles.
+- Add new roles with a stable unique code and user-facing label.
+- Edit mutable labels.
+- Activate or deactivate roles without invalidating historical links.
+- Do not allow changing a stable code after creation.
+- Do not delete roles that are referenced by document-person links.
+
+### 5. Prompt Maintenance
+
+- List prompt markdown files from the configured prompt directory.
+- View a prompt with a concise explanation of its purpose and use.
+- Edit an existing prompt as plain markdown text.
+- Validate the filename boundary and reject empty prompt content.
+- Save changes explicitly and report filesystem failures.
+- Preserve submission-time prompt text and hash already frozen on existing Jobs.
+- Define a safe-write and recovery approach before this feature is considered final scope.
+
+## Proposed Out of Scope
+
+- Viewing or editing raw `.env` files.
+- Displaying or changing provider API keys and other secrets.
+- Editing host, port, database connection, upload paths, or other restart-sensitive runtime settings.
+- Arbitrary file browsing or arbitrary prompt paths.
+- Runtime theme/CSS editing.
+- Installing themes or plugins.
+- Source movement between Documents as part of reordering.
+- Automatic ordering based on filenames, OCR, or image content.
+- FamilySearch API synchronization.
+- A generic external-reference registry.
+- Ancestry references and Google Maps links.
+
+## Proposed Design Decisions
+
+### A. Reordering Is Set-Based
+
+- The client submits the full ordered list of Source IDs for one Document.
+- The service validates membership, completeness, duplicates, and authorization/context before writing.
+- All page-number updates occur in one transaction.
+
+### B. Registry Codes Are Immutable
+
+- Document Type and Person Role codes are stable identifiers.
+- Labels and active state remain mutable.
+- Historical references remain valid when a registry entry is inactive.
+
+### C. No Raw Environment Editor
+
+- `.env` may contain secrets and values that are not safely reloadable.
+- V4.2 exposes only purpose-built forms backed by explicit validation and service methods.
+
+### D. Prompt Editing Is Constrained
+
+- Prompt maintenance is limited to direct children of the configured prompt directory.
+- Existing Job provenance is never rewritten when a prompt file changes.
+- The UI must distinguish editing the default for future submissions from inspecting historical Job prompts.
+
+## Decisions Required Before Scope Freeze
+
+1. Choose the reorder interaction: move-up/down controls, drag-and-drop, or both.
+2. Decide whether reordering is allowed while the Document has a queued or processing Job.
+3. Define prompt backup, atomic-write, and recovery behavior.
+4. Decide whether prompt creation and deletion are needed or whether V4.2 edits existing prompts only.
+5. Confirm whether registry sort-order maintenance is needed for Person Roles as well as Document Types.
+6. Confirm that settings changes remain local to the current installation and do not require an API surface.
+
+## Draft Acceptance Criteria
+
+1. Reordering a Document's Sources produces contiguous page numbers and updates every ordered view consistently.
+2. Invalid, incomplete, duplicate, cross-Document, or stale reorder requests make no changes.
+3. Document Type and Person Role maintenance preserves stable codes and historical references.
+4. Inactive registry entries remain visible on historical records but are excluded from default create selectors.
+5. Prompt edits are restricted to valid markdown files in the configured prompt directory.
+6. A prompt edit affects future Jobs only and leaves stored Job provenance unchanged.
+7. No Settings page exposes secrets or unrestricted filesystem access.
+8. Focused service and UI tests pass without regressing V4.1 workflows.
+
+## Scope Freeze Gate
+
+V4.2 implementation should not begin until:
+
+- V4.1 has been used sufficiently to validate priorities.
+- The six open decisions above are resolved.
+- The prompt-write safety policy is documented.
+- The implementation plan is revised from draft to committed delivery plan.
+
+## Related Local References
+
+- [Draft V4.2 Implementation Plan](implementation_plan_v4_2.md)
+- [V4.1 Scope Boundary](../ver4.1/scope_boundary_v4_1.md)
+- [V4 Architecture](../ver4/architecture_v4.md)
+- [V4 Schema](../ver4/schema_v4.md)