generated from john/python-template
Finalized v4.3 scope
This commit is contained in:
Jim Lancaster
@@ -1,14 +1,17 @@
|
||||
# Draft Implementation Plan (Version 4.3)
|
||||
# Implementation Plan (Version 4.3)
|
||||
|
||||
## Goal
|
||||
|
||||
Prepare a safe implementation path for constrained application settings. This plan remains provisional until the V4.3 scope-freeze decisions are resolved.
|
||||
Deliver constrained, installation-local application settings while preserving the completed V4.2 behavioral baseline and historical provenance.
|
||||
|
||||
## Planning Constraints
|
||||
|
||||
- V4, V4.1, and V4.2 remain the behavioral baseline.
|
||||
- Settings must use explicit domain operations rather than direct database, environment-file, or arbitrary filesystem access from UI pages.
|
||||
- Prompt changes must preserve historical Job provenance and use a defined safe-write policy.
|
||||
- Source Page Reordering is excluded.
|
||||
- Database, integration, and UI tests must use confirmed isolated test data and must never modify `data/transcription.db`.
|
||||
- Potentially destructive tests must run only through `tools/run_destructive_tests.py`.
|
||||
|
||||
## Expected Project Impact
|
||||
|
||||
@@ -16,59 +19,67 @@ Prepare a safe implementation path for constrained application settings. This pl
|
||||
| --- | --- |
|
||||
| Documents service | Expand controlled Document Type maintenance operations. |
|
||||
| People service | Expand controlled Person Role maintenance operations. |
|
||||
| Prompt adapter/service | Add constrained listing, reading, validation, and safe writing of prompt artifacts. |
|
||||
| Prompt adapter/service | Add constrained listing, reading, validation, atomic writing, backup, and explicit recovery of existing prompt artifacts. |
|
||||
| UI composition/navigation | Register Settings routes and navigation without moving persistence into UI code. |
|
||||
| Tests | Add registry lifecycle, prompt safety, and UI workflow coverage. |
|
||||
| Tests | Add isolated registry lifecycle, prompt safety, and UI workflow coverage. |
|
||||
|
||||
## Proposed Implementation Phases
|
||||
## Implementation Phases
|
||||
|
||||
### 1. Resolve Scope-Freeze Decisions
|
||||
### 1. Define Service Contracts
|
||||
|
||||
- Define prompt atomic-write, backup, and recovery policy.
|
||||
- Decide whether prompt creation/deletion is excluded.
|
||||
- Finalize registry ordering requirements.
|
||||
- Remove the Draft designation only after these decisions are reflected in scope and acceptance criteria.
|
||||
|
||||
### 2. Define Service Contracts
|
||||
|
||||
- Define Document Type maintenance commands for create, relabel, sort, activate, and deactivate.
|
||||
- Define Person Role maintenance commands for create, relabel, activate, and deactivate.
|
||||
- Define a Prompt Store interface for constrained list/read/write behavior.
|
||||
- Define Document Type maintenance commands for create, relabel, sort, activate, deactivate, and delete-if-unreferenced.
|
||||
- Define Person Role maintenance commands for create, relabel, activate, deactivate, and delete-if-unreferenced.
|
||||
- Define a Prompt Store interface for constrained list, read, write, backup-status, and explicit recovery behavior.
|
||||
- Map validation, conflict, not-found, dependency, and filesystem failures to existing `AppError` categories.
|
||||
|
||||
### 3. Expand Registry Maintenance Services
|
||||
### 2. Expand Registry Maintenance Services
|
||||
|
||||
- Reuse existing Document and People service ownership.
|
||||
- Add explicit write methods rather than passing UI-mutated ORM objects directly where practical.
|
||||
- Normalize and validate new stable codes.
|
||||
- Reject duplicate codes deterministically.
|
||||
- Block deletion or omit deletion entirely; use activation state for lifecycle management.
|
||||
- Permit deletion only after a service-owned reference check proves the entry is unreferenced.
|
||||
- Reject deletion of referenced entries deterministically without partial mutation.
|
||||
- Permit label changes whether or not an entry is referenced.
|
||||
- Preserve inactive entries for historical reads.
|
||||
- Add service tests for create, relabel, activation, deactivation, duplicates, immutable codes, and referenced records.
|
||||
- Maintain existing Document Type sort order.
|
||||
- Order Person Roles deterministically by label and then code without adding a schema field.
|
||||
- Add service tests for create, relabel, activation, deactivation, duplicates, immutable codes, ordering, allowed deletion, and blocked referenced deletion.
|
||||
|
||||
### 4. Add Constrained Prompt Storage
|
||||
### 3. Add Constrained Prompt Storage
|
||||
|
||||
- Place filesystem access behind a dedicated Prompt Store/service boundary.
|
||||
- Resolve all filenames directly beneath the configured prompt root and reject traversal.
|
||||
- Permit only the agreed markdown extension and reject empty content.
|
||||
- Implement the approved safe-write strategy, including flush/replace behavior and backup/recovery if selected.
|
||||
- Permit only existing files with the agreed Markdown extension and reject empty content.
|
||||
- Exclude prompt creation and deletion.
|
||||
- Write new content to a sibling temporary file, flush and sync it, preserve the active file as the sole previous-version backup, and atomically replace the active file.
|
||||
- Expose explicit backup recovery through the same filename validation and safe-write path; never perform automatic rollback.
|
||||
- Clean up temporary files after failed writes while preserving the active prompt and any valid backup.
|
||||
- Preserve file encoding and provide explicit failures for read-only or unavailable storage.
|
||||
- Do not modify any Job row when prompt defaults change.
|
||||
- Add unit tests for valid reads/writes, traversal, invalid names, empty content, filesystem failures, and unchanged Job provenance.
|
||||
- Add unit tests for valid reads/writes, traversal, invalid names, nonexistent-file creation attempts, empty content, atomic replacement failures, single-backup rotation, explicit recovery, filesystem failures, and unchanged Job provenance.
|
||||
|
||||
### 5. Build the Settings UI
|
||||
### 4. Build the Settings UI
|
||||
|
||||
- Register a Settings landing page and navigation entry.
|
||||
- Add separate pages or panels for Document Types, Person Roles, and Prompts.
|
||||
- Keep pages responsible for orchestration and notifications only.
|
||||
- Use service callbacks for all mutations.
|
||||
- Explain stable codes, inactive historical entries, and future-only prompt effects in the UI.
|
||||
- Present deletion only for unreferenced registry entries and preserve clear conflict feedback if references appear before submission.
|
||||
- Present prompt backup availability and recovery as an explicit operator action.
|
||||
- Do not render raw environment values or secrets.
|
||||
- Add no settings API routes.
|
||||
|
||||
### 6. Verification and Rollout
|
||||
### 5. Verification and Rollout
|
||||
|
||||
- Confirm every database, integration, and UI test is configured for an isolated test database before execution.
|
||||
- Never run those tests against live data and never modify or replace `data/transcription.db`.
|
||||
- Invoke potentially destructive tests only through `tools/run_destructive_tests.py`.
|
||||
- Run focused service tests before UI integration tests.
|
||||
- Verify inactive registry behavior in both historical display and create/edit selectors.
|
||||
- Verify referenced entries can be relabeled or deactivated but not deleted.
|
||||
- Verify unreferenced entries can be deleted.
|
||||
- Verify prompt changes are picked up by newly created Jobs while historical Jobs retain frozen content/hash.
|
||||
- Run the relevant regression suite.
|
||||
|
||||
@@ -77,27 +88,33 @@ Prepare a safe implementation path for constrained application settings. This pl
|
||||
- Existing registry records remain valid.
|
||||
- Prompt editing changes mutable application files, not database provenance already captured on Jobs.
|
||||
- V4.3 must not require users to recreate existing Sources, Documents, People, roles, or types.
|
||||
- Person Role ordering requires no schema migration.
|
||||
- Registry deletion introduces no cascade behavior; references always block deletion.
|
||||
|
||||
## Proposed Delivery Order
|
||||
## Delivery Order
|
||||
|
||||
1. Freeze the remaining decisions.
|
||||
2. Implement registry maintenance service operations.
|
||||
3. Implement the Prompt Store and safety policy.
|
||||
4. Build Settings pages.
|
||||
5. Run integration and regression verification.
|
||||
1. Implement registry maintenance service operations.
|
||||
2. Implement the Prompt Store and safety policy.
|
||||
3. Build Settings pages.
|
||||
4. Run isolated integration and regression verification.
|
||||
|
||||
## Draft Done Criteria
|
||||
## Done Criteria
|
||||
|
||||
- All V4.3 acceptance criteria are testable and satisfied.
|
||||
- Settings mutations cross explicit service or adapter boundaries.
|
||||
- Registry codes cannot be accidentally changed.
|
||||
- Referenced registry entries can be relabeled or deactivated but cannot be deleted.
|
||||
- Unreferenced registry entries can be deleted without cascade behavior.
|
||||
- Prompt writes cannot escape the configured directory or rewrite historical provenance.
|
||||
- Prompt writes are atomic, retain one backup, and support explicit recovery.
|
||||
- No secret or raw environment editor exists.
|
||||
- No settings API surface exists.
|
||||
- V4.1 and V4.2 workflows remain intact.
|
||||
- Verification does not touch live data or `data/transcription.db`.
|
||||
|
||||
## Related Local References
|
||||
|
||||
- [Draft V4.3 Scope Boundary](scope_boundary_v4_3.md)
|
||||
- [V4.3 Scope Boundary](scope_boundary_v4_3.md)
|
||||
- [V4.2 Implementation Plan](../ver4.2/implementation_plan_v4_2.md)
|
||||
- [V4.1 Implementation Plan](../ver4.1/implementation_plan_v4_1.md)
|
||||
- [V4 Implementation Plan](../ver4/implementation_plan_v4.md)
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
# Draft V4.3 Scope Boundary
|
||||
# V4.3 Scope Boundary
|
||||
|
||||
This document defines the proposed boundary for the constrained-settings revision that follows the V4.2 evidence-and-provenance work. It is intentionally a draft until the earlier revisions have been used and the remaining workflows have been validated.
|
||||
This document defines the frozen boundary for the constrained-settings revision that follows the completed V4.2 evidence-and-provenance work. V4, V4.1, and V4.2 remain the behavioral and architecture baseline.
|
||||
|
||||
## Purpose
|
||||
|
||||
- Provide a constrained Settings area for safe maintenance of selected application-managed configuration.
|
||||
- Avoid exposing secrets, restart-sensitive settings, or unrestricted filesystem editing through the UI.
|
||||
|
||||
## Proposed In Scope
|
||||
## In Scope
|
||||
|
||||
### 1. Settings Navigation
|
||||
|
||||
@@ -22,7 +22,8 @@ This document defines the proposed boundary for the constrained-settings revisio
|
||||
- Edit mutable labels and sort order.
|
||||
- Activate or deactivate types without invalidating historical Documents.
|
||||
- Do not allow changing a stable code after creation.
|
||||
- Do not delete types that are referenced by Documents.
|
||||
- Allow deletion only when no Document references the type.
|
||||
- Allow label changes regardless of whether the type is referenced.
|
||||
|
||||
### 3. Person Role Maintenance
|
||||
|
||||
@@ -31,7 +32,9 @@ This document defines the proposed boundary for the constrained-settings revisio
|
||||
- Edit mutable labels.
|
||||
- Activate or deactivate roles without invalidating historical links.
|
||||
- Do not allow changing a stable code after creation.
|
||||
- Do not delete roles that are referenced by document-person links.
|
||||
- Order roles deterministically by label and then code; do not add persisted role sort order.
|
||||
- Allow deletion only when no document-person link references the role.
|
||||
- Allow label changes regardless of whether the role is referenced.
|
||||
|
||||
### 4. Prompt Maintenance
|
||||
|
||||
@@ -41,9 +44,17 @@ This document defines the proposed boundary for the constrained-settings revisio
|
||||
- Validate the filename boundary and reject empty prompt content.
|
||||
- Save changes explicitly and report filesystem failures.
|
||||
- Preserve submission-time prompt text and hash already frozen on existing Jobs.
|
||||
- Define a safe-write and recovery approach before this feature is considered final scope.
|
||||
- Edit existing prompt files only; prompt creation and deletion are excluded.
|
||||
- Save through a sibling temporary file, flush and sync file content, retain one previous-version backup, and atomically replace the active file.
|
||||
- Provide an explicit recovery operation that restores the retained backup through the same safe-write path; do not silently roll back a failed or unwanted edit.
|
||||
|
||||
## Proposed Out of Scope
|
||||
### 5. Deployment Boundary
|
||||
|
||||
- Settings changes apply only to the current installation.
|
||||
- V4.3 adds no settings API endpoints.
|
||||
- Service contracts must remain independent of the UI so a separately authorized API can be considered later.
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- Viewing or editing raw `.env` files.
|
||||
- Displaying or changing provider API keys and other secrets.
|
||||
@@ -54,17 +65,22 @@ This document defines the proposed boundary for the constrained-settings revisio
|
||||
- Source page renumbering or reordering.
|
||||
- Source movement between Documents.
|
||||
- Automatic ordering based on filenames, OCR, or image content.
|
||||
- Prompt creation, deletion, and multi-version history.
|
||||
- Persisted sort-order maintenance for Person Roles.
|
||||
- Settings read or write API endpoints.
|
||||
- FamilySearch API synchronization.
|
||||
- A generic external-reference registry.
|
||||
- Ancestry references and Google Maps links.
|
||||
|
||||
## Proposed Design Decisions
|
||||
## Locked Design Decisions
|
||||
|
||||
### A. Registry Codes Are Immutable
|
||||
|
||||
- Document Type and Person Role codes are stable identifiers.
|
||||
- Labels and active state remain mutable.
|
||||
- Historical references remain valid when a registry entry is inactive.
|
||||
- Labels may be updated for referenced and unreferenced entries.
|
||||
- Unreferenced entries may be deleted; referenced entries may only be deactivated.
|
||||
|
||||
### B. No Raw Environment Editor
|
||||
|
||||
@@ -77,35 +93,60 @@ This document defines the proposed boundary for the constrained-settings revisio
|
||||
- Existing Job provenance is never rewritten when a prompt file changes.
|
||||
- The UI must distinguish editing the default for future submissions from inspecting historical Job prompts.
|
||||
|
||||
## Decisions Required Before Scope Freeze
|
||||
### D. Prompt Writes Are Atomic and Recoverable
|
||||
|
||||
1. Define prompt backup, atomic-write, and recovery behavior.
|
||||
2. Decide whether prompt creation and deletion are needed or whether V4.3 edits existing prompts only.
|
||||
3. Confirm whether registry sort-order maintenance is needed for Person Roles as well as Document Types.
|
||||
4. Confirm that settings changes remain local to the current installation and do not require an API surface.
|
||||
- Writes use a sibling temporary file and atomic replacement so readers observe either the old or new complete prompt.
|
||||
- The immediately previous prompt version is retained as the sole backup.
|
||||
- Recovery is an explicit operator action and uses the same validated safe-write path.
|
||||
- Prompt creation and deletion are not available in V4.3.
|
||||
|
||||
## Draft Acceptance Criteria
|
||||
### E. Person Role Ordering Is Deterministic, Not Persisted
|
||||
|
||||
- Person Roles are ordered by label and then stable code.
|
||||
- V4.3 does not add a `sort_order` field to Person Roles.
|
||||
- Document Type sort-order maintenance remains in scope because `DocumentType.sort_order` is already part of the V4 schema.
|
||||
|
||||
### F. Settings Are Installation-Local
|
||||
|
||||
- V4.3 provides Settings through the local application UI and domain services only.
|
||||
- No settings API surface is introduced.
|
||||
|
||||
## Data and Compatibility Policy
|
||||
|
||||
- V4.3 does not rewrite existing Documents, document-person links, Jobs, Sources, execution evidence, or prompt provenance.
|
||||
- Deactivation preserves referenced registry entries for historical display while excluding them from default create selectors.
|
||||
- Deletion checks are performed at the service boundary and must fail deterministically when references exist.
|
||||
- Prompt files are constrained to existing Markdown files that are direct children of the configured prompt root.
|
||||
- Settings UI code performs no direct database, environment-file, or arbitrary filesystem mutations.
|
||||
- Source page numbering and ordering behavior is unchanged.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
1. Document Type and Person Role maintenance preserves stable codes and historical references.
|
||||
2. Inactive registry entries remain visible on historical records but are excluded from default create selectors.
|
||||
3. Prompt edits are restricted to valid markdown files in the configured prompt directory.
|
||||
4. A prompt edit affects future Jobs only and leaves stored Job provenance unchanged.
|
||||
5. No Settings page exposes secrets or unrestricted filesystem access.
|
||||
6. Focused service and UI tests pass without regressing V4.1 or V4.2 workflows.
|
||||
3. Labels can be changed for referenced or unreferenced registry entries.
|
||||
4. An unreferenced Document Type or Person Role can be deleted, while deletion of a referenced entry fails without partial mutation.
|
||||
5. Document Type sort order is maintainable; Person Roles use deterministic label/code ordering without a schema addition.
|
||||
6. Prompt edits are restricted to existing Markdown files directly beneath the configured prompt directory.
|
||||
7. Prompt saves use atomic replacement, retain exactly one previous-version backup, and support explicit recovery.
|
||||
8. A prompt edit affects future Jobs only and leaves stored Job provenance unchanged.
|
||||
9. No Settings page exposes secrets, unrestricted filesystem access, or a settings API.
|
||||
10. Focused service and UI tests pass without regressing V4.1 or V4.2 workflows.
|
||||
11. Database, integration, and UI tests use confirmed isolated test data and never modify `data/transcription.db`; potentially destructive tests run only through `tools/run_destructive_tests.py`.
|
||||
|
||||
## Scope Freeze Gate
|
||||
|
||||
V4.3 implementation should not begin until:
|
||||
V4.3 is sufficiently frozen to begin implementation:
|
||||
|
||||
- V4.1 has been used sufficiently to validate priorities.
|
||||
- V4.2 evidence and provenance work has been completed and validated.
|
||||
- The four open decisions above are resolved.
|
||||
- The prompt-write safety policy is documented.
|
||||
- The implementation plan is revised from draft to committed delivery plan.
|
||||
- V4.2 is the completed behavioral baseline.
|
||||
- Registry lifecycle and ordering behavior are resolved.
|
||||
- Prompt lifecycle, atomic-write, backup, and recovery behavior are resolved.
|
||||
- The installation-local deployment boundary is resolved.
|
||||
- The implementation plan is a committed delivery plan.
|
||||
|
||||
## Related Local References
|
||||
|
||||
- [Draft V4.3 Implementation Plan](implementation_plan_v4_3.md)
|
||||
- [V4.3 Implementation Plan](implementation_plan_v4_3.md)
|
||||
- [V4.2 Scope Boundary](../ver4.2/scope_boundary_v4_2.md)
|
||||
- [V4.1 Scope Boundary](../ver4.1/scope_boundary_v4_1.md)
|
||||
- [V4 Architecture](../ver4/architecture_v4.md)
|
||||
|
||||
Reference in New Issue
Block a user