V4.6 Code review: V4 architecture-conformance and reliability release

2026-08-16 09:38:59 -05:00
parent 7054cd8af9
commit d1321fd709
3 changed files with 635 additions and 0 deletions
@@ -0,0 +1,338 @@
 # Implementation Plan (Version 4.6)
 ## Goal
 Bring the completed V4.5 implementation into conformance with the documented V4 service, provenance, transaction,
 worker-concurrency, and evidence boundaries while preserving all historical Source and execution behavior.
 ## Planning Status
 - V4.5 is the completed implementation baseline.
 - The V4.6 scope is frozen and sufficiently detailed to begin implementation.
 - V4.6 is a compatibility-preserving architecture and reliability revision, not a V5 redesign.
 - Scope additions require an explicit amendment or a later revision.
 ## Planning Constraints
 - Original Source bytes and historical evidence remain immutable.
 - Existing JobSource compatibility projections remain supported.
 - Human revisions remain independent and retain display and print precedence.
 - Candidate promotion semantics remain unchanged.
 - Provider network work occurs outside database transactions.
 - Concurrent page tasks never share an AsyncSession.
 - UI pages do not own persistence transactions.
 - Refactoring proceeds behind stable public service contracts.
 - New shared code has a cohesive domain or infrastructure owner; no generic helper dumping grounds are introduced.
 - Database, integration, and UI tests use confirmed isolated data and never modify `data/transcription.db`.
 - Potentially destructive tests run only through `tools/run_destructive_tests.py`.
 ## Expected Project Impact
 | Area | Expected impact |
 | --- | --- |
 | Job submission | Add one typed frozen request specification used by normal and retranscription workflows. |
 | Upload workflow | Move transaction and compensation ownership from the UI and direct persistence functions into orchestration. |
 | Source media | Validate content and extension consistently while preserving original bytes. |
 | Job service | Add atomic claim behavior and stable lifecycle transitions. |
 | Provider adapters | Remove mutable per-call evidence state and carry evidence through results and errors. |
 | Worker workflows | Add bounded concurrent page execution with isolated sessions and durable page outcomes. |
 | Source service | Extract cohesive collaborators behind the existing SourceService facade. |
 | Evidence persistence | Make attempt numbering, first selection, artifact creation, and cleanup conflict-safe. |
 | Read paths | Add lightweight Source and attempt projections that defer large evidence fields. |
 | Compatibility | Isolate legacy transcription and storage aliases without adding new behavior. |
 | Documentation | Align status terminology, ownership, concurrency, and provenance descriptions. |
 | Tests | Add failure-injection, concurrency, query-shape, compatibility, and preservation coverage. |
 ## Implementation Phases
 ### 1. Add Characterization and Failure-Injection Coverage
 - Record the current externally visible Source, Job, JobSource, attempt, artifact, revision, candidate, and export
  behavior before extraction.
 - Add tests proving:
  - Historical Jobs with missing request fields remain readable.
  - JobSource SDK snapshots retain their existing evidence label.
  - Source preferred-machine fields and human revisions survive failed processing.
  - Per-page outcomes remain durable before sibling pages finish.
  - Candidate promotion updates both Source projection fields atomically.
 - Add reusable test doubles for:
  - Provider success, response failure, connection failure, timeout, and cancellation.
  - Database flush and commit failure at named workflow phases.
  - Filesystem write, replace, and unlink failure.
  - Controlled overlapping page calls.
 - Keep all providers fake and all databases and filesystem roots isolated.
 - Use these tests as behavior gates for every later extraction.
 ### 2. Define the Frozen Job Submission Specification
 - Move runtime prompt loading, prompt hashing, requested provider/model resolution, and explicit parameter-state capture
  into a focused prompt/job-submission module.
 - Define one immutable typed value containing:
  - Requested provider.
  - Requested model.
  - Prompt name and SHA-256.
  - System and user prompt content.
  - Temperature and top-p values.
  - Explicitly supplied versus omitted state where applicable.
 - Reuse the constrained PromptStore's path policy rather than maintaining a second incompatible prompt-path policy.
 - Keep prompt editing, backup, and recovery behavior in the existing prompt-storage owner.
 - Validate model selection against `provider_models` for both normal transcription and retranscription.
 - Populate all new Job request fields before adding the Job to the session.
 - Stop mutating Job provider and model during attempt persistence.
 - Store provider-reported resolved identity on ExecutionAttempt only.
 - Retain compatibility reads for historical Jobs whose request fields are null.
 - Add tests for pre-queue completeness, allowlist rejection, prompt-edit isolation, requested/resolved model differences, and
  historical null behavior.
 ### 3. Establish Content-Aware Source Media Policy
 - Move accepted Source extensions, canonical MIME mapping, validation, and managed payload loading into one Source-media
  module.
 - Validate filename and non-empty content first.
 - Validate JPEG, PNG, and TIFF using bounded raster decoding appropriate to the existing Pillow dependency.
 - Validate PDF structure without rendering or modifying the file.
 - Reject a supported extension whose content is corrupt or belongs to a different supported type.
 - Return a typed validated-media result containing canonical media type, digest, byte size, and original bytes or a
  stable staged reference.
 - Ensure hashing uses the exact bytes later persisted as the original Source.
 - Keep orientation normalization separate and downstream of original-media validation.
 - Preserve compatibility aliases for existing MIME and payload-loading imports.
 - Add fixtures for valid, corrupt, truncated, mismatched, uppercase-extension, empty, and unsupported content.
 ### 4. Replace Direct Upload Persistence with an Ingestion Workflow
 - Introduce a Source-ingestion workflow that accepts validated inputs and service dependencies.
 - Generate Document and Source identifiers before file placement so managed relative paths remain deterministic.
 - Stage all Source files beneath the configured application root.
 - Build the frozen Job submission specification before starting database mutation.
 - Open one workflow-owned transaction.
 - Use DocumentService, JobService, and SourceService session-aware writes to create:
  - The Document when required.
  - The queued Job.
  - Ordered Sources.
  - Pending JobSource projections.
 - Commit once after every record has flushed successfully.
 - Delete files staged by the operation on validation, flush, or commit failure.
 - Return a typed result only after commit succeeds.
 - Move worker notification after successful workflow return.
 - Remove session-scope imports and direct persistence ownership from the Jobs UI.
 - Keep old storage entry points as thin compatibility adapters that invoke the workflow.
 - Add tests for deterministic ordering, page numbering, duplicate names, every failure phase, cleanup, commit-before-notify,
  and no direct UI persistence.
 ### 5. Add Atomic Job Claiming and Canonical Lifecycle Semantics
 - Add `claim_next_queued_job` to JobService or a Job-owned persistence collaborator.
 - Select candidates deterministically by creation time and UUID.
 - Transition one candidate using a conditional write that succeeds only while status is queued.
 - Retry candidate selection when another claimant wins, without issuing provider work for the lost claim.
 - Commit the processing transition before provider-input preparation.
 - Route worker processing exclusively through the claim operation.
 - Preserve stale-processing recovery and require recovered Jobs to be claimed again.
 - Define `transcribed` as the newly emitted all-success terminal status.
 - Continue reading `completed` as a historical terminal status.
 - Add tests with concurrent claimers on SQLite and the portable SQL path used for PostgreSQL.
 - Add a lifecycle transition-table test covering queued, processing, transcribed, partial success, failed, cancellation,
  resubmission, and stale recovery.
 ### 6. Make Provider Calls Stateless
 - Extend provider success and error contracts so each call directly carries:
  - RequestManifest.
  - TransportEvidence or explicit no-response evidence.
  - Failure phase.
  - Parsed SDK snapshot when available.
  - Normalized metadata when available.
 - Remove `current_request_manifest` and `current_transport_evidence` from provider adapters and worker persistence.
 - Ensure local timeout and cancellation retain the call-specific request manifest without reading mutable adapter state.
 - Keep response-body capture and safe-header filtering in the OpenRouter adapter.
 - Preserve the distinction between exact OpenRouter-boundary response bytes, SDK snapshot, and normalized fields.
 - Decide adapter reuse by concurrency safety; create per-call adapters if the SDK client cannot safely support concurrent
  calls.
 - Add overlapping-call tests in which success, HTTP failure, timeout, and no-response failure complete in different
  orders.
 - Assert that no call receives another call's source digest, prompt hash, response body, or identifiers.
 ### 7. Extract Source-Owned Evidence and Artifact Collaborators
 - Keep SourceService as the public facade while moving cohesive implementation behind it.
 - Extract Source persistence and query behavior first without changing method contracts.
 - Extract ExecutionAttempt and JobSource compatibility persistence into a Source-owned evidence writer.
 - Extract processing-artifact file, integrity, and metadata behavior into a Source-owned artifact store.
 - Extract evidence export into a read-only projection builder.
 - Extract provider-input preparation so it coordinates:
  - Managed original Source access.
  - Orientation normalization.
  - Derivative artifact creation.
  - Exact SourceEvidenceReference construction.
 - Keep candidate promotion in Source persistence.
 - Keep normalization and quality analysis in their existing focused modules.
 - Make dependencies explicit through typed constructor parameters rather than importing service globals.
 - Do not allow extracted collaborators to call unrelated aggregate services.
 - Add contract tests showing SourceService behavior is unchanged while collaborators can be tested independently.
 ### 8. Make Artifact Creation and Cleanup Explicit
 - Define one artifact-creation operation that owns temporary path creation, atomic file replacement, database metadata,
  and compensation.
 - Track every path created before commit within the owning workflow.
 - On flush or commit failure, remove newly created content and surface cleanup failure separately when removal also
  fails.
 - Constrain temporary and final paths beneath the artifact root and preserve safe relative references.
 - Add startup or explicit maintenance cleanup for abandoned temporary files without deleting final referenced
  artifacts.
 - For deletion:
  - Verify retention policy and references before mutation.
  - Commit metadata changes according to the explicit deletion contract.
  - Attempt external cleanup.
  - Report incomplete cleanup with correlation identity when unlink fails.
 - Do not silently skip unsafe references; classify them as validation or conflict failures as appropriate.
 - Add tests for write, replace, flush, commit, unlink, cancellation, missing file, digest mismatch, and unsafe reference
  failures.
 ### 9. Make Attempt Persistence and First Selection Conflict-Safe
 - Stop relying solely on an unprotected `max(attempt_number) + 1` calculation.
 - Allocate or insert attempt numbers inside the protected Job processing boundary.
 - Treat the database uniqueness constraint as a final integrity guard and translate conflicts deterministically.
 - Ensure every completed provider call either persists one immutable attempt or leaves a visible persistence failure
  that does not masquerade as provider success.
 - Perform first-success Source selection with a conditional update requiring both preferred provenance and machine
  projection to remain unset.
 - When first-selection loses a race, retain the new successful attempt as a candidate.
 - Keep explicit candidate promotion as a separate atomic command.
 - Never modify earlier attempt text, transport evidence, warnings, or artifact associations.
 - Add concurrency tests for duplicate attempt allocation, simultaneous successes, success/failure overlap, candidate
  promotion, and transaction rollback.
 ### 10. Add Bounded Concurrent Page Processing
 - Add `worker_concurrency` to Settings with default `2`, positive validation, and a documented safe upper bound.
 - Refactor the single-page body of `process_queued_job` into one typed page-attempt operation.
 - Prepare each page's provider input and persist any derivative using its own session and committed preparation phase.
 - Release all database transactions before awaiting provider network work.
 - Use an asyncio semaphore or TaskGroup-based worker pattern to cap in-flight page calls.
 - Give every page outcome persistence operation its own session and transaction.
 - Shield only the minimum durable outcome write required to preserve a completed provider call during cancellation.
 - Collect typed success, failure, and cancellation outcomes without sharing mutable lists across tasks.
 - Compute the aggregate status after all started tasks reach durable outcomes.
 - Stop scheduling unstarted pages after cancellation or an externally terminal Job transition.
 - Preserve deterministic source ordering for scheduling and deterministic aggregate reporting independent of completion
  order.
 - Add tests for concurrency bounds `1`, `2`, and a larger configured value; out-of-order completion; mixed outcomes;
  timeout; cancellation; external stop; durable early completion; and no shared session.
 ### 11. Introduce Purpose-Specific Read Projections
 - Add a Source-list projection containing only fields needed by the Sources table.
 - Filter Document and Job scopes in SQL.
 - Add a Source-detail projection or explicit eager-loading contract for the relationships rendered by the page.
 - Add an attempt-summary projection containing candidate metadata, transcription text or an explicit preview policy,
  warning summary, and preferred-selection identity.
 - Defer transport body, SDK snapshot, request manifest, and other large evidence values from summaries.
 - Keep one explicit full-attempt evidence read for detailed inspection.
 - Keep evidence export complete and integrity-verified.
 - Replace all-sibling materialization in Source navigation with predecessor/successor queries using page number and UUID.
 - Add query-count and unloaded-attribute assertions so future changes cannot reintroduce accidental evidence loading.
 - Preserve deterministic ordering and avoid lazy database access from UI components.
 ### 12. Isolate Compatibility Adapters and Cohesive Shared Policies
 - Keep `services/transcription.py` as a documented compatibility facade with imports and aliases only.
 - Keep legacy upload/storage aliases as thin calls into the new ingestion and media owners.
 - Mark compatibility entry points in tests so removal can be evaluated separately.
 - Consolidate Document Type and Person Role label normalization in a registry-domain policy module.
 - Share canonical evidence digest primitives only where serialization and digest meaning are identical.
 - Do not merge prompt backup writing, original Source storage, and immutable artifact storage into one generic helper;
  their lifecycle and recovery semantics remain domain-owned.
 - Remove unused service-base state only after confirming no external caller depends on it.
 - Add import-compatibility tests and direct tests for each extracted policy.
 ### 13. Align UI and API Boundaries
 - Remove machine-local stored paths from routine Source Detail presentation.
 - Keep stored filename and evidence-safe identifiers available.
 - Ensure normal Job creation uses the configured provider and allowlisted model selector rather than arbitrary text input.
 - Keep retranscription Source and Document context locked.
 - Keep evidence export explicit and avoid loading export-only content during page render.
 - Continue translating AppError through established UI and API presenters without parsing error messages.
 - Keep sessions, session factories beyond dependency construction, and persistence operations out of reusable components.
 - Add UI tests for allowlisted selection, no path disclosure, unchanged candidate actions, and error preservation.
 ### 14. Align Authoritative Documentation
 - Update V4 architecture for:
  - Frozen Job request versus resolved attempt identity.
  - Atomic Job claims.
  - Bounded page concurrency and isolated page sessions.
  - Source facade and internal collaborators.
  - Explicit artifact lifecycle ownership.
 - Update V4 schema documentation only for actual additive schema changes.
 - Update V4 requirements and verification mapping for atomic claims, media validation, and concurrency tests.
 - Update V4 error handling for filesystem/database partial cleanup.
 - Define `transcribed` as the current all-success terminal status and `completed` as historical compatibility.
 - Correct stale V3 module descriptions and UI documentation that exposes or promises machine-local paths.
 - Update documentation together with the behavior it describes; do not prestate unimplemented behavior as complete.
 ### 15. Verification and Preservation Audit
 - Run focused pure tests for submission specs, media validation, provider contracts, query projections, and policies.
 - Run isolated service tests for claims, attempts, artifacts, ingestion, candidate selection, and compatibility.
 - Run worker tests with fake providers and controlled concurrent completion.
 - Run UI and API tests against isolated application state.
 - Run schema tests only through the required destructive-test wrapper when applicable.
 - Run the broader non-external regression suite after focused targets pass.
 - Verify that tests did not change:
  - `data/transcription.db`.
  - Curated Source files.
  - Existing artifact files.
  - Prompt files outside temporary fixtures.
 - Perform an explicit preservation audit showing that upgrading does not rewrite existing Job, Source, JobSource,
  ExecutionAttempt, ProcessingArtifact, or revision content.
 ## Delivery Order
 1. Characterization and failure-injection tests.
 2. Frozen submission specification and prompt ownership.
 3. Content-aware Source media validation.
 4. Ingestion workflow and UI transaction-boundary correction.
 5. Atomic Job claiming and lifecycle terminology.
 6. Stateless provider evidence.
 7. Source evidence/artifact/provider-input extraction behind SourceService.
 8. Artifact compensation and conflict-safe attempt/selection persistence.
 9. Purpose-specific read projections.
 10. Bounded concurrent page processing.
 11. Compatibility isolation, UI/API alignment, and documentation.
 12. Full preservation and regression verification.
 ## Done Criteria
 - All V4.6 acceptance criteria are implemented and testable.
 - New Jobs contain complete request provenance before queueing and are not rewritten by execution.
 - Every submitted model is allowlisted and requested/resolved identities remain distinct.
 - Upload UI code owns no database transaction.
 - Source content validation verifies supported media without modifying originals.
 - One queued Job can be claimed only once.
 - Provider evidence is call-local and safe under overlapping execution.
 - Page calls run concurrently within the configured bound and never share sessions.
 - Every actual provider call has one immutable attempt.
 - Concurrent first successes preserve one preferred result and every other success as a candidate.
 - Artifact commit and cleanup failures are visible and compensated where safely possible.
 - Source list and candidate views avoid loading full evidence bodies.
 - SourceService remains compatible while extracted responsibilities are independently testable.
 - Human revisions, preferred-machine projections, prior attempts, JobSource snapshots, and artifacts retain their original
  meaning.
 - The worker consistently emits transcribed, partial_success, or failed for new processing.
 - Verification uses isolated data, fake providers, and does not modify operator or curated evidence data.
 ## Related Local References
 - [V4.6 Scope Boundary](scope_boundary_v4_6.md)
 - [V4.5 Scope Boundary](../ver4.5/scope_boundary_v4_5.md)
 - [V4.5 Implementation Plan](../ver4.5/implementation_plan_v4_5.md)
 - [V4 Architecture](../ver4/architecture_v4.md)
 - [V4 Schema](../ver4/schema_v4.md)
 - [V4 Requirements](../ver4/requirements_v4.md)
 - [V4 Error Handling Policy](../ver4/error_handling_v4.md)
 - [AI Evidence and Provenance Invariant](../invariant/ai_evidence_and_provenance.md)
 - [Error Handling Invariant](../invariant/error_handling.md)
@@ -0,0 +1,296 @@
 # V4.6 Scope Boundary
 This document defines the frozen boundary for architecture conformance, worker reliability, and Source-service
 maintainability after the completed V4.5 revision. V4 through V4.5 remain the behavioral and evidence baseline.
 V4.6 corrects implementation divergence and reorganizes internal ownership without replacing the V4 data model,
 public identity rules, or preservation contracts.
 ## Purpose
 - Freeze complete Job request provenance before work is queued.
 - Make job claiming, attempt persistence, and first-success selection safe under concurrency.
 - Implement the bounded page-processing concurrency required by the V4 architecture.
 - Restore UI, workflow, service, transaction, provider, and filesystem ownership boundaries.
 - Separate cohesive Source-domain responsibilities behind compatibility-preserving service contracts.
 - Avoid loading large evidence fields for list and candidate-summary views.
 - Correct documentation that no longer describes the implemented lifecycle accurately.
 ## In Scope
 ### 1. Frozen Job Submission Specification
 - Every newly queued Job records its requested provider, model, prompt name, prompt hash, full prompt content,
  and explicitly configured generation parameters before the Job transaction commits.
 - Normal transcription and Source retranscription use the same typed submission-specification builder.
 - Provider and model values are resolved from validated application configuration.
 - Every model submitted through the UI or another application workflow is validated against the configured model
  allowlist.
 - A provider response may report a resolved model that differs from the requested model. The resolved provider and
  model belong to the immutable ExecutionAttempt and do not rewrite the Job request specification.
 - Existing Jobs with null or incomplete historical fields remain readable and are not backfilled with facts that
  were not frozen at submission.
 - Prompt edits continue to affect future Jobs only and never rewrite stored Job or attempt provenance.
 ### 2. Source Ingestion Workflow Ownership
 - Source ingestion is an application workflow, not a UI transaction and not a generic file utility.
 - The ingestion workflow owns:
  - Source-media validation.
  - Original-file staging and hashing.
  - Document lookup or creation when applicable.
  - Job creation from a frozen submission specification.
  - Ordered Source creation.
  - JobSource creation.
  - One database commit for the related records.
  - Compensating cleanup when persistence fails before the workflow completes.
 - UI pages resolve input, invoke the workflow, notify the worker only after successful commit, and navigate or
  display errors.
 - UI pages do not import session scopes, open transactions, execute queries, or commit and roll back persistence.
 - Workflow code coordinates service contracts rather than inserting Document, Job, Source, or JobSource rows
  directly.
 ### 3. Source Media Validation
 - Filename extension remains part of the accepted-format policy but is not sufficient proof of media type.
 - New Source uploads are validated as non-empty, structurally recognizable JPEG, PNG, TIFF, or PDF content.
 - The validated content type must agree with the supported filename extension.
 - Raster validation may decode enough content to establish that the file is a supported image, but validation does
  not transform, re-encode, or replace the original bytes.
 - PDF validation establishes that the content is a recognizable PDF without rendering, rewriting, or normalizing it.
 - Original uploaded bytes, SHA-256 digest, byte size, upload name, and evidence identity remain unchanged.
 - Historical Sources are not rejected merely because they predate content-aware validation.
 ### 4. Atomic Job Claiming
 - Claiming a queued Job is one persistence operation that conditionally transitions
  `queued -> processing`.
 - Two workers or application instances cannot both successfully claim the same queued Job.
 - Claim ordering remains deterministic by creation time and UUID.
 - A worker that loses a claim performs no provider call and creates no execution evidence for that Job.
 - Claim behavior remains portable across supported SQLite and PostgreSQL deployments.
 - Startup recovery may requeue stale processing Jobs according to existing policy, but recovery does not bypass the
  normal claim operation.
 ### 5. Stateless Provider Execution Evidence
 - Request manifest and transport evidence belong to one provider call and are returned through that call's result or
  exception.
 - Provider adapters do not expose mutable "current request" or "current response" state for worker persistence.
 - Success, HTTP failure, connection failure, local timeout, cancellation, and response-validation failure retain
  their own request and transport evidence without cross-attribution.
 - Provider adapters remain responsible for exact provider-boundary capture and safe-header allowlisting.
 - Provider calls occur outside database transactions.
 - Provider adapter instances may be reused only when their per-call state is concurrency-safe.
 ### 6. Bounded Page Concurrency
 - One claimed multi-page Job may process page provider calls concurrently.
 - A validated `worker_concurrency` setting defines the maximum number of in-flight page calls.
 - The default concurrency is `2`; allowed values are positive and bounded to a documented safe maximum.
 - The worker continues to orchestrate one claimed Job at a time within one application worker loop. V4.6 does not add
  distributed scheduling or simultaneous aggregate processing of multiple Jobs in one process.
 - Each page task uses its own database session for preparation and outcome persistence; an AsyncSession is never
  shared across concurrent page tasks.
 - Provider-input preparation completes and commits before that page's provider call begins.
 - Each completed page outcome is committed durably without waiting for every sibling page.
 - One page failure does not cancel successful independent pages.
 - Aggregate Job status is computed after every started page task reaches a durable outcome or explicit cancellation.
 - Cancellation prevents unstarted work and preserves durable outcomes from calls that already completed.
 - Concurrency does not change provider cost policy: quality warnings never trigger automatic provider calls.
 ### 7. Conflict-Safe Attempts and Machine Selection
 - Every actual provider call creates exactly one immutable ExecutionAttempt, including failure and timeout outcomes.
 - Attempt numbering remains unique for `(job_id, source_id, attempt_number)` and does not use an unprotected
  application-only `max + 1` assumption.
 - A duplicate persistence race fails deterministically without overwriting existing evidence.
 - The first successful attempt may establish preferred machine output only when the Source has no selected machine
  provenance at the atomic write boundary.
 - If another success wins first-selection concurrently, the losing successful attempt remains an unselected
  candidate.
 - Candidate promotion continues to update `Source.preferred_execution_attempt_id` and
  `Source.raw_transcription` atomically.
 - Human revision remains independent and is never cleared or changed by execution or promotion.
 ### 8. Processing Artifact Lifecycle
 - Processing-artifact content and database metadata have one explicit creation owner.
 - When artifact persistence fails or its enclosing commit fails, files created by that operation are removed when
  safely possible and the failure remains visible.
 - Temporary artifact files use constrained application-managed paths and are recoverable or removable after
  interruption.
 - Artifact integrity verification continues to distinguish unavailable content, malformed metadata, digest mismatch,
  and unsafe external references.
 - Deleting artifact metadata and deleting external content is an explicit partial-failure-capable operation.
 - A failed external-file deletion is logged with correlation identity and reported as incomplete cleanup rather than
  silent success.
 - No cleanup path deletes an artifact still referenced by retained immutable evidence.
 ### 9. Source Service Responsibility Boundaries
 - `SourceService` remains the public Source aggregate facade during V4.6.
 - Internally cohesive Source-domain collaborators may own:
  - Source persistence, queries, navigation, and revisions.
  - Source media policy and managed original-file access.
  - Prompt execution snapshots.
  - Provider-input resolution and normalization-artifact association.
  - ExecutionAttempt and JobSource projection persistence.
  - Processing-artifact storage and integrity.
  - Evidence export.
  - Provider execution and error translation.
 - Candidate promotion remains in the Source aggregate persistence boundary because it atomically changes the Source's
  selected provenance and compatibility text projection.
 - Orientation transformation remains in the focused normalization module.
 - Deterministic warning analysis remains in the focused quality module.
 - Extraction does not create a generic `utils.py`, `helpers.py`, or `common.py` service dumping ground.
 - Services remain independent; multi-service behavior stays in workflow functions or workflow objects.
 ### 10. Read Projections and Query Efficiency
 - Source list, Source Detail, candidate summary, and evidence export use purpose-specific read contracts.
 - Candidate and list views do not load exact transport bodies, full SDK snapshots, or other large evidence fields
  that they do not render.
 - Full evidence bodies remain available through explicit evidence inspection or export.
 - Job-filtered Source queries filter in SQL rather than loading every Source and filtering JobSource relationships in
  application memory.
 - Source navigation preserves deterministic `(page_number, UUID)` ordering without requiring all sibling Source
  records to be materialized.
 - Query optimization does not introduce lazy-loading behavior into detached UI models.
 ### 11. Lifecycle and Terminology Alignment
 - `transcribed` is the canonical successful terminal Job status for the current transcription workflow.
 - `completed` remains a readable historical/compatibility status but is not newly emitted by the V4.6 transcription
  worker.
 - `partial_success` and `failed` retain their existing aggregate meanings.
 - Architecture, schema, requirements, error-handling, UI, and model documentation use the same status terminology.
 - JobSource remains the mutable queue and compatibility projection.
 - ExecutionAttempt remains the authoritative append-only execution history.
 ### 12. Compatibility Adapter Isolation
 - Existing imports from `transcription.services.transcription` continue to resolve during V4.6.
 - Existing upload/storage aliases may remain temporarily at documented compatibility boundaries.
 - Compatibility adapters contain no new business logic, persistence, provider calls, or filesystem policy.
 - New application code imports the domain-owned implementation rather than adding more compatibility aliases.
 - Removal of compatibility imports is deferred until usage is measured and a separate deprecation decision is made.
 ## Out of Scope
 - Replacing JobSource with an event store or removing its compatibility fields.
 - Rewriting, deleting, or relabeling historical ExecutionAttempt evidence.
 - Changing Source candidate, promotion, or human-revision precedence.
 - Changing the V4.5 orientation transformation or quality-warning rules.
 - Reprocessing existing Sources solely to apply new media validation.
 - Supporting additional media formats.
 - Multiple selectable providers in the UI.
 - Distributed queues, external brokers, distributed locks, or multi-host worker coordination.
 - Concurrent processing of multiple aggregate Jobs in one application worker loop.
 - Automatic paid retry triggered by quality warnings.
 - Source page reordering or movement between Documents.
 - A generic repository framework, generic service container framework, or generic filesystem utility layer.
 - Destructive schema replacement or a V5 API redesign.
 ## Locked Design Decisions
 ### A. V4.6 Preserves the V4 Evidence Contract
 - Original Source bytes remain primary evidence.
 - ExecutionAttempt remains append-only and authoritative.
 - JobSource remains a compatibility projection.
 - Human revisions remain separate from all machine output.
 ### B. Requested and Resolved Model Identity Are Different Facts
 - Job stores the requested provider and model frozen before queueing.
 - ExecutionAttempt stores the provider and model observed for the individual execution.
 - Worker completion never rewrites the Job request specification.
 ### C. Concurrency Is Page-Bounded, Not Job-Distributed
 - V4.6 adds bounded page concurrency within one claimed Job.
 - One worker loop continues to own one Job aggregate at a time.
 - Every page task owns its own session and durable outcome transaction.
 ### D. The Source Facade Remains Stable
 - Source internals may be extracted, but callers retain a stable SourceService contract during migration.
 - Candidate promotion stays transactionally close to Source persistence.
 - Provider, prompt, artifact, and export concerns become replaceable collaborators rather than additional aggregates.
 ### E. Filesystem and Database Partial Failure Is Explicit
 - Cross-resource operations cannot rely on a relational transaction alone.
 - Observable commit failures receive compensation.
 - Unavoidable cleanup failure is reported and remains diagnosable rather than being represented as complete success.
 ### F. Refactoring Does Not Rewrite History
 - Existing nulls remain null unless the operator performs a separately specified maintenance action.
 - Historical SDK snapshots are not reclassified as transport evidence.
 - Existing preferred-machine projections and human revisions remain unchanged.
 ## Data and Compatibility Policy
 - V4.6 prefers no schema change where conditional writes and focused read projections are sufficient.
 - Any required schema addition is additive and portable across SQLite and PostgreSQL.
 - Existing Job, Source, JobSource, ExecutionAttempt, ProcessingArtifact, Document, and Person identifiers remain valid.
 - Existing Source files and artifact references retain their stored meaning.
 - Existing `completed` Job rows remain readable.
 - Existing API response fields are not removed by this revision.
 - Compatibility adapters remain read- and call-compatible while internal ownership moves.
 - Database, integration, and UI tests use confirmed isolated data and never modify `data/transcription.db`.
 - Potentially destructive tests run only through `tools/run_destructive_tests.py`.
 ## Acceptance Criteria
 1. Every newly queued Job has complete requested provider, model, prompt, prompt hash, and parameter provenance before
   worker execution.
 2. A provider-reported resolved model is stored on the ExecutionAttempt and does not change the Job's requested model.
 3. Normal and retranscription Job creation reject models outside the configured allowlist.
 4. Upload UI code opens no database session and performs no direct persistence.
 5. A failed ingestion transaction removes files staged by that failed operation and leaves no partial database rows.
 6. Supported Source content is validated beyond its extension without transforming original bytes.
 7. Two concurrent claimers cannot both claim or execute the same queued Job.
 8. Overlapping provider calls retain distinct request manifests and transport evidence.
 9. Page execution concurrency never exceeds the configured bound and uses no shared AsyncSession.
 10. Completed pages are durable while slower sibling calls remain in flight.
 11. Every actual provider call creates one immutable attempt and no duplicate attempt number.
 12. Concurrent first successes result in one preferred attempt and retained candidates for all other successes.
 13. Commit failure during external artifact creation does not silently leave an untracked artifact.
 14. External cleanup failure is visible and does not delete retained evidence metadata deceptively.
 15. Source list and candidate-summary views do not load exact transport bodies.
 16. Job-filtered Source lists filter in the database and preserve deterministic ordering.
 17. SourceService callers remain compatible while focused collaborators become independently testable.
 18. The worker emits `transcribed`, `partial_success`, or `failed` consistently, and historical `completed` rows remain
    readable.
 19. Previous Jobs, attempts, artifacts, Source projections, and human revisions remain unchanged by the upgrade.
 20. Verification uses isolated databases, fake providers, and does not modify curated Source or evidence files.
 ## Scope Freeze Gate
 V4.6 is sufficiently defined to begin implementation:
 - Submission-time versus execution-time provenance ownership is resolved.
 - UI, workflow, service, and transaction boundaries are resolved.
 - Claiming, attempt persistence, and first-selection concurrency rules are resolved.
 - Page-level concurrency scope and default bound are resolved.
 - Source extraction seams and compatibility policy are resolved.
 - Media validation, artifact compensation, and query-projection expectations are resolved.
 - Successful Job status terminology is resolved.
 - V4.2 and V4.5 evidence, candidate, and revision contracts remain unchanged.
 Any expansion into distributed work scheduling, additional providers or media types, evidence-model replacement,
 candidate-policy changes, or incompatible API/schema redesign requires an explicit V4.6 scope amendment or V5.
 ## Related Local References
 - [V4.6 Implementation Plan](implementation_plan_v4_6.md)
 - [V4.5 Scope Boundary](../ver4.5/scope_boundary_v4_5.md)
 - [V4.5 Implementation Plan](../ver4.5/implementation_plan_v4_5.md)
 - [V4 Architecture](../ver4/architecture_v4.md)
 - [V4 Schema](../ver4/schema_v4.md)
 - [V4 Requirements](../ver4/requirements_v4.md)
 - [V4 Error Handling Policy](../ver4/error_handling_v4.md)
 - [AI Evidence and Provenance Invariant](../invariant/ai_evidence_and_provenance.md)
 - [Error Handling Invariant](../invariant/error_handling.md)
@@ -29,3 +29,4 @@ Version 4 is the architecture baseline for the personal-scale application used t
 - [V4.3 Settings Scope](../ver4.3/scope_boundary_v4_3.md) and [Implementation Plan](../ver4.3/implementation_plan_v4_3.md)
 - [V4.4 Semantic Registries, Linked People, and Printing Scope](../ver4.4/scope_boundary_v4_4.md) and [Implementation Plan](../ver4.4/implementation_plan_v4_4.md)
 - [V4.5 Transcription Input Normalization and Quality Scope](../ver4.5/scope_boundary_v4_5.md) and [Implementation Plan](../ver4.5/implementation_plan_v4_5.md)
 - [V4.6 Architecture Conformance and Reliability Scope](../ver4.6/scope_boundary_v4_6.md) and [Implementation Plan](../ver4.6/implementation_plan_v4_6.md)