transcription/docs/ver4.5/scope_boundary_v4_5.md

# V4.5 Scope Boundary

This document defines the frozen boundary for transcription input normalization and selective transcription-quality improvement after the completed V4.4 revision. V4 through V4.4 remain the architecture and behavioral baseline except where this document explicitly changes Source processing, transcription selection, or Source Detail behavior.

## Purpose

- Ensure model inputs are physically upright when curated image files rely on orientation metadata.
- Distinguish typewritten, typeset, handwritten, and mixed document bodies consistently.
- Let the user selectively retranscribe an unsatisfactory Source with an approved alternate vision model.
- Preserve every machine result while allowing the user to choose which result is the preferred machine transcription.

## In Scope

### 1. Metadata-Driven Orientation Normalization

- The original uploaded Source remains immutable archival evidence.
- Before a supported raster image is sent to a transcription provider, the application reads recognized orientation metadata.
- When the metadata requires rotation, the application creates a physically upright model-input derivative and resets or removes the derivative's orientation metadata.
- The provider receives the normalized derivative rather than upside-down stored pixels.
- When no supported orientation transformation is required, the original Source may remain the provider input.
- The derivative records:
  - Source UUID.
  - Original and derivative SHA-256 digests and byte sizes.
  - Original and derivative dimensions and media types.
  - Applied orientation transformation.
  - Transformation implementation and version.
  - Creation timestamp.
- The derivative uses the existing processing-artifact and evidence architecture rather than replacing the Source file.
- PDF orientation, visual orientation inference, manual rotation controls, deskewing, cropping, contrast changes, and general image enhancement are not part of V4.5.

### 2. Transcription Medium Contract

- The durable document-medium rules are defined in the cross-version [Transcription Methodology](../invariant/transcription_methodology.md).
- The transcription prompt distinguishes these document-body media:
  - `[document body handwritten]`
  - `[document body typewritten]`
  - `[document body typeset]`
  - `[document body mixed]`
- A typewriter's uneven impressions, monospaced characters, or mechanical defects do not by themselves indicate handwriting.
- The transcript contains exactly one applicable document-body marker.
- A wholly handwritten body uses the one body marker rather than wrapping every line in `[handwritten: ...]`.
- Typewritten and typeset bodies do not use handwriting wrappers unless a genuinely handwritten annotation or signature appears.
- A mixed body may use localized handwriting markers only for the handwritten portions.
- Stored transcription output remains plain text. Model-generated HTML entities are not required for ordinary characters.
- The prompt includes layout guidance for tables of contents, tables, forms, columns, captions, marginalia, page numbers, dotted leaders, and associated page references.
- Line-break hyphenation rules continue to preserve intentional hyphens while rejoining words split only by line wrapping.

### 3. Quality Warnings

- The application evaluates successful machine output for deterministic warning conditions, including:
  - Unicode replacement characters such as `�`.
  - A whole-body handwritten marker combined with repeated whole-line handwriting wrappers.
  - More than one document-body medium marker.
  - Unresolved HTML entities in otherwise plain transcription text.
- Warnings do not silently rewrite model output.
- Warnings do not automatically trigger another paid provider request.
- Source Detail displays warnings with the relevant machine result so the user can decide whether to revise or retranscribe it.
- V4.5 does not assign or display a transcription-confidence percentage. Provider self-assessments and token probabilities are not treated as calibrated transcription confidence.

### 4. Source Retranscription Entry Point

- Source Detail adds a **Retranscribe Source** action.
- The action opens Create Processing Job with the Source preselected and locked.
- The Source's existing Document is derived from its relationship and cannot be changed in this flow.
- The new Job contains a `JobSource` only for the selected Source; it does not retranscribe every Source in the Document.
- Provider is populated from the configured `PROVIDER` value and is read-only while only one provider is configured.
- Model is selected from an operator-configured allowlist.
- The configured default model is initially selected.
- Creating the Job freezes the selected provider, model, prompt, prompt hash, parameters, and Source evidence according to the existing provenance contract.
- Retranscription creates a new Job and new execution evidence. It does not reuse, mutate, or erase a previous Job.

### 5. Configured Vision-Model Allowlist

- `PROVIDER_MODEL` remains the default transcription model.
- `PROVIDER_MODELS` defines the models available in the Create Processing Job model selector.
- The environment representation is a JSON array, for example:

```dotenv
PROVIDER=openrouter
PROVIDER_MODEL=google/gemini-2.5-flash
PROVIDER_MODELS=["google/gemini-2.5-flash","google/gemini-2.5-pro","anthropic/claude-sonnet-4"]
```

- If `PROVIDER_MODELS` is omitted, the selector contains only `PROVIDER_MODEL`.
- The default model appears exactly once and first; remaining configured models retain their relative order.
- Empty, malformed, or duplicate values produce deterministic configuration validation.
- The UI never accepts an arbitrary model identifier outside the configured allowlist.
- The allowlist controls availability, not claims of quality, price, or provider compatibility. The operator is responsible for configuring models supported by the selected provider.

### 6. Candidate Machine Transcriptions

- The first successful transcription becomes preferred automatically whenever the Source has no preferred machine output, regardless of whether it came from an initial or retranscription Job.
- Once a Source has a preferred machine transcription, every later successful result is stored as a candidate regardless of Job purpose and cannot replace the preferred result automatically.
- Every candidate remains associated with its immutable Job, JobSource, ExecutionAttempt, provider, model, prompt, parameters, timestamps, warnings, and normalized-input evidence.
- Source Detail presents:
  - The current preferred machine transcription.
  - Available successful candidates with date, provider, model, Job ID, and warning state.
  - A comparison between the current preferred machine transcription and one selected candidate.
  - A **Use this transcription** action for a successful candidate.
- Before any successful result exists, Source Detail displays an explicit no-machine-transcription state.
- When a preferred result exists but no candidates exist, Source Detail omits comparison controls and displays an explicit no-candidates state.
- Promoting a candidate:
  - Verifies that the successful execution belongs to the Source.
  - Records the selected execution as the preferred machine-output provenance.
  - Updates `Source.raw_transcription` as the preferred-machine-text projection.
  - Does not modify `Source.revised_text`.
  - Does not delete or alter any previous machine result.
- If a human revision exists, it remains the current human-preferred text used by normal display and printing after a machine candidate is promoted.

### 7. Evidence and Failure Behavior

- The original Source and every normalized derivative are content-addressed and traceable.
- Every provider request identifies the exact original Source and model-input artifact used.
- Every execution attempt records the exact immutable model-input artifact it consumed, including when normalized bytes are reused.
- Every retranscription attempt follows the V4.2 immutable execution-attempt contract.
- A normalization failure prevents the provider request and produces an explicit actionable error.
- A provider or persistence failure leaves the current preferred machine transcription and human revision unchanged.
- Candidate promotion is atomic: provenance selection and the preferred-machine-text projection either both commit or both remain unchanged.
- Provider network work occurs outside database transactions.

### 8. Source Detail Terminology

- **Original Source** means the immutable uploaded file.
- **Model input** means the original Source or normalized derivative actually sent to the provider.
- **Machine attempt** means one immutable provider execution.
- **Candidate transcription** means a successful machine result not currently selected as preferred.
- **Preferred machine transcription** means the selected machine result projected through `Source.raw_transcription`.
- **Human revision** means `Source.revised_text`, which remains independent of every machine result.
- These distinctions use concise labels and progressive disclosure; routine satisfactory Sources do not display a mandatory side-by-side original/normalized image comparison.
- When normalization occurred, Source Detail displays an orientation-normalized indicator and makes transformation evidence inspectable through the existing evidence UI.

## Out of Scope

- Manual image rotation or image-editing controls.
- Visual orientation detection when metadata is absent or incorrect.
- Deskewing, cropping, contrast normalization, denoising, sharpening, or restoration.
- Replacing or modifying original Source files.
- Automatically retranscribing every Source.
- Automatic provider retries triggered by quality warnings.
- Arbitrary model identifiers entered by users.
- Multiple provider selection in the UI.
- Model benchmarking, pricing recommendations, or automatic model ranking.
- A provider-independent transcription-confidence percentage.
- Silent cleanup or rewriting of model output.
- Deleting unsuccessful, superseded, or unselected machine attempts.
- Promoting a machine candidate over a human revision.
- Source page renumbering or reordering.

## Locked Design Decisions

### A. Curated Originals Remain Authoritative

- V4.5 corrects metadata-directed orientation only for model processing.
- The archival upload is never replaced by the normalized derivative.

### B. Orientation Is Automatic and Metadata-Driven

- No manual orientation workflow is introduced.
- V4.5 does not guess orientation from page content.

### C. Selective Retranscription Replaces Automatic Escalation

- The normal default model remains efficient for satisfactory Sources.
- The user explicitly chooses when an alternate approved model is worth another provider request.
- Quality warnings inform that choice but never incur cost automatically.

### D. Retranscription Produces Candidates

- Alternate results remain immutable and comparable.
- The user explicitly promotes the preferred machine result.
- Human revision remains a separate, higher-precedence layer.

### E. Model Choice Is Operator-Controlled

- Environment configuration defines the finite allowed model set.
- Job records freeze the actual selected model and request parameters.

### F. Confidence Is Evidence-Based, Not Invented

- V4.5 does not present model self-rating as objective confidence.
- Review uses visible output, deterministic warnings, provenance, and human judgment.

## Acceptance Criteria

1. A JPEG with EXIF Orientation 3 produces an upright model-input derivative while the original bytes remain unchanged.
2. A Source requiring no recognized orientation transformation is not unnecessarily altered.
3. Orientation transformation metadata and hashes identify the exact provider input.
4. The prompt distinguishes handwritten, typewritten, typeset, and mixed bodies with exactly one body marker.
5. Typewritten text is not wrapped line by line as handwriting.
6. Tables of contents retain row associations and page references without handwriting wrappers.
7. Deterministic warnings identify replacement characters and contradictory body markers without changing output.
8. Source Detail provides Retranscribe Source for an existing Source.
9. Create Processing Job locks the Source and Document, uses configured Provider, and restricts Model to the configured allowlist.
10. Retranscription creates a new single-Source Job with complete frozen request and execution evidence.
11. The first successful result is selected automatically; every later successful result remains a candidate and cannot replace the preferred machine transcription automatically.
12. Source Detail can compare the preferred machine transcription with one candidate and promote that candidate explicitly.
13. Candidate promotion records exact successful-execution provenance and updates the machine-text projection atomically.
14. Candidate promotion never changes or clears a human revision.
15. Earlier machine attempts remain inspectable after retranscription and promotion.
16. No confidence percentage, manual image editor, automatic quality retry, or arbitrary model input is introduced.
17. Database, integration, and UI verification uses isolated test data and never modifies `data/transcription.db`.

## Scope Freeze Gate

V4.5 is sufficiently frozen to begin implementation:

- Orientation behavior and preservation rules are resolved.
- Prompt medium categories and marker behavior are resolved.
- Warning behavior and the absence of automatic retry are resolved.
- Retranscription entry point, single-Source scope, and model configuration are resolved.
- Candidate preservation, comparison, promotion, and human-revision precedence are resolved.
- The broader future-feature list has been reviewed, and no additional V4.5 features are required.

Any expansion into manual image editing, visual orientation inference, automatic retries, multiple providers, confidence scoring, or additional processing features requires an explicit V4.5 scope amendment or a later revision.

## Related Local References

- [V4.5 Implementation Plan](implementation_plan_v4_5.md)
- [V4.4 Scope Boundary](../ver4.4/scope_boundary_v4_4.md)
- [V4.4 Implementation Plan](../ver4.4/implementation_plan_v4_4.md)
- [V4.2 Evidence and Provenance Scope](../ver4.2/scope_boundary_v4_2.md)
- [V4 Architecture](../ver4/architecture_v4.md)
- [V4 Schema](../ver4/schema_v4.md)
- [V4 Requirements](../ver4/requirements_v4.md)
- [Transcription Methodology](../invariant/transcription_methodology.md)
- [AI Evidence and Provenance Invariant](../invariant/ai_evidence_and_provenance.md)