generated from john/python-template
164 lines
10 KiB
Markdown
164 lines
10 KiB
Markdown
# V4.2 Scope Boundary
|
|
|
|
This document defines the boundary for the digital-evidence and AI-provenance revision that follows V4.1 and precedes the planned V4.3 settings work. V4 remains the architecture baseline; V4.2 makes the existing evidence claims precise and adds a provider-neutral foundation for future processing artifacts.
|
|
|
|
## Purpose
|
|
|
|
- Align the application with the [Digital Evidence and AI Processing Provenance invariant](../invariant/ai_evidence_and_provenance.md).
|
|
- Preserve provider-boundary evidence before SDK parsing can remove unknown fields.
|
|
- Make successful and failed processing attempts inspectable without storing secrets.
|
|
- Support future OCR and layout outputs without coupling the database to one vendor.
|
|
- Establish a repeatable method for comparing transcription models against this archive.
|
|
|
|
## In Scope
|
|
|
|
### 1. Evidence Terminology and Existing-Data Compatibility
|
|
|
|
- Define transport response, router-normalized response, SDK response, normalized metadata, and derived artifact consistently in code, schema documentation, and UI labels.
|
|
- Treat existing `JobSource.raw_api_response` values as historical SDK response snapshots.
|
|
- Preserve every existing `Job`, `Source`, and `JobSource` row.
|
|
- Use additive migrations and compatibility reads; do not reinterpret previously stored values as exact transport captures.
|
|
- Correct the “Page-Level Execution and AI Outputs” rule in `docs/ver4/schema_v4.md` that currently describes `JOB_SOURCE` as storing a complete provider response envelope. The corrected rule must identify `raw_api_response` as an SDK-serialized OpenRouter response snapshot and state that it is neither the exact HTTP body nor the native upstream-provider response.
|
|
|
|
### 2. Secret-Safe Request Manifests
|
|
|
|
- Persist the effective request specification for each page execution without storing credentials or duplicate base64 media.
|
|
- Include requested provider/model, routing constraints, prompt content and hash, explicitly supplied parameters, source digest, media type, dimensions when known, and page identity.
|
|
- Distinguish an omitted optional parameter from an explicitly supplied null or value.
|
|
- Record application, provider-adapter, Python client, and relevant schema versions.
|
|
- Use source or derivative references in place of embedded media bytes.
|
|
|
|
### 3. Provider-Boundary Response Capture
|
|
|
|
- Capture the exact HTTP response body before OpenRouter SDK parsing for non-streaming transcription calls.
|
|
- Store HTTP status and an explicit allowlist of safe response headers.
|
|
- Store router request/generation identifiers and resolved model/provider-routing metadata when exposed.
|
|
- Preserve the current parsed SDK snapshot and normalized metadata where useful.
|
|
- Keep exact body, parsed representation, and normalized fields distinguishable.
|
|
|
|
### 4. Failure Evidence and Timing
|
|
|
|
- Create or update a page execution record for every attempted provider call.
|
|
- Persist safe response evidence for non-success HTTP responses.
|
|
- Distinguish HTTP response failures, connection failures, local timeouts, response-validation failures, and transcription-quality failures.
|
|
- Store execution start/end times or duration using a clearly defined clock policy.
|
|
- Do not collapse a provider error body into only a generic user-facing message.
|
|
|
|
### 5. Generic Processing Artifacts
|
|
|
|
- Add a provider-neutral representation for versioned derived artifacts.
|
|
- Support inline JSON and externally stored payloads with a digest and stable reference.
|
|
- Record artifact type, format, schema/version, producer/version, source, producing execution, and creation time.
|
|
- Define coordinate-system metadata sufficient for word, line, block, or page geometry.
|
|
- Permit future OCR/layout/confidence results without implementing a vendor-specific table for each provider.
|
|
|
|
### 6. Evidence Inspection and Export
|
|
|
|
- Expand Source Detail and/or Job Detail to identify the evidence layer being displayed.
|
|
- Provide readable JSON inspection for request manifests, transport metadata, parsed responses, normalized metadata, and derived artifacts.
|
|
- Provide a safe export containing evidence content or references, relationships, schema versions, and digests.
|
|
- Clearly label evidence that was not captured for historical records.
|
|
- Do not display or export credentials, unrestricted headers, or embedded base64 source media.
|
|
|
|
### 7. Representative-Corpus Benchmark Protocol
|
|
|
|
- Define a private benchmark manifest referencing source digests rather than duplicating archival media.
|
|
- Include representative printed, typed, handwritten, degraded, tabular, and spatially complex pages.
|
|
- Pair each benchmark item with a manually reviewed literal transcription.
|
|
- Score character error rate, word error rate, omissions, inventions, silent normalization, uncertainty handling, layout fidelity, cost, and latency.
|
|
- Preserve the complete execution provenance for every benchmark run.
|
|
- Keep the current model as a baseline; do not change the application default solely from vendor benchmarks.
|
|
|
|
### 8. Migration, Integrity, and Verification
|
|
|
|
- Provide non-destructive upgrade behavior for supported SQLite and PostgreSQL deployments.
|
|
- Backfill only facts that can be derived reliably from existing records.
|
|
- Mark unavailable historical evidence as unavailable rather than fabricating it.
|
|
- Add digest, serialization, header-allowlist, failure-path, compatibility, artifact, export, and UI inspection tests.
|
|
- Run destructive tests only through the repository's required backup-and-restore wrapper.
|
|
|
|
## Out of Scope
|
|
|
|
- Selecting or declaring a permanent best transcription model.
|
|
- Changing the default transcription model without benchmark evidence and a separate decision.
|
|
- Integrating Azure Document Intelligence, Google Document AI, Transkribus, Mistral OCR, or another OCR provider in V4.2.
|
|
- Generating bounding boxes retroactively for existing transcriptions.
|
|
- Bulk reprocessing the archive.
|
|
- Packet capture, TLS evidence, full unrestricted request/response headers, or credential retention.
|
|
- Storing duplicate base64 source images in request manifests.
|
|
- Guaranteeing byte-identical reproduction from nondeterministic or updated hosted models.
|
|
- Automatic entity extraction, biography generation, or genealogical inference.
|
|
- Replacing the relational database with an event store or content-addressed object store.
|
|
- Destructive renaming or removal of `raw_api_response`.
|
|
|
|
## Locked Design Decisions
|
|
|
|
### A. The Original Source Is Primary Evidence
|
|
|
|
- Original uploaded bytes and their digest remain authoritative.
|
|
- Processing derivatives and outputs are independently identified derived evidence.
|
|
- Future OCR/layout work reuses the original or a documented derivative.
|
|
|
|
### B. Evidence Is Layered
|
|
|
|
- Exact transport evidence, SDK-parsed objects, normalized metadata, and transcription text serve different purposes.
|
|
- One representation must not silently stand in for another.
|
|
- UI and export labels name the stored evidence layer.
|
|
|
|
### C. History Is Append-Only
|
|
|
|
- A retry or reprocessing attempt creates new execution evidence.
|
|
- Convenience caches may change, but historical execution output does not.
|
|
- Human revisions remain separate from machine output.
|
|
|
|
### D. Capture Is Secret-Safe by Construction
|
|
|
|
- Safe headers are allowlisted.
|
|
- Authorization, cookies, API keys, and unrestricted headers are never persisted.
|
|
- Request manifests reference source digests instead of embedding source bytes.
|
|
|
|
### E. Derived Artifacts Are Generic and Versioned
|
|
|
|
- Artifact storage is not limited to bounding boxes.
|
|
- Coordinate metadata declares units, origin, dimensions, and transformations.
|
|
- Provider-specific payloads may be retained without making provider-specific fields the durable application contract.
|
|
|
|
### F. Existing Evidence Keeps Its Original Meaning
|
|
|
|
- Existing `raw_api_response` data remains an SDK response snapshot.
|
|
- A migration may label or classify it but may not claim that missing transport data was captured.
|
|
- Historical nulls and absent fields remain distinguishable from new explicitly captured values.
|
|
|
|
## Data and Compatibility Policy
|
|
|
|
- All schema changes are additive in V4.2.
|
|
- Existing source files, hashes, transcriptions, revisions, prompts, jobs, and relationships remain valid.
|
|
- Compatibility reads continue to display historical SDK snapshots.
|
|
- Large derived artifacts may be stored outside the database when the database retains a stable reference, digest, media type, and schema identity.
|
|
- JSON evidence must remain portable across SQLite and PostgreSQL.
|
|
- Exports use explicit schema versions so later releases can interpret older packages.
|
|
|
|
## Acceptance Criteria
|
|
|
|
1. A new execution can be traced from its source digest through its frozen request manifest, transport response, parsed/normalized data, and derived outputs.
|
|
2. Exact response content is captured before SDK parsing and is clearly distinguished from the existing SDK snapshot.
|
|
3. Failed HTTP calls retain safe provider evidence; calls with no response record that fact explicitly.
|
|
4. Omitted parameters remain distinguishable from explicit values.
|
|
5. No persisted request, header set, UI display, log, or export contains API credentials.
|
|
6. Retrying or reprocessing does not overwrite prior execution evidence.
|
|
7. Historical records remain readable and are not mislabeled as exact transport captures.
|
|
8. A versioned generic artifact can represent OCR/layout JSON and its coordinate system without a provider-specific schema change.
|
|
9. Evidence exports include relationships, schema identities, and digests sufficient for independent integrity checks.
|
|
10. The benchmark protocol can compare the current baseline with another model on the same private corpus and scoring rules.
|
|
11. Additive migrations and focused tests work across the supported persistence model.
|
|
12. All destructive-test runs comply with the backup-and-restore protocol.
|
|
|
|
## Related Local References
|
|
|
|
- [V4.2 Implementation Plan](implementation_plan_v4_2.md)
|
|
- [Digital Evidence and AI Processing Provenance](../invariant/ai_evidence_and_provenance.md)
|
|
- [V4 Architecture](../ver4/architecture_v4.md)
|
|
- [V4 Schema](../ver4/schema_v4.md)
|
|
- [V4 Requirements](../ver4/requirements_v4.md)
|
|
- [Transcription Methodology](../invariant/transcription_methodology.md)
|