V4.2 Updated what ai_raw_response data is being captured. The changes were more extensive than I expected.

docs/ver4/architecture_v4.md

@@ -26,7 +26,10 @@ This document describes the production architecture of the document transcriptio
 - Isolate page failures so multi-page jobs can complete with partial success.
 - Operate across supported platforms through Python-based application and maintenance tooling.
 
-V4.2 extends this baseline with exact OpenRouter transport evidence and provider-neutral derived-artifact provenance. See the [V4.2 Scope Boundary](../ver4.2/scope_boundary_v4_2.md).
+V4.2 extends this baseline with immutable execution attempts, exact OpenRouter transport evidence, safe
+versioned exports, and provider-neutral derived-artifact provenance. `JobSource` remains the mutable queue and
+compatibility projection; `ExecutionAttempt` is the authoritative append-only processing history. See the
+[V4.2 Scope Boundary](../ver4.2/scope_boundary_v4_2.md).
 
 ## Technical Stack
 
@@ -135,8 +138,11 @@ Responsibilities:
 1. User uploads one or more images for a `Document`.
 2. System stores files, hashes them, creates ordered `Source` rows, and creates a `Job`.
 3. Worker claims the job, marks it `processing`, and executes page calls concurrently.
-4. Each page writes a `JobSource` result with machine output, normalized metadata, and an SDK-serialized OpenRouter response snapshot.
-5. Aggregate status becomes `completed`, `partial_success`, or `failed`.
+4. Each provider call appends an `ExecutionAttempt` with its request manifest, transport evidence, SDK snapshot,
+   normalized metadata, timing, and outcome.
+5. The linked `JobSource` is updated as a compatibility projection, and a successful attempt updates the
+   `Source.raw_transcription` latest-success projection.
+6. Aggregate status becomes `completed`, `partial_success`, or `failed`.
 
 ### 2. Document-Person Relationship Management
 
@@ -160,6 +166,9 @@ Responsibilities:
 - Human corrections occur only in `Source.revised_text`.
 - Prompt and parameter provenance is frozen on `Job` at submission time.
 - The SDK-serialized OpenRouter response snapshot is stored on `JobSource` for each successful page execution.
+- Every V4.2 provider call appends a distinct `ExecutionAttempt`; retries never rewrite earlier attempts.
+- Exact response bytes identify the OpenRouter HTTP boundary and are not labeled as native upstream-provider JSON.
+- Generic `ProcessingArtifact` records use versioned schemas, digests, and one inline or external content location.
 - `DocumentPerson` links are unique for `(document_id, person_id, role_id)`.
 - Relationship mutations are deterministic and set-based.
 - `DocumentType.code` is stable; `DocumentType.label` may evolve.

docs/ver4/requirements_v4.md

@@ -9,7 +9,7 @@ This document defines the baseline requirements for the document transcription s
 | REQ-0 | System | Provide end-to-end multi-page document transcription with persistent, inspectable async job states. | demonstration |
 | REQ-1 | Functional | Allow users to upload one or more images as ordered `Source` pages under a `Document`. | test |
 | REQ-2 | Functional | Process page transcription asynchronously using an `asyncio` worker pool bounded by rate limits. | test |
-| REQ-3 | Functional | Persist submission-time prompt configuration and full page-level provider response evidence for every job execution. | test |
+| REQ-3 | Functional | Persist submission-time request provenance and accurately labeled page-level SDK evidence; V4.2 adds exact OpenRouter-boundary transport evidence for new attempts. | test |
 | REQ-4 | Functional | Support job states `queued`, `processing`, `completed`, `partial_success`, and `failed`, plus page states `pending`, `transcribed`, and `failed`. | inspection |
 | REQ-5 | Functional | Allow users to manage historical `Person` records and link multiple people per role to a `Document`. | test |
 | REQ-6 | Functional | Support an extensible role taxonomy for document-person relationships. | inspection |

docs/ver4/schema_v4.md

@@ -109,6 +109,46 @@ TEXT error_detail
 TIMESTAMPTZ executed_at
 }
 
+EXECUTION_ATTEMPT {
+UUID id PK
+UUID job_source_id FK
+UUID job_id FK
+UUID source_id FK
+INTEGER attempt_number
+VARCHAR status
+JSONB request_manifest
+TEXT request_manifest_sha256
+INTEGER transport_status_code
+BINARY transport_body
+JSONB transport_safe_headers
+JSONB sdk_response_snapshot
+JSONB normalized_metadata
+JSONB software_context
+TEXT raw_transcription
+TEXT failure_phase
+TIMESTAMPTZ started_at
+TIMESTAMPTZ finished_at
+INTEGER duration_ms
+}
+
+PROCESSING_ARTIFACT {
+UUID id PK
+UUID source_id FK
+UUID execution_attempt_id FK
+TEXT artifact_type
+TEXT media_type
+TEXT schema_name
+TEXT schema_version
+TEXT producer
+TEXT producer_version
+JSONB inline_payload
+TEXT external_reference
+TEXT payload_sha256
+BIGINT byte_size
+JSONB coordinate_metadata
+TIMESTAMPTZ created_at
+}
+
 DOCUMENT_TYPE ||--o{ DOCUMENT : classifies
 DOCUMENT ||--o{ DOCUMENT_PERSON : has_people
 PERSON ||--o{ DOCUMENT_PERSON : appears_in
@@ -117,6 +157,9 @@ DOCUMENT ||--o{ JOB : has_jobs
 DOCUMENT ||--o{ SOURCE : contains_pages
 JOB ||--o{ JOB_SOURCE : executes
 SOURCE ||--o{ JOB_SOURCE : processed_in
+JOB_SOURCE ||--o{ EXECUTION_ATTEMPT : projects
+SOURCE ||--o{ PROCESSING_ARTIFACT : derives
+EXECUTION_ATTEMPT ||--o{ PROCESSING_ARTIFACT : produces
 ```
 
 ## Domain Invariants and Provenance Rules
@@ -125,9 +168,23 @@ SOURCE ||--o{ JOB_SOURCE : processed_in
 
 - Every single page execution by an AI model produces a dedicated `JOB_SOURCE` record.
 - Every `JOB` stores the frozen prompt identifier, prompt text, and hyperparameters used at submission time.
-- Every `JOB_SOURCE` stores the complete provider response envelope and page-level operational metadata.
+- `JOB_SOURCE.raw_api_response` is a compatibility projection containing an SDK-serialized OpenRouter response
+  snapshot. It is neither the exact HTTP body nor the native upstream-provider response.
+- Every new provider call creates an immutable `EXECUTION_ATTEMPT` containing the frozen request manifest,
+  exact OpenRouter-boundary response bytes when received, safe transport metadata, SDK snapshot, normalized
+  metadata, timing, and outcome.
+- `EXECUTION_ATTEMPT(job_id, source_id, attempt_number)` is unique; retries increment the persisted attempt number.
+- Historical `JOB_SOURCE` rows without an `EXECUTION_ATTEMPT` remain SDK snapshots and are explicitly labeled as
+  lacking transport evidence.
 - `SOURCE.raw_transcription` caches the latest successful machine output for that page.
 
+### Generic Processing Artifacts
+
+- `PROCESSING_ARTIFACT` stores provider-neutral versioned derived outputs.
+- Exactly one of `inline_payload` and `external_reference` is populated.
+- Externally stored artifacts use application-managed relative references and are verified by SHA-256 and byte size.
+- Coordinate metadata declares units, origin, dimensions, and transformations when geometry is present.
+
 ### Image Storage and Integrity
 
 - Binary images are stored on disk; `SOURCE.file_path` stores the persisted path.