V4.2 Updated what ai_raw_response data is being captured. The changes were more extensive than I expected.

docs/ver4/schema_v4.md

@@ -109,6 +109,46 @@ TEXT error_detail
 TIMESTAMPTZ executed_at
 }
 
+EXECUTION_ATTEMPT {
+UUID id PK
+UUID job_source_id FK
+UUID job_id FK
+UUID source_id FK
+INTEGER attempt_number
+VARCHAR status
+JSONB request_manifest
+TEXT request_manifest_sha256
+INTEGER transport_status_code
+BINARY transport_body
+JSONB transport_safe_headers
+JSONB sdk_response_snapshot
+JSONB normalized_metadata
+JSONB software_context
+TEXT raw_transcription
+TEXT failure_phase
+TIMESTAMPTZ started_at
+TIMESTAMPTZ finished_at
+INTEGER duration_ms
+}
+
+PROCESSING_ARTIFACT {
+UUID id PK
+UUID source_id FK
+UUID execution_attempt_id FK
+TEXT artifact_type
+TEXT media_type
+TEXT schema_name
+TEXT schema_version
+TEXT producer
+TEXT producer_version
+JSONB inline_payload
+TEXT external_reference
+TEXT payload_sha256
+BIGINT byte_size
+JSONB coordinate_metadata
+TIMESTAMPTZ created_at
+}
+
 DOCUMENT_TYPE ||--o{ DOCUMENT : classifies
 DOCUMENT ||--o{ DOCUMENT_PERSON : has_people
 PERSON ||--o{ DOCUMENT_PERSON : appears_in
@@ -117,6 +157,9 @@ DOCUMENT ||--o{ JOB : has_jobs
 DOCUMENT ||--o{ SOURCE : contains_pages
 JOB ||--o{ JOB_SOURCE : executes
 SOURCE ||--o{ JOB_SOURCE : processed_in
+JOB_SOURCE ||--o{ EXECUTION_ATTEMPT : projects
+SOURCE ||--o{ PROCESSING_ARTIFACT : derives
+EXECUTION_ATTEMPT ||--o{ PROCESSING_ARTIFACT : produces
 ```
 
 ## Domain Invariants and Provenance Rules
@@ -125,9 +168,23 @@ SOURCE ||--o{ JOB_SOURCE : processed_in
 
 - Every single page execution by an AI model produces a dedicated `JOB_SOURCE` record.
 - Every `JOB` stores the frozen prompt identifier, prompt text, and hyperparameters used at submission time.
-- Every `JOB_SOURCE` stores the complete provider response envelope and page-level operational metadata.
+- `JOB_SOURCE.raw_api_response` is a compatibility projection containing an SDK-serialized OpenRouter response
+  snapshot. It is neither the exact HTTP body nor the native upstream-provider response.
+- Every new provider call creates an immutable `EXECUTION_ATTEMPT` containing the frozen request manifest,
+  exact OpenRouter-boundary response bytes when received, safe transport metadata, SDK snapshot, normalized
+  metadata, timing, and outcome.
+- `EXECUTION_ATTEMPT(job_id, source_id, attempt_number)` is unique; retries increment the persisted attempt number.
+- Historical `JOB_SOURCE` rows without an `EXECUTION_ATTEMPT` remain SDK snapshots and are explicitly labeled as
+  lacking transport evidence.
 - `SOURCE.raw_transcription` caches the latest successful machine output for that page.
 
+### Generic Processing Artifacts
+
+- `PROCESSING_ARTIFACT` stores provider-neutral versioned derived outputs.
+- Exactly one of `inline_payload` and `external_reference` is populated.
+- Externally stored artifacts use application-managed relative references and are verified by SHA-256 and byte size.
+- Coordinate metadata declares units, origin, dimensions, and transformations when geometry is present.
+
 ### Image Storage and Integrity
 
 - Binary images are stored on disk; `SOURCE.file_path` stores the persisted path.