V3 Updated V3 core documents. Added data folder backup/restore before/after running destructive tests.

This commit is contained in:
Jim Lancaster
2026-08-09 10:51:30 -05:00
parent b59d3da23e
commit e6549277c6
14 changed files with 417 additions and 271 deletions
+8 -8
View File
@@ -1,6 +1,6 @@
# Database Schema (Version 3)
This document describes the relational schema for the transcription platform. It incorporates multi-image batch orchestration, page-level execution tracking, many-to-many author/recipient attribution, input prompt provenance capture, and raw API payload evidence for archival auditing.
This document describes the relational schema for the transcription platform. It incorporates multi-image batch orchestration, page-level execution tracking, many-to-many author/recipient attribution, submission-time prompt snapshot capture, and raw API payload evidence for archival auditing.
The schema uses generic JSON columns compatible with SQLite in local development and PostgreSQL native JSONB/UUID types in production.
@@ -54,6 +54,12 @@ JOB {
INTEGER retry_count
TEXT provider
TEXT model
TEXT prompt_name
TEXT prompt_hash
TEXT system_prompt
TEXT user_prompt
FLOAT temperature
FLOAT top_p
TIMESTAMPTZ date_created
TIMESTAMPTZ date_updated
}
@@ -79,12 +85,6 @@ JOB_SOURCE {
UUID source_id FK
VARCHAR status "pending | transcribed | failed"
TEXT raw_transcription
TEXT prompt_name
TEXT prompt_hash
TEXT system_prompt
TEXT user_prompt
FLOAT temperature
FLOAT top_p
JSONB ai_metadata
JSONB raw_api_response
TEXT error_detail
@@ -104,7 +104,7 @@ SOURCE ||--o{ JOB_SOURCE : "processed_in"
### Page-Level Execution & AI Outputs
* **Execution Granularity:** Every single image execution attempt by an AI model produces a dedicated record in `job_source`.
* **Page-Level Input Provenance:** Every `job_source` execution captures its exact hyperparameters (`temperature`, `top_p`), prompt identifier details (`prompt_name`, `prompt_hash`), and full prompt text strings (`system_prompt`, `user_prompt`) used for that specific page call.
* **Submission Snapshot Provenance:** Every `job` captures the frozen prompt identifier details (`prompt_name`, `prompt_hash`), full prompt text strings (`system_prompt`, `user_prompt`), and hyperparameters (`temperature`, `top_p`) at submission time.
* **Point-in-Time Output Auditability:** `job_source.raw_api_response` stores the complete, unedited provider REST response envelope for that specific image page call. `job_source.ai_metadata` stores spatial bounding boxes, normalized token usage, latency, and cost details for fast querying.
* **Active Output Caching:** Upon successful completion of an image call, `source.raw_transcription` is updated with the latest output string from `job_source.raw_transcription` for fast UI rendering.