V4.1 revisions in preparation for v4.2. AI data capture now better defined.

docs/Raw API Response Q&A - part 2.md

@@ -0,0 +1,163 @@
+# Raw API Response Q&A - part 2
+In this exchange I sought a second opinion about what raw API response data I should be collecting in my transcription app. I uploaded part 1 to Gemini 3.6 Flash, then posted my first prompt.
+
+## My prompt  
+```I'm trying to decide what data I should capture from the AI doing transcriptions for my transcription app. I will have 2000-3000 documents at most (not millions!) These are family heirlooms I am transcribing for posterity, and I want to use the archival methods/practices that I can. Review the attached document and tell me what you think.```  
+
+## AI Response (Gemini 3.6 Flash)
+The assistant's breakdown of `ai_metadata` vs. `raw_api_response` is technically spot-on, but **Response #4 gives advice meant for massive, enterprise-scale software, not a focused archival project.**
+
+When building an application for 2,000–3,000 family heirloom documents, the trade-offs change completely:
+
+---
+
+### 1. The Scale Reality: Ignore the Storage Warnings
+
+In Response #4, the assistant cautions about storage costs, database bloat, and needing external cloud blob stores.
+
+* **The Math:** A rich, full JSON response from an LLM API averages **5 KB to 15 KB**.
+* **Your Total Volume:** $3\text{,}000 \text{ documents} \times 15\text{ KB} = \mathbf{\sim 45\text{ MB}}$ total.
+
+45 Megabytes is negligible. It can easily sit inside PostgreSQL JSONB columns forever without causing performance or cost issues. You do not need complex retention policies, blob storage buckets, or truncated responses.
+
+---
+
+### 2. Archival Science Perspective: Provenance Matters
+
+From a formal archival and documentary editing perspective, AI-assisted transcription is a form of document processing that requires **provenance** (origin and chain of custody).
+
+In 20 years, when AI models have shifted dramatically, having the record of *how* a draft was generated is invaluable. Storing complete AI responses ensures:
+
+* **Auditability:** You can trace why a specific transcription error, hallucination, or bracketed guess `[inaudible]` occurred.
+* **Reproducibility:** You capture the exact model string, parameters, and model outputs used at that point in history.
+* **Future Extraction:** If OpenRouter or model providers start embedding new provider-level metadata (such as reasoning traces or extended token details), you won’t regret having thrown it away.
+
+---
+
+### 3. What You Should Actually Capture
+
+To balance clean querying with archival completeness, stick with the two-field approach, but don't hold back on what you put in them.
+
+#### A. Operational Summary (`ai_metadata`)
+
+Use this for fast filtering, UI display, and analytics in your app.
+
+```json
+{
+  "provider": "openrouter",
+  "model_requested": "google/gemini-2.5-flash",
+  "model_used": "google/gemini-2.5-flash",
+  "finish_reason": "stop",
+  "tokens": {
+    "prompt_tokens": 1250,
+    "completion_tokens": 340,
+    "total_tokens": 1590
+  },
+  "latency_ms": 1420,
+  "cost_usd": 0.00018
+}
+
+```
+
+#### B. Complete Payload (`raw_api_response`)
+
+Store the **complete, unfiltered SDK response dictionary** returned by the API client. If the OpenRouter Python client hands back a response object, serialize its entire `.model_dump()` or `.dict()` straight into your `raw_api_response` JSONB column.
+
+#### C. Input Parameters (Don't Forget the Prompt!)
+
+To have a true archival record, you also need to store what you *sent* the AI, not just what it sent back. Either on the attempt record or in the metadata, record:
+
+* The exact system prompt version used.
+* The temperature / top_p settings.
+* The image filename / hash sent to the vision model.
+
+---
+
+### 4. The Reality on Bounding Boxes & Confidence Scores
+
+Regarding your second prompt about bounding boxes and confidence scores:
+
+1. **Bounding Boxes (Coordinates):** Standard Vision LLMs (like Gemini Flash, Claude 3.5 Sonnet, or GPT-4o) do **not** automatically return bounding boxes in API response metadata. OCR-specific tools (like Amazon Textract or Google Cloud Vision) return explicit spatial coordinates, but chat/vision LLMs only do so if you specifically instruct them in your prompt to format their output as JSON containing coordinates.
+2. **Confidence Scores:** Standard completion APIs do not return a clean "95% confidence" score. The closest equivalent is token `logprobs` (log probabilities), which can be requested via API parameters (`logprobs: true`), but token probability measures statistical model likelihood during token generation, not necessarily historical accuracy.
+
+---
+
+### Summary Recommendation
+
+Save the full, unfiltered API response object directly into `raw_api_response` without stripping anything out, alongside a clean `ai_metadata` summary. At 3,000 items, keeping complete raw evidence is effortless, aligns perfectly with documentary preservation standards, and protects your project's historical audit trail.
+
+---
+
+## My Prompt
+```  
+1. You mentioned the raw api response could "easily sit inside PostgreSQL JSONB columns". I'm currently running on SQLite during development and testing, but I had already anticipated moving to PostgresSQL before I went into production. Should I go ahead an move to PostgresSQL before implementing the capture of the raw api responses?
+
+2. Can you elaborate on "C. Input Parameters (Don't Forget the Prompt!)". 
+- I'm currently capturing the *name* of the prompt, but not the full text.
+- what are the temperature / top_p settings?
+- I'm already capturing the image_filename (It's being stored outside of the database in a folder in the python project. Can/should these images be stored in Postgres too?)  
+```
+
+## AI Response (Gemini 3.6 Flash)
+### 1. Should you move to PostgreSQL now or stay on SQLite?
+
+**Recommendation: Stay on SQLite for now. Do not delay your development.**
+
+Modern SQLite handles JSON natively via its standard JSON extension and functions. You do not need PostgreSQL’s dedicated `JSONB` binary data type just to capture and store complete API responses during development.
+
+#### Why Staying on SQLite First Makes Sense:
+
+* **Feature Parity for Raw Payloads:** Storing a raw API payload as a JSON-formatted string in a standard SQLite column behaves conceptually the same as putting it into PostgreSQL's `JSONB` column.
+* **No Database Drift:** If you use an ORM (like SQLAlchemy or Tortoise ORM), defining a generic `JSON` column type translates smoothly between both engines.
+* **Seamless Migration Later:** Since your total collection will max out at 2,000–3,000 documents, your entire dataset will easily fit in a small SQLite file. Migrating that entire dataset to PostgreSQL before going live will take minutes, not days.
+
+Keep building features and capturing response data on SQLite. Move to PostgreSQL only when you are ready to deploy your production environment.
+
+---
+
+### 2. Elaborating on Input Parameters
+
+To satisfy archival provenance standards, capturing *how* you prompted the AI is just as crucial as capturing *what* the AI returned.
+
+#### A. Full Text of the Prompt (Beyond the Name)
+
+Storing only the prompt's reference name (e.g., `v1_transcription_prompt`) leaves a critical gap: if you tweak that prompt later, you lose the exact instructions used for past runs.
+
+**What to capture:**
+
+* **System Prompt:** The exact system instructions provided to the model.
+* **User Prompt:** The exact dynamic prompt template merged with document-specific context.
+* **Prompt Version/Hash:** An explicit version identifier or SHA-256 hash of the full prompt string for quick grouping.
+
+#### B. Temperature and Top_P Settings
+
+These are the foundational hyper-parameters controlling the randomness and creativity of the AI model's output:
+
+| Parameter | What It Controls | Typical Value for OCR/Transcription | Why Record It? |
+| --- | --- | --- | --- |
+| **`temperature`** | Randomness of token selection. `0.0` is deterministic; higher values (e.g., `0.7`) introduce variability. | `0.0` to `0.2` | Confirms whether the run was deterministic (strict extraction) or creative (guessing missing words). |
+| **`top_p`** (Nucleus Sampling) | Cuts off low-probability tokens. `0.1` means only top 10% likely words are evaluated. | `1.0` or model default | Documents constraints on the vocabulary pool available to the model during execution. |
+
+Storing these alongside the prompt ensures you can accurately evaluate why two different runs on the same document yielded different results.
+
+#### C. Where Should the Input Images Be Stored?
+
+Keep storing the actual image files **in the filesystem**, not in PostgreSQL or SQLite binary `BLOB` columns.
+
+```
+/project_root
+├── /storage
+│   └── /images
+│       └── document_00101_raw.jpg
+└── /database
+    └── (stores path: "storage/images/document_00101_raw.jpg")
+
+```
+
+##### Why File-System Storage is Preferred:
+
+1. **Database Simplicity:** Databases perform best when handling structured metadata, indexes, and JSON queries. Storing heavy binary images in database tables inflates backup sizes and slows down general queries.
+2. **Web Server & UI Efficiency:** Serving image files directly from the disk path (or an internal file server) allows web clients to leverage standard HTTP byte-range requests, browser caching, and instant thumbnail generation without overloading database connections.
+3. **Archival Safety:** Image files stored cleanly in standard folder hierarchies (e.g., JPEG, TIFF, PNG) are portable, easier to back up incrementally, and accessible independently of any database engine.
+
+**Database Rule of Thumb:** Store the **file path**, **filename**, **file size**, and **cryptographic hash** (like SHA-256) inside PostgreSQL/SQLite. Store the **binary image bytes** on disk.