bbchops/transcription
1
0
Fork 0
generated from john/python-template
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

V4.1 major revision to docs. Removed all obsolete documents, updated v4.2 implementation scope and plan.

Browse Source
This commit is contained in:
Jim Lancaster
2026-08-13 15:32:40 -05:00
parent 171132919d
commit 28811d79ce
60 changed files with 1170 additions and 7471 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ver4/architecture_v4.md
+34 -6
View File
@@ -4,15 +4,40 @@ This document describes the production architecture of the document transcriptio
## Architecture Objectives
- Preserve original source material and immutable machine transcription output.
- Preserve original source material, per-execution machine output, and separate human revision.
- Support batching one or more images into ordered multi-page documents.
- Capture complete submission-time prompt provenance and per-page provider response evidence.
- Capture submission-time prompt provenance and a per-page OpenRouter SDK response snapshot.
- Execute page transcription concurrently with bounded `asyncio` workers.
- Maintain relational portability across SQLite and PostgreSQL.
- Keep operator workflows cross-platform and Python-driven.
- Support many-to-many document-person relationships with extensible roles.
- Support registry-driven document type classification.
## Core Capabilities
- Ingest one or more images into sequential `Source` pages under a `Document`.
- Execute asynchronous vision transcription with bounded worker concurrency.
- Preserve original source files with SHA-256 digests and byte sizes.
- Freeze prompt text, prompt hash, model, and explicitly configured sampling parameters on each `Job`.
- Preserve page-level machine output, normalized metadata, and an SDK-serialized OpenRouter response snapshot on `JobSource`.
- Organize historical `Person` records through many-to-many Document relationships and extensible roles.
- Classify Documents through a registry with stable type codes.
- Maintain human revision separately from machine-generated text.
- Isolate page failures so multi-page jobs can complete with partial success.
- Operate across supported platforms through Python-based application and maintenance tooling.
V4.2 extends this baseline with exact OpenRouter transport evidence and provider-neutral derived-artifact provenance. See the [V4.2 Scope Boundary](../ver4.2/scope_boundary_v4_2.md).
## Technical Stack
- **Runtime:** Python 3.12 or later.
- **Web application:** FastAPI and NiceGUI.
- **Persistence:** SQLModel and SQLAlchemy, with SQLite and PostgreSQL support.
- **Validation and settings:** Pydantic V2 and pydantic-settings.
- **Concurrency:** Python `asyncio` workers.
- **Vision integration:** OpenRouter through the application's provider adapter.
- **Testing and quality:** pytest, pytest-asyncio, Ruff, and ty.
## Runtime Topology
The runtime operates as an asynchronous Python application:
@@ -110,7 +135,7 @@ Responsibilities:
1. User uploads one or more images for a `Document`.
2. System stores files, hashes them, creates ordered `Source` rows, and creates a `Job`.
3. Worker claims the job, marks it `processing`, and executes page calls concurrently.
4. Each page writes a `JobSource` result with raw output, metadata, and full provider response evidence.
4. Each page writes a `JobSource` result with machine output, normalized metadata, and an SDK-serialized OpenRouter response snapshot.
5. Aggregate status becomes `completed`, `partial_success`, or `failed`.
### 2. Document-Person Relationship Management
@@ -128,12 +153,13 @@ Responsibilities:
3. Persistence stores the `document_type_id` reference.
4. Inactive types remain valid for historical rows but are excluded from default selectors.
## Domain Invariants
## V4 Domain Rules
- `Source.raw_transcription` stores immutable machine output.
- `JobSource.raw_transcription` preserves page output for its Job execution.
- `Source.raw_transcription` is the latest-success machine-output projection for a page.
- Human corrections occur only in `Source.revised_text`.
- Prompt and parameter provenance is frozen on `Job` at submission time.
- Provider output evidence is stored on `JobSource` for each page execution.
- The SDK-serialized OpenRouter response snapshot is stored on `JobSource` for each successful page execution.
- `DocumentPerson` links are unique for `(document_id, person_id, role_id)`.
- Relationship mutations are deterministic and set-based.
- `DocumentType.code` is stable; `DocumentType.label` may evolve.
@@ -158,3 +184,5 @@ Responsibilities:
- [System Requirements](requirements_v4.md)
- [Data Model](schema_v4.md)
- [Error Handling Policy](error_handling_v4.md)
- [Error Handling Invariant](../invariant/error_handling.md)
- [Digital Evidence and AI Processing Provenance](../invariant/ai_evidence_and_provenance.md)