generated from john/python-template
This commit is contained in:
+62
-7
@@ -1,6 +1,6 @@
|
||||
# System Architecture (Current Baseline: V5.1)
|
||||
# System Architecture (Current Baseline: V6.1)
|
||||
|
||||
This document defines the current V5.1 architecture baseline.
|
||||
This document defines the current V6.1 architecture baseline.
|
||||
|
||||
## Architecture Objectives
|
||||
|
||||
@@ -13,10 +13,11 @@ This document defines the current V5.1 architecture baseline.
|
||||
|
||||
- **Runtime:** Python 3.12+
|
||||
- **Web application:** FastAPI + NiceGUI
|
||||
- **Persistence:** SQLModel / SQLAlchemy (SQLite-first, PostgreSQL-compatible model design)
|
||||
- **Persistence:** SQLModel / SQLAlchemy — PostgreSQL in production, SQLite for local development and tests
|
||||
- **Validation and settings:** Pydantic V2 + pydantic-settings
|
||||
- **Concurrency:** asyncio worker loop
|
||||
- **Provider integration:** OpenRouter adapter behind provider interface
|
||||
- **Deployment:** Docker Compose (app, worker, PostgreSQL, Cloudflare Tunnel)
|
||||
- **Quality and tests:** Ruff, ty, pytest, pytest-asyncio
|
||||
|
||||
## Runtime Topology
|
||||
@@ -25,13 +26,31 @@ This document defines the current V5.1 architecture baseline.
|
||||
flowchart LR
|
||||
U[Browser User] --> A[FastAPI + NiceGUI App]
|
||||
A --> W[Asyncio Worker]
|
||||
A --> DB[(SQLite/PostgreSQL Model)]
|
||||
A --> DB[(PostgreSQL / SQLite)]
|
||||
W --> P[Provider Adapter]
|
||||
W --> DB
|
||||
```
|
||||
|
||||
For production split-process deployments, the worker may run as a dedicated service
|
||||
while the app process runs with `RUN_EMBEDDED_WORKER=false`.
|
||||
The worker loop drains two queues in the same pass: queued transcription Jobs and queued
|
||||
`MaintenanceRun` records. When neither has work, it idles.
|
||||
|
||||
### Production deployment
|
||||
|
||||
Production runs as a Docker Compose stack with the app and worker as separate services, so the app
|
||||
process runs with `RUN_EMBEDDED_WORKER=false` and the worker process owns queue draining. Local
|
||||
development runs a single process with the worker embedded.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
I[Internet] --> CF[cloudflared tunnel + Access]
|
||||
CF --> APP[app service]
|
||||
APP --> PG[(postgres service)]
|
||||
WK[worker service] --> PG
|
||||
WK --> PROV[OpenRouter]
|
||||
```
|
||||
|
||||
Deployment, rollback, and recovery procedures are in [Production Runbook](production-runbook.md);
|
||||
backup configuration and restore are in [Backup and Restore](backup_restore.md).
|
||||
|
||||
## Layered Boundaries
|
||||
|
||||
@@ -48,11 +67,18 @@ Responsibilities:
|
||||
|
||||
### Service and Orchestration Layer
|
||||
|
||||
Aggregate services:
|
||||
|
||||
- `src/transcription/services/documents.py`
|
||||
- `src/transcription/services/people.py`
|
||||
- `src/transcription/services/jobs.py`
|
||||
- `src/transcription/services/sources.py`
|
||||
- `src/transcription/services/evidence.py`
|
||||
- `src/transcription/services/photos.py`
|
||||
- `src/transcription/services/maintenance.py`
|
||||
- `src/transcription/services/evidence.py` (read/projection only)
|
||||
|
||||
Orchestration modules:
|
||||
|
||||
- `src/transcription/services/store.py`
|
||||
- `src/transcription/services/workflows.py`
|
||||
|
||||
@@ -61,6 +87,11 @@ Responsibilities:
|
||||
- Aggregate ownership and invariants.
|
||||
- Transaction-aware write helpers.
|
||||
- Cross-service workflows in orchestration modules (`store.py`, `workflows.py`).
|
||||
- Lookup-table CRUD through the generic `RegistryService` base (`registry.py`), which is not an
|
||||
aggregate owner itself.
|
||||
|
||||
Module classification and per-model ownership are defined in
|
||||
[services instructions](../.github/instructions/services.instructions.md).
|
||||
|
||||
### Persistence Layer
|
||||
|
||||
@@ -88,7 +119,11 @@ Responsibilities:
|
||||
- `Job` is an aggregate processing run with status and frozen prompt/runtime settings.
|
||||
- `JobSource` is queue/membership state for one `(job, source)` pair.
|
||||
- `ExecutionAttempt` is append-only evidence for each provider call.
|
||||
- `Photo` is person imagery owned by `PhotosService`.
|
||||
- `MaintenanceRun` is one queued or executed operational maintenance run.
|
||||
- `DocumentType` and `PersonRole` are UUID-backed registries with optional protected `semantic_key`.
|
||||
- `Tag` is a shared registry reached through both document and person tagging, linked by
|
||||
`DocumentTag` and `PersonTag`.
|
||||
|
||||
## Processing and Evidence Workflow
|
||||
|
||||
@@ -109,6 +144,24 @@ Responsibilities:
|
||||
- **Job statuses:** `queued`, `processing`, `transcribed`, `partial_success`, `failed`
|
||||
- Operational success path resolves to `transcribed`.
|
||||
- **JobSource statuses:** `pending`, `transcribed`, `failed`, `cancelled`
|
||||
- **MaintenanceRun statuses:** `queued`, `processing`, `succeeded`, `failed`
|
||||
- Maintenance uses `succeeded` rather than `transcribed`; the transcription vocabulary does not
|
||||
apply to operational runs.
|
||||
- **Maintenance job types:** `backup`, `storage_reconciliation`
|
||||
|
||||
## Maintenance Execution
|
||||
|
||||
Operational maintenance is queue-backed rather than run inline from the UI, so it survives request
|
||||
lifetime and is recorded:
|
||||
|
||||
1. Settings enqueues a `MaintenanceRun` with `status=queued` and a `triggered_by` marker.
|
||||
2. The worker claims the oldest queued run with a conditional update, moving it to `processing`.
|
||||
3. `backup` runs the deploy backup script; `storage_reconciliation` compares stored media against
|
||||
`Document`/`Source` records.
|
||||
4. The run finalizes to `succeeded` or `failed` with summary, timing, log path, and `error_detail`.
|
||||
|
||||
`MaintenanceRun` records operational history and is not evidence in the `ExecutionAttempt` sense;
|
||||
append-only guarantees apply to transcription attempts.
|
||||
|
||||
## Security and Path Handling Boundaries
|
||||
|
||||
@@ -168,5 +221,7 @@ Current architecture rules live in `docs/*`.
|
||||
- [System Requirements](requirements.md)
|
||||
- [Data Model](schema.md)
|
||||
- [Error Handling Policy](error_handling.md)
|
||||
- [Production Runbook](production-runbook.md)
|
||||
- [Backup and Restore](backup_restore.md)
|
||||
- [Error Handling invariant](./invariant/error_handling.md)
|
||||
- [AI evidence invariant](./invariant/ai_evidence_and_provenance.md)
|
||||
|
||||
Reference in New Issue
Block a user