16 Commits
Author SHA1 Message Date
John Lancaster 661e2b1bec smoothed readme and startup 2026-08-01 09:51:21 -05:00
John Lancaster d75083a666 shutdown fixes 2026-08-01 09:36:34 -05:00
John Lancaster d0a3ca0289 WIP theming 2026-07-31 19:34:12 -05:00
John Lancaster 209c48987c separated cli settings 2026-07-31 19:23:12 -05:00
John Lancaster 4ed1f43eda ui instructions 2026-07-31 16:01:23 -05:00
John Lancaster ce8fcce6b0 Merge commit '4ae8e5be4f60059ba611ceea0cf553b009e6a88e' into session-engine 2026-07-31 15:40:20 -05:00
John Lancaster 6b5b0500b3 unified implementation plan 2026-07-31 10:28:47 -05:00
John Lancaster bbf7fe28c2 cleanup 2026-07-31 10:20:02 -05:00
John Lancaster c4d25c1be8 Merge remote-tracking branch 'origin/doc_update' into session-engine 2026-07-31 10:16:17 -05:00
John Lancaster 1fa5eb1127 doc updates for pydantic 2026-07-31 10:12:34 -05:00
John Lancaster ec6617a1c4 updates 2026-07-30 23:16:17 -05:00
John Lancaster 9eb0f40c08 session and engine 2026-07-30 22:33:49 -05:00
John Lancaster f769d29da1 uv.lock update 2026-07-30 21:17:09 -05:00
John Lancaster 8afc462a6d startup 2026-07-30 21:16:48 -05:00
John Lancaster 3d6daec561 moved models to db pkg 2026-07-30 21:10:13 -05:00
John Lancaster 1cc2f319d5 uvicorn startup 2026-07-30 21:09:51 -05:00
56 changed files with 2147 additions and 944 deletions
+49 -2
View File
@@ -1,6 +1,53 @@
--- ---
description: Copilot rules for modifying the UI description: "Use when modifying the NiceGUI application under src/transcription/ui. Defines ownership and dependency boundaries for UI registration, pages, components, services, persistence, state, and static assets."
applyTo: 'src/transcription/ui/**/*.py' applyTo: 'src/transcription/ui/**/*.py'
--- ---
The UI is forbidden from directly touching the database. All interactions should be done using the [services](../../src/transcription/services/) # UI Conceptual Boundaries
Keep dependencies flowing in this direction:
`ui/__init__.py` -> `pages` -> `components`
Pages may depend on application services and framework-provided dependencies. Components may depend on smaller components and shared presentation helpers. Services and domain modules must never depend on the UI.
## Package Root
- Keep `ui/__init__.py` as the UI composition root: register global assets, register pages, and mount NiceGUI on FastAPI.
- Do not put feature rendering, service calls, persistence, or route-specific state in the package root.
## Pages
- Pages own route registration and route-level orchestration.
- Resolve request or application dependencies, call [services](../../src/transcription/services/), adapt returned data for presentation when needed, and coordinate refresh, navigation, and notifications here.
- Do not query, mutate, commit, or roll back the database from a page. Do not import database engines, sessions, operations, or query-building APIs. Persistence belongs to services or workflow functions.
- Framework dependency types may cross into page handlers only to construct or invoke services; do not pass sessions or session factories into components.
- Keep business rules, lifecycle transitions, transaction boundaries, and cross-service workflows out of page callbacks.
## Components
- Components own reusable rendering, widget-local state, input normalization, and presentation-only formatting.
- Expose user actions through typed callback parameters. The calling page decides which service or workflow runs and what refresh or navigation follows.
- Do not register routes, resolve request/app state, instantiate services, or access persistence from components.
- Components may accept ORM models returned by services as read-only snapshots. Only use fields and relationships that the service loaded eagerly; never mutate models, trigger lazy loading, or expose session behavior.
- A component may compose lower-level components, but it must not import from `pages`.
## Shared UI Infrastructure
- Keep app-wide navigation and layout primitives in `components/app_shell.py`.
- Keep generic table/event adaptation in `components/table/common.py`; feature-specific columns, row read models, and formatting belong in the feature table module.
- Keep exception normalization and user-facing error display in `components/error_presenter.py`; preserve `AppError` details and operation identifiers at page/component boundaries.
## CSS Assets
- Keep CSS under `ui/static` and split it into manageable, feature-oriented files. Do not grow a monolithic stylesheet or embed substantial style blocks in Python components.
- Load each stylesheet from the page, component, or composition root that needs it with `ui.add_css(...)`. Use shared registration only for genuinely application-wide styles.
- Read stylesheet text through `importlib.resources.files(...)` so loading works from installed packages and is independent of the working directory.
- Centralize CSS reading in one typed helper cached by relative resource path with `functools.cache` or an equivalent unbounded `lru_cache`. Cache the immutable stylesheet text to avoid repeated resource I/O during component renders; keep NiceGUI registration decisions at the caller.
- Do not encode application behavior in CSS or other static assets.
## State and Side Effects
- Limit component state to ephemeral interaction state such as loading flags, form values, dialogs, and expansion state.
- Application and worker state must be resolved at the page or application boundary and passed through narrow interfaces such as callbacks or notifier protocols.
- Keep filesystem, network, provider, and worker orchestration behind application services or dedicated adapters. UI code may trigger those operations but must not implement them.
+4 -8
View File
@@ -8,14 +8,10 @@
"module": "debugpy", "module": "debugpy",
"args": [ "args": [
"-m", "-m",
"uvicorn", "transcription",
"transcription.app:create_app", "--host", "127.0.0.1",
"--factory", "--port", "9999",
"--host", "--database.driver", "sqlite"
// "127.0.0.1",
"0.0.0.0",
"--port",
"8080"
], ],
"justMyCode": true, "justMyCode": true,
"console": "integratedTerminal", "console": "integratedTerminal",
+72 -6
View File
@@ -22,18 +22,80 @@ uv sync
### 2) Configure environment ### 2) Configure environment
Create a `.env` file in the project root (minimum required setting shown): Create a `.env` file in the project root with the required OpenRouter API key:
```env ```env
OPENROUTER_API_KEY=your_openrouter_api_key OPENROUTER_API_KEY=your_openrouter_api_key
``` ```
Optional settings (defaults shown): Settings are read from CLI arguments first, then environment variables, then `.env`, then the defaults below.
### Configuration Source Precedence
When the same setting is provided in multiple places, the value is chosen in this order (highest priority first):
1. CLI arguments (for example `--port 9999`)
2. Settings constructor arguments (used mainly in tests)
3. Environment variables
4. `.env` file values
5. Model defaults in code
Practical examples:
- `--port 9999` overrides both `PORT=8000` in the shell and `PORT=7000` in `.env`.
- `DATABASE__PATH=prod.db` in the shell overrides `DATABASE__PATH=dev.db` in `.env`.
#### Server and runtime
| Environment variable | Default | Description |
| --- | --- | --- |
| `HOST` | `0.0.0.0` | Address on which the server listens. |
| `PORT` | `8000` | Server port. |
| `LOG_LEVEL` | `info` | Uvicorn and application log level. |
| `RELOAD` | `false` | Restart the development server when source files change. |
| `ENVIRONMENT` | `development` | Runtime environment: `development`, `test`, or `production`. |
#### Provider
| Environment variable | Default | Description |
| --- | --- | --- |
| `PROVIDER` | `openrouter` | Transcription provider. |
| `OPENROUTER_API_KEY` | Required | OpenRouter API key. |
| `PROVIDER_MODEL` | Provider default | Optional model override. |
| `OPENROUTER_HTTP_REFERER` | Unset | Optional OpenRouter attribution URL. |
| `OPENROUTER_APP_TITLE` | Unset | Optional OpenRouter attribution title. |
#### Database and files
Use nested env vars for database settings (recommended):
```env ```env
DATABASE_URL=sqlite:///./transcription.db DATABASE__DRIVER=sqlite
DATABASE__PATH=app.db
# BOOTSTRAP_SCHEMA_ON_STARTUP=true
SQLITE_CHECK_SAME_THREAD=false
UPLOAD_DIR=./uploads UPLOAD_DIR=./uploads
PROMPT_DIR=./prompts PROMPT_DIR=./prompts
```
For PostgreSQL:
```env
DATABASE__DRIVER=postgres
DATABASE__HOST=localhost
DATABASE__PORT=5432
DATABASE__DATABASE=transcription
DATABASE__USER=postgres
DATABASE__PASSWORD=change-me
```
This uses Pydantic nested settings (`env_nested_delimiter='__'`) and avoids JSON blobs in `.env`. A top-level `DATABASE={...}` JSON value is still supported as a fallback, and nested keys such as `DATABASE__PATH` take precedence over conflicting JSON keys.
`BOOTSTRAP_SCHEMA_ON_STARTUP` creates missing tables when the app starts. When unset, it is enabled in `development` and `test`, and disabled in `production`; set it explicitly to override that policy. `SQLITE_CHECK_SAME_THREAD` defaults to `false`.
#### Worker
```env
WORKER_MAX_RETRIES=0 WORKER_MAX_RETRIES=0
WORKER_RETRY_BACKOFF_SECONDS=0 WORKER_RETRY_BACKOFF_SECONDS=0
WORKER_PROVIDER_TIMEOUT_SECONDS=20 WORKER_PROVIDER_TIMEOUT_SECONDS=20
@@ -45,13 +107,17 @@ WORKER_FAIL_ON_FINISH_REASON_LENGTH=false
### 3) Run the app ### 3) Run the app
```bash ```bash
uv run uvicorn transcription.app:create_app --factory --reload uv run python -m transcription --port 9999 --reload --database.driver sqlite --bootstrap-schema-on-startup
``` ```
This starts the development server with SQLite, creates missing tables, and enables automatic reload. Run `uv run python -m transcription --help` for all CLI options; CLI names use kebab case and nested database options use dot notation, such as `--database.path ./data/transcription.db`.
### 4) Open in browser ### 4) Open in browser
- GUI: [http://[IP_ADDRESS]:8000/ui](http://[IP_ADDRESS]:8000/ui) - GUI: [http://localhost:9999/ui](http://localhost:9999/ui)
- Health check: [http://[IP_ADDRESS]:8000/healthz](http://[IP_ADDRESS]:8000/healthz) - Health check: [http://localhost:9999/healthz](http://localhost:9999/healthz)
Replace `localhost` with the server's hostname or IP address when connecting from another machine.
## How to navigate the GUI ## How to navigate the GUI
+41 -232
View File
@@ -1,248 +1,57 @@
# Implementation Plan (Version 2) # implementation_plan_v2
This plan defines the path from the V1 baseline to **Version 2 complete**, aligned to the updated multi-image and multi-person relational domain model: ## Goal
* `Document` acts as a logical parent container for physical artifacts, supporting multi-author and multi-recipient relationships via `DocumentPerson`. Replace the current V1 SQLModel schema with the approved V2 schema and make sure every database operation works through the existing async SQLAlchemy/SQLModel session layer.
* `Source` represents an individual image page within a document, maintaining sequential order (`page_number`), cached active machine output (`raw_transcription`), and inline single user revisions (`revised_text`).
* `Job` acts as an overarching batch orchestrator for multi-page async processing tasks.
* `JobSource` records individual point-in-time API executions per image page, storing Pydantic-validated `ai_metadata` and raw REST envelopes (`raw_api_response`).
* **Pydantic V2** acts as the single source of truth for runtime validation, API payload parsing, and PostgreSQL JSONB serialization.
The objective is to complete the V2 scope with production readiness while keeping non-V2 enhancements out of active delivery. Use a fresh database. There will be no migrations, data conversion, legacy compatibility shims, or parallel V1/V2 code paths.
--- ## Current Project Impact
## V2 Completion Definition - `src/transcription/db/models.py` still defines the V1 `Document`, `Source`, `Job`, and `Revision` tables.
- The V2 target adds `Person`, `DocumentPerson`, and `JobSource`, moves revisions onto `Source`, and removes the direct `Source.job_id` relationship.
- The engine, session factory, transaction handling, and PostgreSQL async support already exist and do not need to be rewritten.
- Async CRUD currently lives in `DocumentService`, `JobService`, `TranscriptionService`, and the upload record helper. Their queries and eager-loading options depend on V1 relationships.
- Existing tests cover only part of the schema and CRUD surface.
V2 is complete when all of the following are true: ## Implementation
1. **Functional complete** ### 1. Update the schema
* Multi-image and whole-folder uploads assign sequential page numbers to `Source` records under a single `Document`.
* Batch jobs process pages concurrently using an `asyncio` worker pool with semaphore rate limiting.
* Partial job failures resolve cleanly to `partial_success`, allowing single-page retries without re-running successful pages.
* Multi-author and multi-recipient tagging is supported on `Document`.
- Replace the models in `src/transcription/db/models.py` with the approved V2 tables, enums, relationships, foreign keys, constraints, and indexes.
- Remove `Revision`, `Source.job_id`, and the transcription fields that no longer belong on `Job`.
- Keep `create_all()` as the schema bootstrap for a fresh database.
- Delete `_ensure_sqlite_compat_columns()` and all schema patching from `src/transcription/db/operations.py`.
- Keep the Python models, `docs/schema_v2.md`, and `docs/ddl_v2.sql` consistent.
2. **Data-model complete** ### 2. Align the async CRUD methods
* SQLite is fully replaced with PostgreSQL (using `asyncpg` or `psycopg3`).
* Pydantic V2 models validate all API payloads, database row mappings, and `JSONB` structures.
- Keep the existing `ServiceBase` session and transaction pattern.
- Update document CRUD to load and manage its ordered `Source` rows and `DocumentPerson` links.
- Update job CRUD and queue queries to use `JobSource` instead of `Source.job_id`.
- Add the missing async CRUD operations for `Person`, `Source`, `DocumentPerson`, and `JobSource` using the existing service style. Do not add another repository abstraction.
- Replace revision CRUD with direct updates to `Source.revised_text` and `Source.date_revised`.
- Remove the temporary transcript compatibility aliases instead of redirecting them.
- Update only direct database call sites that construct or query these records; UI and worker feature changes are not part of this work.
3. **Operational complete** ### 3. Verify the schema and CRUD
* Concurrency controls, worker pool metrics, and database connections operate safely under batch load.
- Update the schema bootstrap test to expect `person`, `document`, `document_person`, `source`, `job`, and `job_source`, with no `revision` table.
- Add async create, read, update, delete, list, and filtered-query tests for each entity that exposes those operations.
- Test relationship loading, page ordering, uniqueness constraints, delete behavior, status values, and `JobSource` JSON fields.
- Test both service-owned sessions and caller-provided sessions so flush/commit behavior remains correct.
- Run the focused database and service tests, then the full suite with `uv run pytest`.
4. **Documentation complete** ## Done When
* `schema_v2.md`, `DDL_v2.sql`, Pydantic model contracts are updated and consistent.
---
## Phase 1 — Data Contract Stabilization & Pydantic Baseline
**Goal:** Lock the PostgreSQL schema, DDL, and Pydantic V2 models before refactoring service logic.
### Tasks
1. Finalize DDL for PostgreSQL native types (`UUID`, `TIMESTAMPTZ`, `JSONB`) and junction tables (`document_person`, `job_source`).
2. Build core Pydantic V2 schemas (`Person`, `Document`, `Source`, `Job`, `JobSource`, `PageAIMetadata`).
3. Confirm and document data invariants:
* `source.raw_transcription` and `job_source.raw_transcription` are immutable machine outputs.
* `source.revised_text` holds user edits. UI renders `COALESCE(revised_text, raw_transcription)`.
* Page sequence is strictly ordered by `source.page_number ASC`.
4. Freeze V2 job status values (`queued`, `processing`, `completed`, `partial_success`, `failed`) and page execution status values (`pending`, `transcribed`, `failed`).
### Deliverables
* Canonical `docs/schema_v2.md` and `docs/DDL_v2.sql`.
* Centralized Pydantic validation suite in `models/schemas_v2.py`.
### Exit Criteria
* All database tables, relationships, and JSONB structures have corresponding Pydantic V2 models passing unit validation tests.
---
## Phase 2 — Persistence Layer Transition (SQLite to PostgreSQL)
**Goal:** Replace the SQLite storage layer with an asynchronous PostgreSQL driver (`asyncpg` or `psycopg3`).
### Tasks
1. Configure PostgreSQL database connection pooling and environment configuration.
2. Refactor `services/store.py` / repository layers to execute parameterized async SQL queries (`$1`, `$2`).
3. Implement JSONB serialization and deserialization helpers using Pydantic's `.model_dump_json()` and `.model_validate()`.
4. Implement database bootstrap routines for PostgreSQL table creation and index initialization.
### Deliverables
* PostgreSQL-native database connection and query service modules.
* Integration test suite confirming connection pooling and JSONB CRUD operations.
### Exit Criteria
* All database reads/writes run asynchronously against PostgreSQL with zero remaining SQLite driver dependencies.
---
## Phase 3 — Service Layer & `asyncio` Engine Refactor
**Goal:** Implement batch orchestration and parallel single-image API execution.
### Tasks
1. Refactor upload service to process folder/multi-image input:
* Group files into a single `Document`.
* Create ordered `Source` rows (`page_number = 1..N`).
2. Refactor `services/workflows.py` with `asyncio` worker pools:
* Use `asyncio.Semaphore` to enforce API provider rate limits.
* Issue parallel single-image requests to Vision APIs (OpenAI/Claude).
* Parse API responses directly into Pydantic models (`PageAIMetadata`).
3. Update execution tracking:
* Create a `JobSource` row per page call to record `raw_transcription`, `ai_metadata`, and `raw_api_response`.
* Update active `source.raw_transcription` upon task completion.
* Calculate aggregate batch status (`completed`, `partial_success`, `failed`) on the parent `Job`.
4. Refactor `services/person.py` and `services/documents.py` to handle multi-person roles via `document_person`.
### Deliverables
* Asynchronous batch execution engine in `services/workflows.py`.
* Service routines for multi-person tagging and page-level retries.
### Exit Criteria
* Executing a folder upload of 10+ images processes concurrently, populates page-level `JobSource` entries, and handles partial worker errors without crashing the batch.
---
## Phase 4 — UI & API Contract Alignment
**Goal:** Update API endpoints and frontend/UI views to render multi-page documents and person roles.
### Tasks
1. Update document and job API endpoints to accept batch file arrays and multi-person ID payloads.
2. Update UI document views:
* Render multi-page document transcriptions sequentially by `page_number`.
* Display author and recipient chips/cards linked from `document_person`.
3. Update job detail UI to show page-level execution statuses (`transcribed` vs. `failed`) and provide a "Retry Failed Pages" action for `partial_success` jobs.
4. Align inline page editing controls to update `source.revised_text` and `source.date_revised`.
### Deliverables
* Refactored API routes and UI components supporting multi-page rendering and person management.
### Exit Criteria
* UI successfully displays multi-page document text, allows per-page human revisions, and shows author/recipient metadata.
---
## Phase 5 — Test Suite Realignment & Concurrency Testing
**Goal:** Ensure end-to-end system stability under concurrent async execution and load.
### Tasks
1. Write unit tests for Pydantic models, custom validators, and JSONB conversions.
2. Write integration tests for async database operations:
* CRUD for `Document`, `Person`, `DocumentPerson`, `Source`, `Job`, and `JobSource`.
3. Write mock-backed async workflow tests:
* Verify `asyncio.Semaphore` bounds concurrent tasks properly.
* Validate state transition logic for `completed`, `partial_success`, and `failed` jobs.
* Confirm retry routines process only targeted `JobSource` records marked as `failed`.
4. Re-enable CI quality gates (linting, type checking with Pyright/mypy, pytest).
### Deliverables
* Passing asynchronous test suite covering core workflows, edge cases, and failure recoveries.
### Exit Criteria
* CI pipeline is green with comprehensive coverage across database operations, Pydantic models, and worker queues.
---
## Phase 6 — Reliability, Operations, and Release Readiness
**Goal:** Prepare V2 for production deployment and operator management.
### Tasks
1. Verify structured logging includes `job_id`, `document_id`, `source_id`, and `person_id`.
2. Tune PostgreSQL connection pool limits and `asyncio` concurrency thresholds for production infrastructure.
3. Update operational documentation:
* Review and update `docs/schema_v2.md` as needed.
* Create `docs/runbook_v2.md` detailing PostgreSQL maintenance, JSONB index management, and worker queue monitoring.
* Create `docs/release_checklist_v2.md` for launch sign-off.
### Deliverables
* Updated project documentation and operational runbooks.
* V2 release sign-off checklist.
### Exit Criteria
* All documentation reflects V2 architecture; launch checklist is fully verified.
---
## Requirement Traceability Focus
Maintain evidence against these V2 requirement groups:
* **Batch & Multi-Image Pipeline:** Folder ingestion, page ordering, async worker execution.
* **Database & Persistence:** PostgreSQL, native UUIDs, JSONB execution storage, `asyncpg` pooling.
* **Validation & Schemas:** Pydantic V2 models for DB rows, API requests, and AI vision responses.
* **Attribution & Metadata:** Multi-author and multi-recipient tagging, biographical entity management.
* **Error Recovery:** Partial success states, page-level status flags, isolated retry execution.
---
## Scope Discipline Rule (V2 Focus)
* Only tasks required for V2 scope (PostgreSQL, Pydantic V2, folder/async processing, multi-person roles) enter this plan.
* V3 candidate features (such as side-by-side multi-provider model output comparison) remain strictly in the future backlog.
* Any schema adjustments during implementation require immediate updates to `DDL_v2.sql`, Pydantic models, and `schema_v2.md`.
---
## Technology References
- [FastAPI documentation](https://fastapi.tiangolo.com/)
- [NiceGUI documentation](https://nicegui.io/documentation)
- [PostgreSQL documentation](https://www.postgresql.org/docs/)
- [Python asyncio](https://docs.python.org/3/library/asyncio.html#module-asyncio)
- [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)
- [Pydantic AI](https://pydantic.dev/docs/ai/overview/)
## Related Local References
- [System Overview](index_v2.md)
- [System Design Intent](intent.md)
- [Transcription Methodology](transcription_methodology.md)
- [System Architecture](architecture_v2.md)
- [System Requirements](requirements_v2.md)
- [Data model](schema_v2.md)
- [Error Handling Policy](error_handling_v2.md)
- Implementation Plan (this document)
- A fresh database is created directly from the V2 SQLModel metadata.
- All async CRUD methods pass against the V2 relationships and fields.
- No code references `Revision`, `Source.job_id`, removed `Job` transcription fields, or compatibility aliases.
- The focused tests and full test suite pass.
## Out of Scope
- Database migrations or preservation of V1 data
- Legacy compatibility code
- Database engine or session-layer rewrites
- UI redesign, batch orchestration, worker concurrency, deployment, and operational runbooks
+1 -1
View File
@@ -19,7 +19,7 @@ Read [architecture_v2.md](architecture_v2.md) first for technical overview and s
## Technical Stack ## Technical Stack
* **Application Web Framework:** FastAPI + NiceGUI * **Application Web Framework:** FastAPI + NiceGUI
* **Persistence Engine:** PostgreSQL 13+ * **Persistence Engine:** PostgreSQL 18+
* **Data Validation & Schemas:** Pydantic V2 * **Data Validation & Schemas:** Pydantic V2
* **Concurrency & Workers:** Python `asyncio` worker pool with `asyncio.Semaphore` * **Concurrency & Workers:** Python `asyncio` worker pool with `asyncio.Semaphore`
* **Vision Providers:** OpenAI (GPT-4o) and Anthropic (Claude 3.5 Sonnet) via native SDKs * **Vision Providers:** OpenAI (GPT-4o) and Anthropic (Claude 3.5 Sonnet) via native SDKs
+35
View File
@@ -0,0 +1,35 @@
# AI Coding Assistant Project Briefing & Context
## Project Mission
This application is a family history archival and transcription platform. Its primary goal is to accept scanned document images (letters, postcards, logbooks, diaries), execute OCR and structured transcription via AI vision models (OpenAI GPT-4o, Anthropic Claude 3.5 Sonnet), and manage historical metadata (authors, recipients, dates, and locations).
---
## Technical Stack & Architecture
* **Database:** PostgreSQL 13+ with native `UUID` (`gen_random_uuid()`) and `JSONB` columns.
* **Backend Runtime / Concurrency:** Python utilizing `asyncio` for concurrent HTTP API calls to AI providers, with strict rate-limiting via `asyncio.Semaphore`.
* **Validation & Types:** Python with **Pydantic** model definitions. Incoming AI responses must be parsed and validated with Pydantic models *before* database insertion.
* **ORM / Database Access:** SQLModel and SQLAlchemy, using parameterized statements and PostgreSQL-native types.
---
## Core System Directives for AI Code Generation
### 1. Data Immutability vs. Human Corrections
* `job_source.raw_transcription` and `source.raw_transcription` represent original, point-in-time machine outputs and are **immutable**.
* Human corrections occur on `source.revised_text`.
* When fetching text for the UI, always display `COALESCE(source.revised_text, source.raw_transcription)`.
### 2. Async Execution & Batching Rules
* A `job` represents an overarching execution run for a folder/group of images belonging to a single `document`.
* Images are submitted to AI APIs **one at a time in rapid succession** using `asyncio` worker pools.
* Each single-image API call populates a row in `job_source` with its own `status`, `raw_transcription`, `ai_metadata`, and `raw_api_response`.
* If 9 of 10 pages succeed and 1 fails, `job_source.status` for the failed image becomes `'failed'`, while `job.status` becomes `'partial_success'`. Do not mark the entire batch as failed if partial results exist.
### 3. Entity Relationships
* **Authors/Recipients:** A `document` can have multiple authors and recipients. Do NOT put direct `author_id` foreign keys on `document`. Query authors/recipients via `document_person` where `role = 'author'` or `role = 'recipient'`.
* **Page Ordering:** Multi-page documents must always be queried using `ORDER BY page_number ASC`.
### 4. Database Mutations
* Always use parameterized SQL queries (`$1`, `$2`) to prevent SQL injection.
* Store datetimes using UTC ISO 8601 strings or native PostgreSQL `TIMESTAMPTZ`.
+416
View File
@@ -0,0 +1,416 @@
# SQLModel Table Models
These models implement the canonical [Version 2 database schema](../schema_v2.md). Each schema entity is represented by exactly one `SQLModel` table class. Because `SQLModel` is built on Pydantic and SQLAlchemy, these classes provide application validation and PostgreSQL mappings without parallel row and create models.
Database-generated UUIDs and timestamps are `None` until PostgreSQL supplies their values during insert. The database columns remain non-nullable. `Person.metadata_` maps to the `metadata` column because `metadata` is reserved by SQLAlchemy's declarative API.
```python
from datetime import date
from datetime import datetime
from enum import StrEnum
from uuid import UUID
from pydantic import JsonValue
from sqlalchemy import Column
from sqlalchemy import Date
from sqlalchemy import DateTime
from sqlalchemy import ForeignKey
from sqlalchemy import Index
from sqlalchemy import Integer
from sqlalchemy import String
from sqlalchemy import Text
from sqlalchemy import UniqueConstraint
from sqlalchemy import text
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.dialects.postgresql import UUID as PostgreSQLUUID
from sqlmodel import Field
from sqlmodel import Relationship
from sqlmodel import SQLModel
class PersonRole(StrEnum):
AUTHOR = "author"
RECIPIENT = "recipient"
class JobStatus(StrEnum):
QUEUED = "queued"
PROCESSING = "processing"
COMPLETED = "completed"
PARTIAL_SUCCESS = "partial_success"
FAILED = "failed"
class JobSourceStatus(StrEnum):
PENDING = "pending"
TRANSCRIBED = "transcribed"
FAILED = "failed"
class Person(SQLModel, table=True):
__tablename__ = "person"
__table_args__ = (Index("idx_person_full_name", "full_name"),)
id: UUID | None = Field(
default=None,
sa_column=Column(
PostgreSQLUUID(as_uuid=True),
primary_key=True,
server_default=text("gen_random_uuid()"),
),
)
full_name: str = Field(sa_column=Column(Text, nullable=False))
display_name: str | None = Field(default=None, sa_column=Column(Text))
maiden_name: str | None = Field(default=None, sa_column=Column(Text))
birth_date: date | None = Field(default=None, sa_column=Column(Date))
birth_date_raw: str | None = Field(default=None, sa_column=Column(Text))
birth_place: str | None = Field(default=None, sa_column=Column(Text))
death_date: date | None = Field(default=None, sa_column=Column(Date))
death_date_raw: str | None = Field(default=None, sa_column=Column(Text))
death_place: str | None = Field(default=None, sa_column=Column(Text))
biography: str | None = Field(default=None, sa_column=Column(Text))
portrait_path: str | None = Field(default=None, sa_column=Column(Text))
metadata_: JsonValue | None = Field(
default_factory=dict,
sa_column=Column(
"metadata",
JSONB,
server_default=text("'{}'::jsonb"),
),
)
created_at: datetime | None = Field(
default=None,
sa_column=Column(
DateTime(timezone=True),
nullable=False,
server_default=text("now()"),
),
)
updated_at: datetime | None = Field(
default=None,
sa_column=Column(
DateTime(timezone=True),
nullable=False,
server_default=text("now()"),
),
)
document_people: list["DocumentPerson"] = Relationship(
back_populates="person",
sa_relationship_kwargs={"lazy": "raise", "passive_deletes": True},
)
class Document(SQLModel, table=True):
__tablename__ = "document"
__table_args__ = (Index("idx_document_date", "document_date"),)
id: UUID | None = Field(
default=None,
sa_column=Column(
PostgreSQLUUID(as_uuid=True),
primary_key=True,
server_default=text("gen_random_uuid()"),
),
)
name: str = Field(sa_column=Column(Text, nullable=False))
document_type: str | None = Field(default=None, sa_column=Column(Text))
document_date: date | None = Field(default=None, sa_column=Column(Date))
document_date_raw: str | None = Field(default=None, sa_column=Column(Text))
location_created: str | None = Field(default=None, sa_column=Column(Text))
notes: str | None = Field(default=None, sa_column=Column(Text))
archive_identifier: str | None = Field(default=None, sa_column=Column(Text))
created_at: datetime | None = Field(
default=None,
sa_column=Column(
DateTime(timezone=True),
nullable=False,
server_default=text("now()"),
),
)
updated_at: datetime | None = Field(
default=None,
sa_column=Column(
DateTime(timezone=True),
nullable=False,
server_default=text("now()"),
),
)
document_people: list["DocumentPerson"] = Relationship(
back_populates="document",
sa_relationship_kwargs={"lazy": "raise", "passive_deletes": True},
)
jobs: list["Job"] = Relationship(
back_populates="document",
sa_relationship_kwargs={"lazy": "raise", "passive_deletes": True},
)
sources: list["Source"] = Relationship(
back_populates="document",
sa_relationship_kwargs={"lazy": "raise", "passive_deletes": True},
)
class DocumentPerson(SQLModel, table=True):
__tablename__ = "document_person"
__table_args__ = (
UniqueConstraint(
"document_id",
"person_id",
"role",
name="unique_document_person_role",
),
Index("idx_document_person_doc", "document_id"),
Index("idx_document_person_per", "person_id"),
)
id: UUID | None = Field(
default=None,
sa_column=Column(
PostgreSQLUUID(as_uuid=True),
primary_key=True,
server_default=text("gen_random_uuid()"),
),
)
document_id: UUID = Field(
sa_column=Column(
PostgreSQLUUID(as_uuid=True),
ForeignKey("document.id", ondelete="CASCADE"),
nullable=False,
),
)
person_id: UUID = Field(
sa_column=Column(
PostgreSQLUUID(as_uuid=True),
ForeignKey("person.id", ondelete="CASCADE"),
nullable=False,
),
)
role: PersonRole = Field(sa_column=Column(String(20), nullable=False))
created_at: datetime | None = Field(
default=None,
sa_column=Column(
DateTime(timezone=True),
nullable=False,
server_default=text("now()"),
),
)
document: Document | None = Relationship(
back_populates="document_people",
sa_relationship_kwargs={"lazy": "raise"},
)
person: Person | None = Relationship(
back_populates="document_people",
sa_relationship_kwargs={"lazy": "raise"},
)
class Job(SQLModel, table=True):
__tablename__ = "job"
__table_args__ = (Index("idx_job_document", "document_id"),)
id: UUID | None = Field(
default=None,
sa_column=Column(
PostgreSQLUUID(as_uuid=True),
primary_key=True,
server_default=text("gen_random_uuid()"),
),
)
document_id: UUID = Field(
sa_column=Column(
PostgreSQLUUID(as_uuid=True),
ForeignKey("document.id", ondelete="CASCADE"),
nullable=False,
),
)
status: JobStatus = Field(
default=JobStatus.QUEUED,
sa_column=Column(
String(50),
nullable=False,
server_default=text("'queued'"),
),
)
retry_count: int = Field(
default=0,
sa_column=Column(
Integer,
nullable=False,
server_default=text("0"),
),
)
provider: str = Field(sa_column=Column(Text, nullable=False))
model: str = Field(sa_column=Column(Text, nullable=False))
prompt_name: str | None = Field(default=None, sa_column=Column(Text))
date_created: datetime | None = Field(
default=None,
sa_column=Column(
DateTime(timezone=True),
nullable=False,
server_default=text("now()"),
),
)
date_updated: datetime | None = Field(
default=None,
sa_column=Column(
DateTime(timezone=True),
nullable=False,
server_default=text("now()"),
),
)
document: Document | None = Relationship(
back_populates="jobs",
sa_relationship_kwargs={"lazy": "raise"},
)
job_sources: list["JobSource"] = Relationship(
back_populates="job",
sa_relationship_kwargs={"lazy": "raise", "passive_deletes": True},
)
class Source(SQLModel, table=True):
__tablename__ = "source"
__table_args__ = (
Index("idx_source_document", "document_id"),
Index("idx_source_page_order", "document_id", "page_number"),
)
id: UUID | None = Field(
default=None,
sa_column=Column(
PostgreSQLUUID(as_uuid=True),
primary_key=True,
server_default=text("gen_random_uuid()"),
),
)
document_id: UUID = Field(
sa_column=Column(
PostgreSQLUUID(as_uuid=True),
ForeignKey("document.id", ondelete="CASCADE"),
nullable=False,
),
)
page_number: int = Field(
default=1,
sa_column=Column(
Integer,
nullable=False,
server_default=text("1"),
),
)
upload_name: str = Field(sa_column=Column(Text, nullable=False))
filename: str = Field(sa_column=Column(Text, nullable=False))
file_path: str = Field(sa_column=Column(Text, nullable=False))
raw_transcription: str | None = Field(default=None, sa_column=Column(Text))
revised_text: str | None = Field(default=None, sa_column=Column(Text))
date_uploaded: datetime | None = Field(
default=None,
sa_column=Column(
DateTime(timezone=True),
nullable=False,
server_default=text("now()"),
),
)
date_revised: datetime | None = Field(
default=None,
sa_column=Column(DateTime(timezone=True)),
)
document: Document | None = Relationship(
back_populates="sources",
sa_relationship_kwargs={"lazy": "raise"},
)
job_sources: list["JobSource"] = Relationship(
back_populates="source",
sa_relationship_kwargs={"lazy": "raise", "passive_deletes": True},
)
class JobSource(SQLModel, table=True):
__tablename__ = "job_source"
__table_args__ = (
UniqueConstraint("job_id", "source_id", name="unique_job_source"),
Index("idx_job_source_job", "job_id"),
Index("idx_job_source_source", "source_id"),
Index(
"idx_job_source_ai_metadata",
"ai_metadata",
postgresql_using="gin",
),
)
id: UUID | None = Field(
default=None,
sa_column=Column(
PostgreSQLUUID(as_uuid=True),
primary_key=True,
server_default=text("gen_random_uuid()"),
),
)
job_id: UUID = Field(
sa_column=Column(
PostgreSQLUUID(as_uuid=True),
ForeignKey("job.id", ondelete="CASCADE"),
nullable=False,
),
)
source_id: UUID = Field(
sa_column=Column(
PostgreSQLUUID(as_uuid=True),
ForeignKey("source.id", ondelete="CASCADE"),
nullable=False,
),
)
status: JobSourceStatus = Field(
default=JobSourceStatus.PENDING,
sa_column=Column(
String(50),
nullable=False,
server_default=text("'pending'"),
),
)
raw_transcription: str | None = Field(default=None, sa_column=Column(Text))
ai_metadata: JsonValue | None = Field(
default=None,
sa_column=Column(JSONB),
)
raw_api_response: JsonValue | None = Field(
default=None,
sa_column=Column(JSONB),
)
error_detail: str | None = Field(default=None, sa_column=Column(Text))
executed_at: datetime | None = Field(
default=None,
sa_column=Column(
DateTime(timezone=True),
nullable=False,
server_default=text("now()"),
),
)
job: Job | None = Relationship(
back_populates="job_sources",
sa_relationship_kwargs={"lazy": "raise"},
)
source: Source | None = Relationship(
back_populates="job_sources",
sa_relationship_kwargs={"lazy": "raise"},
)
```
The enum annotations validate application values while the mapped columns retain the `VARCHAR` types specified by the DDL. PostgreSQL owns generated UUIDs and timestamps through `server_default`; call `session.refresh(instance)` after a flush or commit when those generated values are needed immediately.
`ai_metadata`, `raw_api_response`, and `metadata_` accept any JSON value supported by `JSONB`. Validate provider-specific payload structure before assigning it to these fields, while preserving the complete raw response in `raw_api_response`.
Relationships use `lazy="raise"` to prevent implicit database I/O in async code. Queries must explicitly load relationships they need, for example with `selectinload()`.
The schema's behavioral invariants are enforced outside the table shape where appropriate:
- `PersonRole`, `JobStatus`, and `JobSourceStatus` define the exact values listed by the schema.
- `unique_document_person_role` enforces role uniqueness for `(document_id, person_id, role)`.
- Services order document sources by `Source.document_id` and `Source.page_number`.
- Services derive aggregate `Job.status` from related `JobSource.status` values.
- Services preserve `JobSource.raw_transcription` and `JobSource.raw_api_response` as point-in-time outputs while updating the active text on `Source`.
+27
View File
@@ -0,0 +1,27 @@
import uvicorn
from fastapi import FastAPI
from .app import create_app
from .config import parse_cli_settings
def create_cli_app() -> FastAPI:
"""Create an app from CLI settings for Uvicorn's reload process."""
return create_app(settings=parse_cli_settings())
def main() -> None:
settings = parse_cli_settings()
application = "transcription.__main__:create_cli_app" if settings.reload else create_app(settings=settings)
uvicorn.run(
application,
factory=settings.reload,
host=settings.host,
port=settings.port,
log_level=settings.log_level,
reload=settings.reload,
)
if __name__ == "__main__":
main()
+18 -9
View File
@@ -16,11 +16,14 @@ from fastapi.staticfiles import StaticFiles
from .api.errors import register_error_handlers from .api.errors import register_error_handlers
from .api.health import router as health_router from .api.health import router as health_router
from .config import Settings
from .config import configure_logging from .config import configure_logging
from .config import get_settings from .config import get_settings
from .db import create_all from .db import create_all
from .db import dispose_database_runtime
from .db import initialize_database_runtime from .db import initialize_database_runtime
from .db.engine import get_database_url
from .db.engine import resolve_engine
from .db.session import dispose_session_factory
from .services import ServiceBundle from .services import ServiceBundle
from .services.jobs import JobService from .services.jobs import JobService
from .ui import register_pages from .ui import register_pages
@@ -31,15 +34,14 @@ logger = logging.getLogger(__name__)
@asynccontextmanager @asynccontextmanager
async def _lifespan(app: FastAPI): async def _lifespan(app: FastAPI):
configure_logging()
settings = getattr(app.state, "settings", None) or get_settings() settings = getattr(app.state, "settings", None) or get_settings()
configure_logging(settings)
app.state.settings = settings app.state.settings = settings
app.state.services = ServiceBundle() app.state.services = ServiceBundle()
app.state.runtime = initialize_database_runtime(settings=settings) app.state.runtime = initialize_database_runtime(settings=settings)
if settings.should_bootstrap_schema: if settings.should_bootstrap_schema:
await create_all(engine=app.state.runtime.engine) await create_all(engine=resolve_engine(settings=settings))
settings.upload_dir.mkdir(parents=True, exist_ok=True) settings.upload_dir.mkdir(parents=True, exist_ok=True)
settings.prompt_dir.mkdir(parents=True, exist_ok=True) settings.prompt_dir.mkdir(parents=True, exist_ok=True)
@@ -47,7 +49,10 @@ async def _lifespan(app: FastAPI):
await _recover_stale_processing_jobs(app) await _recover_stale_processing_jobs(app)
async with AsyncExitStack() as stack: async with AsyncExitStack() as stack:
stack.push_async_callback(dispose_database_runtime) stack.push_async_callback(
dispose_session_factory,
database_url=get_database_url(settings),
)
stop_event, worker_notifier = await stack.enter_async_context( stop_event, worker_notifier = await stack.enter_async_context(
worker_consumer_lifespan( worker_consumer_lifespan(
session_factory=app.state.runtime.session_factory, session_factory=app.state.runtime.session_factory,
@@ -73,14 +78,14 @@ async def _recover_stale_processing_jobs(app: FastAPI) -> None:
logger.warning("Recovered %s stale processing job(s) at startup", recovered) logger.warning("Recovered %s stale processing job(s) at startup", recovered)
def create_app() -> FastAPI: def create_app(settings: Settings | None = None) -> FastAPI:
"""Create and configure the FastAPI application.""" """Create and configure the FastAPI application."""
app = FastAPI(title="Transcription", lifespan=_lifespan) app = FastAPI(title="Transcription", lifespan=_lifespan)
settings = get_settings() active_settings = settings or get_settings()
app.state.settings = settings app.state.settings = active_settings
app.mount( app.mount(
"/uploads", "/uploads",
StaticFiles(directory=settings.upload_dir, check_dir=False), StaticFiles(directory=active_settings.upload_dir, check_dir=False),
name="uploads", name="uploads",
) )
@@ -92,6 +97,10 @@ def create_app() -> FastAPI:
async def ui_redirect() -> RedirectResponse: async def ui_redirect() -> RedirectResponse:
return RedirectResponse(url="/ui/upload", status_code=status.HTTP_307_TEMPORARY_REDIRECT) return RedirectResponse(url="/ui/upload", status_code=status.HTTP_307_TEMPORARY_REDIRECT)
@app.get("/healthz")
def health() -> dict[str, str]:
return {"status": "ok"}
register_error_handlers(app) register_error_handlers(app)
register_pages(app) register_pages(app)
app.include_router(health_router) app.include_router(health_router)
+1 -1
View File
@@ -7,7 +7,7 @@ from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
from transcription.db.runtime import DatabaseRuntime from transcription.db.runtime import DatabaseRuntime
from transcription.db.runtime import get_session_factory from transcription.db.session import get_session_factory
from transcription.worker import WorkerNotifier from transcription.worker import WorkerNotifier
from transcription.worker import resolve_worker_notifier from transcription.worker import resolve_worker_notifier
+52 -14
View File
@@ -6,12 +6,17 @@ are resolved by the provider adapters, not here.
""" """
import logging.config import logging.config
from contextvars import ContextVar from collections.abc import Sequence
from enum import StrEnum from enum import StrEnum
from functools import cache
from pathlib import Path from pathlib import Path
from typing import Annotated
from typing import Any
from typing import Literal from typing import Literal
from pydantic import BaseModel
from pydantic import Field from pydantic import Field
from pydantic import SecretStr
from pydantic_settings import BaseSettings from pydantic_settings import BaseSettings
from pydantic_settings import SettingsConfigDict from pydantic_settings import SettingsConfigDict
@@ -22,13 +27,42 @@ class Provider(StrEnum):
OPENROUTER = "openrouter" OPENROUTER = "openrouter"
class SqliteSettings(BaseModel):
driver: Literal["sqlite"] = "sqlite"
path: str = "app.db"
class PostgresSettings(BaseModel):
driver: Literal["postgres"] = "postgres"
host: str
port: int = 5432
database: str
user: str
password: SecretStr
DatabaseSettings = Annotated[
SqliteSettings | PostgresSettings,
Field(discriminator="driver"),
]
class Settings(BaseSettings): class Settings(BaseSettings):
model_config = SettingsConfigDict( model_config = SettingsConfigDict(
env_file=".env", env_file=".env",
env_file_encoding="utf-8", env_file_encoding="utf-8",
extra="ignore", extra="ignore",
env_nested_delimiter="__",
cli_implicit_flags=True,
cli_kebab_case=True,
) )
# --- NiceGUI Server ---
host: str = "0.0.0.0"
port: int = 8000
log_level: Literal["critical", "error", "warning", "info", "debug", "trace"] = "info"
reload: bool = False
# --- AI provider --- # --- AI provider ---
provider: Provider = Provider.OPENROUTER provider: Provider = Provider.OPENROUTER
openrouter_api_key: str openrouter_api_key: str
@@ -40,8 +74,8 @@ class Settings(BaseSettings):
environment: Literal["development", "test", "production"] = "development" environment: Literal["development", "test", "production"] = "development"
# --- persistence --- # --- persistence ---
database_url: str = "sqlite:///./transcription.db" database: DatabaseSettings = Field(default_factory=SqliteSettings)
bootstrap_schema_on_startup: bool | None = None bootstrap_schema_on_startup: bool = False
sqlite_check_same_thread: bool = False sqlite_check_same_thread: bool = False
# --- filesystem paths --- # --- filesystem paths ---
@@ -59,23 +93,24 @@ class Settings(BaseSettings):
@property @property
def should_bootstrap_schema(self) -> bool: def should_bootstrap_schema(self) -> bool:
"""Return whether startup should auto-create schema for this environment.""" """Return whether startup should auto-create schema for this environment."""
if self.bootstrap_schema_on_startup is not None: if "bootstrap_schema_on_startup" in self.model_fields_set:
return self.bootstrap_schema_on_startup return self.bootstrap_schema_on_startup
return self.environment in {"development", "test"} return self.environment in {"development", "test"}
_settings: ContextVar[Settings | None] = ContextVar("settings", default=None) @cache
def get_settings(**kwargs: Any) -> Settings:
"""Load cached settings without reading process CLI arguments."""
return Settings(_cli_parse_args=False, **kwargs) # pyright: ignore[reportCallIssue]
def get_settings(**kwargs) -> Settings: def parse_cli_settings(args: Sequence[str] | None = None) -> Settings:
settings = _settings.get() """Load settings with CLI arguments at the executable boundary."""
if settings is None: cli_args = True if args is None else list(args)
settings = Settings(**kwargs) # pyright: ignore[reportCallIssue] return Settings(_cli_parse_args=cli_args) # pyright: ignore[reportCallIssue]
_settings.set(settings)
return settings
LOGGING_CONFIG: dict[str, object] = { LOGGING_CONFIG: dict[str, Any] = {
"version": 1, "version": 1,
"disable_existing_loggers": False, "disable_existing_loggers": False,
"formatters": { "formatters": {
@@ -105,7 +140,10 @@ LOGGING_CONFIG: dict[str, object] = {
} }
def configure_logging() -> None: def configure_logging(settings: Settings | None = None) -> None:
"""Configure root logging once at startup.""" """Configure root logging once at startup."""
logging.config.dictConfig(LOGGING_CONFIG) cfg = LOGGING_CONFIG.copy()
active_settings = settings or get_settings()
cfg["loggers"]["transcription"]["level"] = active_settings.log_level.upper()
logging.config.dictConfig(cfg)
logger.debug("Logging configured") logger.debug("Logging configured")
+9 -2
View File
@@ -1,6 +1,13 @@
from .operations import create_all from .operations import create_all
from .runtime import dispose_database_runtime from .runtime import dispose_database_runtime
from .runtime import get_session
from .runtime import initialize_database_runtime from .runtime import initialize_database_runtime
from .session import session_scope
from .session import transaction_scope
__all__ = ["create_all", "dispose_database_runtime", "get_session", "initialize_database_runtime"] __all__ = [
"create_all",
"dispose_database_runtime",
"initialize_database_runtime",
"session_scope",
"transaction_scope",
]
+60
View File
@@ -0,0 +1,60 @@
from functools import cache
from typing import Any
from sqlalchemy import URL
from sqlalchemy import StaticPool
from sqlalchemy.ext.asyncio import AsyncEngine
from sqlalchemy.ext.asyncio import create_async_engine
from ..config import PostgresSettings
from ..config import Settings
from ..config import SqliteSettings
from ..config import get_settings
def get_database_url(settings: Settings) -> str:
match settings.database:
case SqliteSettings(path=path):
url = URL.create(
drivername="sqlite+aiosqlite",
database=path,
)
case PostgresSettings() as database:
url = URL.create(
drivername="postgresql+asyncpg",
host=database.host,
port=database.port,
database=database.database,
username=database.user,
password=database.password.get_secret_value(),
)
return url.render_as_string(hide_password=False)
def resolve_engine(settings: Settings | None = None) -> AsyncEngine:
active_settings = settings or get_settings()
return get_engine(get_database_url(active_settings))
@cache
def get_engine(database_url: str) -> AsyncEngine:
kwargs: dict[str, Any] = {"echo": False, "pool_pre_ping": True}
if database_url.startswith("sqlite"):
kwargs["connect_args"] = {"check_same_thread": False}
if ":memory:" in database_url:
kwargs["poolclass"] = StaticPool
return create_async_engine(database_url, **kwargs)
async def dispose_engine(database_url: str) -> None:
engine = get_engine(database_url)
try:
await engine.dispose()
finally:
get_engine.cache_clear()
async def refresh_engine(database_url: str) -> AsyncEngine:
await dispose_engine(database_url)
return get_engine(database_url)
+17 -21
View File
@@ -10,13 +10,25 @@ from sqlmodel import SQLModel
from sqlmodel import select from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
from ..models import Job from .engine import resolve_engine
from ..models import JobStatus from .models import Job
from .runtime import get_engine from .models import JobStatus
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
async def create_all(*, engine: AsyncEngine | None = None) -> None:
"""Create all tables on the selected engine."""
# Import models so SQLModel metadata is fully registered before bootstrap.
from transcription.db import models as _models # noqa: F401
active_engine = engine or resolve_engine()
async with active_engine.begin() as connection:
await connection.run_sync(SQLModel.metadata.create_all)
await connection.run_sync(_ensure_sqlite_compat_columns)
logger.debug("Database schema bootstrap complete for database_url=%s", active_engine.url)
async def get_next_queued_job(*, session: AsyncSession) -> Job | None: async def get_next_queued_job(*, session: AsyncSession) -> Job | None:
"""Get the next queued job, if any.""" """Get the next queued job, if any."""
result = await session.exec( result = await session.exec(
@@ -28,18 +40,6 @@ async def get_next_queued_job(*, session: AsyncSession) -> Job | None:
return result.first() return result.first()
async def create_all(*, engine: AsyncEngine | None = None) -> None:
"""Create all tables on the selected engine."""
# Import models so SQLModel metadata is fully registered before bootstrap.
from transcription import models as _models # noqa: F401
active_engine = engine or get_engine()
async with active_engine.begin() as connection:
await connection.run_sync(SQLModel.metadata.create_all)
await connection.run_sync(_ensure_sqlite_compat_columns)
logger.debug("Database schema bootstrap complete for database_url=%s", active_engine.url)
def _ensure_sqlite_compat_columns(connection: Connection) -> None: def _ensure_sqlite_compat_columns(connection: Connection) -> None:
"""Apply lightweight dev/test SQLite compatibility column patches. """Apply lightweight dev/test SQLite compatibility column patches.
@@ -68,12 +68,8 @@ def _ensure_sqlite_compat_columns(connection: Connection) -> None:
break break
if not has_unique_source: if not has_unique_source:
connection.execute( connection.execute(
text( text("CREATE UNIQUE INDEX IF NOT EXISTS ux_revision_source_id ON revision(source_id)")
"CREATE UNIQUE INDEX IF NOT EXISTS "
"ux_revision_source_id ON revision(source_id)"
)
) )
logger.warning( logger.warning(
"Applied SQLite compatibility schema patch " "Applied SQLite compatibility schema patch table=revision unique_index=ux_revision_source_id"
"table=revision unique_index=ux_revision_source_id"
) )
+6 -58
View File
@@ -1,18 +1,16 @@
import logging import logging
from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager
from contextvars import ContextVar from contextvars import ContextVar
from dataclasses import dataclass from dataclasses import dataclass
from functools import partial
from sqlalchemy.ext.asyncio import AsyncEngine from sqlalchemy.ext.asyncio import AsyncEngine
from sqlalchemy.ext.asyncio import async_sessionmaker from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlalchemy.ext.asyncio import create_async_engine
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
from sqlmodel.pool import StaticPool
from ..config import Settings from ..config import Settings
from ..config import get_settings from ..config import get_settings
from .engine import get_database_url
from .engine import get_engine
from .session import get_session_factory
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -37,33 +35,6 @@ async def dispose_database_runtime() -> None:
_runtime.set(None) _runtime.set(None)
def _to_async_database_url(database_url: str) -> str:
"""Normalize configured database URL to an async SQLAlchemy driver URL."""
if database_url.startswith("sqlite://") and not database_url.startswith("sqlite+aiosqlite://"):
return database_url.replace("sqlite://", "sqlite+aiosqlite://", 1)
if database_url.startswith("postgresql://") and not database_url.startswith("postgresql+asyncpg://"):
return database_url.replace("postgresql://", "postgresql+asyncpg://", 1)
return database_url
def _build_engine(settings: Settings) -> AsyncEngine:
database_url = _to_async_database_url(settings.database_url)
engine_factory = partial(
create_async_engine,
url=database_url,
echo=False,
pool_pre_ping=True,
)
if database_url.startswith("sqlite"):
sqlite_connect_settings = {"check_same_thread": settings.sqlite_check_same_thread}
engine_factory = partial(engine_factory, connect_args=sqlite_connect_settings)
if ":memory:" in database_url:
engine_factory = partial(engine_factory, poolclass=StaticPool)
return engine_factory()
def initialize_database_runtime(*, settings: Settings | None = None) -> DatabaseRuntime: def initialize_database_runtime(*, settings: Settings | None = None) -> DatabaseRuntime:
"""Initialize lifespan-owned async DB resources once per process.""" """Initialize lifespan-owned async DB resources once per process."""
runtime = _runtime.get() runtime = _runtime.get()
@@ -71,33 +42,10 @@ def initialize_database_runtime(*, settings: Settings | None = None) -> Database
return runtime return runtime
active_settings = settings or get_settings() active_settings = settings or get_settings()
engine = _build_engine(active_settings) database_url = get_database_url(active_settings)
session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False) engine = get_engine(database_url)
session_factory = get_session_factory(database_url)
runtime = DatabaseRuntime(engine=engine, session_factory=session_factory) runtime = DatabaseRuntime(engine=engine, session_factory=session_factory)
_runtime.set(runtime) _runtime.set(runtime)
logger.debug("Initialized async database runtime for database_url=%s", engine.url) logger.debug("Initialized async database runtime for database_url=%s", engine.url)
return runtime return runtime
def get_engine(settings: Settings | None = None) -> AsyncEngine:
"""Return the current async SQLAlchemy engine."""
runtime = _runtime.get() or initialize_database_runtime(settings=settings)
return runtime.engine
def get_session_factory(settings: Settings | None = None) -> async_sessionmaker[AsyncSession]:
"""Return the shared async session factory."""
runtime = _runtime.get() or initialize_database_runtime(settings=settings)
return runtime.session_factory
@asynccontextmanager
async def get_session(
*,
settings: Settings | None = None,
session_factory: async_sessionmaker[AsyncSession] | None = None,
) -> AsyncGenerator[AsyncSession]:
"""Yield a database session and ensure cleanup."""
active_session_factory = session_factory or get_session_factory(settings)
async with active_session_factory() as session:
yield session
+99
View File
@@ -0,0 +1,99 @@
from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager
from functools import cache
from typing import Annotated
from fastapi import Depends
from sqlalchemy.ext.asyncio import AsyncSessionTransaction
from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlmodel.ext.asyncio.session import AsyncSession
from ..config import Settings
from ..config import get_settings
from .engine import dispose_engine
from .engine import get_database_url
from .engine import get_engine
type SessionFactory = async_sessionmaker[AsyncSession]
@cache
def get_session_factory(database_url: str) -> SessionFactory:
return async_sessionmaker(
bind=get_engine(database_url),
class_=AsyncSession,
expire_on_commit=False,
)
def resolve_session_factory(
database_url: str | None = None,
*,
settings: Settings | None = None,
) -> SessionFactory:
if database_url is not None:
return get_session_factory(database_url)
return get_session_factory(get_database_url(settings or get_settings()))
type SessionFactoryDep = Annotated[SessionFactory, Depends(resolve_session_factory)]
async def dispose_session_factory(database_url: str) -> None:
get_session_factory.cache_clear()
await dispose_engine(database_url)
@asynccontextmanager
async def session_scope(
*,
settings: Settings | None = None,
database_url: str | None = None,
session_factory: SessionFactory | None = None,
session: AsyncSession | None = None,
) -> AsyncGenerator[AsyncSession]:
if session is not None:
yield session
return
active_session_factory = session_factory or resolve_session_factory(
database_url,
settings=settings,
)
async with active_session_factory() as owned_session:
yield owned_session
type SessionScopeDep = Annotated[AsyncSession, Depends(session_scope)]
@asynccontextmanager
async def transaction_scope(
*,
settings: Settings | None = None,
database_url: str | None = None,
session_factory: SessionFactory | None = None,
session: AsyncSession | AsyncSessionTransaction | None = None,
) -> AsyncGenerator[AsyncSession | AsyncSessionTransaction]:
match session:
case AsyncSession() as async_session:
if not async_session.in_transaction():
raise RuntimeError("A supplied session must have an active transaction")
yield async_session
return
case AsyncSessionTransaction() as async_transaction:
yield async_transaction
return
active_session_factory = session_factory or resolve_session_factory(
database_url,
settings=settings,
)
async with active_session_factory.begin() as owned_session:
yield owned_session
type TransactionScopeDep = Annotated[
AsyncSession | AsyncSessionTransaction,
Depends(transaction_scope),
]
+8 -9
View File
@@ -8,7 +8,8 @@ from sqlmodel.ext.asyncio.session import AsyncSession
from ..config import Settings from ..config import Settings
from ..config import get_settings from ..config import get_settings
from ..db.runtime import get_session_factory from ..db.session import resolve_session_factory
from ..db.session import session_scope
class ServiceBase(ABC): class ServiceBase(ABC):
@@ -24,19 +25,17 @@ class ServiceBase(ABC):
queue: asyncio.Queue | None = None, queue: asyncio.Queue | None = None,
): ):
self.settings = get_settings() self.settings = get_settings()
self.session_factory = session_factory or get_session_factory() self.session_factory = session_factory or resolve_session_factory()
self.queue = queue or asyncio.Queue() self.queue = queue or asyncio.Queue()
@asynccontextmanager @asynccontextmanager
async def _session_scope(self, session: AsyncSession | None = None): async def _session_scope(self, session: AsyncSession | None = None):
"""Provide a transactional scope around a series of operations.""" """Provide a transactional scope around a series of operations."""
if session is not None: async with session_scope(
# Reuse the provided session if one is passed in session_factory=self.session_factory,
yield session session=session,
else: ) as active_session:
# Otherwise, create a new session for this scope yield active_session
async with self.session_factory() as new_session:
yield new_session
async def _finalize( async def _finalize(
self, self,
+1 -1
View File
@@ -9,9 +9,9 @@ from sqlalchemy.orm import selectinload
from sqlmodel import select from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
from ..db.models import Document
from ..errors import AppError from ..errors import AppError
from ..errors import ErrorCategory from ..errors import ErrorCategory
from ..models import Document
from .base import ServiceBase from .base import ServiceBase
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
+4 -8
View File
@@ -7,9 +7,9 @@ from sqlalchemy.orm import selectinload
from sqlmodel import select from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
from ..models import Job from ..db.models import Job
from ..models import JobStatus from ..db.models import JobStatus
from ..models import Source from ..db.models import Source
from .base import ServiceBase from .base import ServiceBase
@@ -170,11 +170,7 @@ class JobService(ServiceBase):
``stale_before`` are considered stale and re-queued. ``stale_before`` are considered stale and re-queued.
""" """
async with self._session_scope(session) as _session: async with self._session_scope(session) as _session:
query = ( query = select(Job).where(Job.status == JobStatus.PROCESSING).where(Job.date_updated < stale_before)
select(Job)
.where(Job.status == JobStatus.PROCESSING)
.where(Job.date_updated < stale_before)
)
stale_jobs = (await _session.exec(query)).all() stale_jobs = (await _session.exec(query)).all()
if not stale_jobs: if not stale_jobs:
return 0 return 0
+3 -3
View File
@@ -11,9 +11,9 @@ from transcription.config import get_settings
from transcription.errors import AppError from transcription.errors import AppError
from transcription.errors import ErrorCategory from transcription.errors import ErrorCategory
from ..models import Document from ..db.models import Document
from ..models import Job from ..db.models import Job
from ..models import Source from ..db.models import Source
from .documents import UploadJobResult from .documents import UploadJobResult
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
+3 -3
View File
@@ -18,11 +18,11 @@ from sqlmodel.ext.asyncio.session import AsyncSession
from transcription.config import Settings from transcription.config import Settings
from transcription.config import get_settings from transcription.config import get_settings
from transcription.db.models import Job
from transcription.db.models import Revision
from transcription.db.models import Source
from transcription.errors import AppError from transcription.errors import AppError
from transcription.errors import ErrorCategory from transcription.errors import ErrorCategory
from transcription.models import Job
from transcription.models import Revision
from transcription.models import Source
from transcription.providers import ProviderAuthError from transcription.providers import ProviderAuthError
from transcription.providers import ProviderError from transcription.providers import ProviderError
from transcription.providers import ProviderResponseError from transcription.providers import ProviderResponseError
+3 -3
View File
@@ -5,13 +5,13 @@ from sqlmodel.ext.asyncio.session import AsyncSession
from ..config import Settings from ..config import Settings
from ..config import get_settings from ..config import get_settings
from ..db.models import Job
from ..db.models import JobStatus
from ..db.models import Source
from ..errors import AppError from ..errors import AppError
from ..errors import ErrorCategory from ..errors import ErrorCategory
from ..errors import classify_unexpected_error from ..errors import classify_unexpected_error
from ..errors import format_error_detail from ..errors import format_error_detail
from ..models import Job
from ..models import JobStatus
from ..models import Source
from ..providers import TranscriptionResult from ..providers import TranscriptionResult
from . import ServiceBundle from . import ServiceBundle
from .transcription import DEFAULT_PROMPT_FILE from .transcription import DEFAULT_PROMPT_FILE
+3 -21
View File
@@ -1,38 +1,20 @@
"""UI page registration exports.""" """UI page registration exports."""
from pathlib import Path
from fastapi import FastAPI from fastapi import FastAPI
from nicegui import app as nicegui_app
from nicegui import ui from nicegui import ui
from transcription.ui.pages.jobs_page import register_page as register_jobs_page from transcription.ui.pages.jobs_page import register_page as register_jobs_page
from transcription.ui.pages.upload_page import register_page as register_upload_page from transcription.ui.pages.upload_page import register_page as register_upload_page
from transcription.ui.resources import read_css
_THEME_REGISTERED_STATE_KEY = "transcription_ui_theme_registered" _THEME_REGISTERED_STATE_KEY = "transcription_ui_theme_registered"
_THEME_COLORS: dict[str, str] = {
"primary": "#6f97e8",
"secondary": "#92b5f5",
"accent": "#7fc0de",
"dark": "#22304a",
"dark_page": "#1a2538",
"positive": "#86c8ad",
"negative": "#d98a9a",
"info": "#7ebdda",
"warning": "#e2c083",
}
def _register_global_styles(app: FastAPI) -> None: def _register_global_styles(app: FastAPI) -> None:
if getattr(app.state, _THEME_REGISTERED_STATE_KEY, False): if getattr(app.state, _THEME_REGISTERED_STATE_KEY, False):
return return
nicegui_app.colors(**_THEME_COLORS) ui.add_css(read_css("theme.css"), shared=True)
css_path = Path(__file__).resolve().parent / "static" / "colors.css"
if css_path.exists():
ui.add_css(css_path, shared=True)
setattr(app.state, _THEME_REGISTERED_STATE_KEY, True) setattr(app.state, _THEME_REGISTERED_STATE_KEY, True)
@@ -42,4 +24,4 @@ def register_pages(app: FastAPI) -> None:
_register_global_styles(app) _register_global_styles(app)
register_upload_page() register_upload_page()
register_jobs_page() register_jobs_page()
ui.run_with(app, mount_path="/ui", show_welcome_message=False, dark=True) ui.run_with(app, mount_path="/ui", show_welcome_message=False, dark=False)
+11 -1
View File
@@ -1,7 +1,17 @@
"""Reusable UI component exports.""" """Reusable UI component exports."""
from transcription.ui.components.app_shell import NAV_ITEMS from transcription.ui.components.app_shell import NAV_ITEMS
from transcription.ui.components.app_shell import render_app_shell
from transcription.ui.components.app_shell import render_navigation_header from transcription.ui.components.app_shell import render_navigation_header
from transcription.ui.components.document_panzoom import render_document_panzoom from transcription.ui.components.document_panzoom import render_document_panzoom
from transcription.ui.components.page_content import render_page_content
from transcription.ui.components.page_header import render_page_header
__all__ = ["NAV_ITEMS", "render_document_panzoom", "render_navigation_header"] __all__ = [
"NAV_ITEMS",
"render_app_shell",
"render_document_panzoom",
"render_navigation_header",
"render_page_content",
"render_page_header",
]
+26 -23
View File
@@ -4,6 +4,8 @@ from __future__ import annotations
from nicegui import ui from nicegui import ui
from transcription.ui.resources import read_css
NAV_ITEMS: tuple[tuple[str, str, str], ...] = ( NAV_ITEMS: tuple[tuple[str, str, str], ...] = (
("Upload", "/upload", "upload_file"), ("Upload", "/upload", "upload_file"),
("Jobs", "/jobs", "work_history"), ("Jobs", "/jobs", "work_history"),
@@ -16,27 +18,17 @@ def _is_active_path(*, current_path: str, item_path: str) -> bool:
return current_path == item_path return current_path == item_path
def _button_props(*, icon: str, is_active: bool) -> str:
if is_active:
return f"icon={icon} no-caps unelevated color=primary text-color=white"
return f"icon={icon} no-caps outline color=secondary text-color=secondary"
def _button_classes(*, is_active: bool) -> str:
base = "w-full sm:w-auto min-h-[40px] px-3 rounded-lg text-body2 text-weight-medium"
if is_active:
return f"{base}"
return f"{base}"
def _render_nav_button(*, label: str, path: str, icon: str, current_path: str) -> None: def _render_nav_button(*, label: str, path: str, icon: str, current_path: str) -> None:
is_active = _is_active_path(current_path=current_path, item_path=path) is_active = _is_active_path(current_path=current_path, item_path=path)
button = ui.button( classes = "app-shell__nav-item"
if is_active:
classes = f"{classes} app-shell__nav-item--active"
ui.button(
label, label,
icon=icon, icon=icon,
on_click=lambda _=None, route=path: ui.navigate.to(route), on_click=lambda _=None, route=path: ui.navigate.to(route),
) ).props("flat no-caps").classes(classes)
button.props(_button_props(icon=icon, is_active=is_active)).classes(_button_classes(is_active=is_active))
def _normalize_path(current_path: str | None) -> str: def _normalize_path(current_path: str | None) -> str:
@@ -46,14 +38,25 @@ def _normalize_path(current_path: str | None) -> str:
return normalized.rstrip("/") or "/" return normalized.rstrip("/") or "/"
def render_navigation_header(*, current_path: str | None = None) -> None: def render_app_shell(*, current_path: str | None = None) -> None:
"""Render a shared app header with links for top-level pages.""" """Render the shared application shell header."""
ui.add_css(read_css("components/app_shell.css"))
normalized_path = _normalize_path(current_path) normalized_path = _normalize_path(current_path)
with ( with ui.header().classes("app-shell"), ui.element("div").classes("app-shell__inner"):
ui.header(elevated=True).props("bordered").classes("bg-dark text-white q-px-sm q-py-xs"), with ui.row().classes("app-shell__brand no-wrap"):
ui.row().classes("w-full items-center justify-end q-gutter-xs"), ui.label("VS").classes("app-shell__brand-mark")
ui.element("div").classes("w-full grid grid-cols-2 gap-2 sm:flex sm:justify-start sm:gap-2"), ui.label("VibeScribe").classes("app-shell__brand-name")
):
with ui.element("nav").props('aria-label="Primary navigation"').classes("app-shell__nav"):
for label, path, icon in NAV_ITEMS: for label, path, icon in NAV_ITEMS:
_render_nav_button(label=label, path=path, icon=icon, current_path=normalized_path) _render_nav_button(label=label, path=path, icon=icon, current_path=normalized_path)
with ui.row().classes("app-shell__actions no-wrap"):
ui.label("Saved").classes("app-shell__save-state")
ui.button(icon="more_horiz").props("flat round dense").tooltip("More actions")
def render_navigation_header(*, current_path: str | None = None) -> None:
"""Render the app shell using the legacy page-level entry point."""
render_app_shell(current_path=current_path)
@@ -10,7 +10,7 @@ from uuid import uuid4
from nicegui import ui from nicegui import ui
from transcription.config import get_settings from transcription.config import get_settings
from transcription.models import Source from transcription.db.models import Source
PANGOZOOM_CDN_URL = "https://unpkg.com/@panzoom/[email protected]/dist/panzoom.min.js" PANGOZOOM_CDN_URL = "https://unpkg.com/@panzoom/[email protected]/dist/panzoom.min.js"
UPLOADS_URL_PREFIX = "/uploads" UPLOADS_URL_PREFIX = "/uploads"
@@ -24,10 +24,10 @@ def render_document_panzoom(*, source: Source) -> None:
document_url = _document_url(source) document_url = _document_url(source)
document_kind = _document_kind(source) document_kind = _document_kind(source)
with ui.card().classes("w-full q-pa-md"): with ui.card().classes("w-full q-pa-md vibe-card"):
with ui.row().classes("w-full items-center justify-between no-wrap"): with ui.row().classes("w-full items-center justify-between no-wrap"):
ui.label("Document preview").classes("text-subtitle1 text-weight-medium") ui.label("Document preview").classes("text-subtitle1 text-weight-medium")
ui.label(source.filename).classes("text-caption text-grey-4 ellipsis").style( ui.label(source.filename).classes("text-caption vibe-text-muted ellipsis").style(
"max-width: 60%; text-align: right;" "max-width: 60%; text-align: right;"
) )
@@ -90,7 +90,7 @@ def _register_panzoom_assets() -> None:
height: 100%; height: 100%;
border: 0; border: 0;
pointer-events: none; pointer-events: none;
background: white; background: var(--theme-surface-raised);
} }
</style> </style>
""", """,
@@ -26,7 +26,7 @@ def show_error(exc: Exception, *, title: str, operation: str) -> None:
close_button="Dismiss", close_button="Dismiss",
) )
with ui.card().classes("bg-red-1 text-red-10 q-mt-md q-pa-md"): with ui.card().classes("vibe-card--error q-mt-md q-pa-md"):
ui.label(title).classes("text-subtitle1") ui.label(title).classes("text-subtitle1")
ui.label(error.message) ui.label(error.message)
ui.label(f"Suggested action: {error.suggestion}").classes("text-weight-medium") ui.label(f"Suggested action: {error.suggestion}").classes("text-weight-medium")
+17 -17
View File
@@ -6,9 +6,9 @@ import logging
from nicegui import ui from nicegui import ui
from transcription.models import Job from transcription.db.models import Job
from transcription.models import Revision from transcription.db.models import Revision
from transcription.models import Source from transcription.db.models import Source
from transcription.ui.components.document_panzoom import render_document_panzoom from transcription.ui.components.document_panzoom import render_document_panzoom
from transcription.ui.components.transcript import render_original_transcription_card from transcription.ui.components.transcript import render_original_transcription_card
from transcription.ui.components.transcript import render_revision_row from transcription.ui.components.transcript import render_revision_row
@@ -18,24 +18,24 @@ logger = logging.getLogger(__name__)
def _status_chip_classes(status: str) -> str: def _status_chip_classes(status: str) -> str:
if status == "queued": if status == "queued":
return "bg-blue-1 text-blue-10" return "vibe-status--queued"
if status == "processing": if status == "processing":
return "bg-amber-1 text-amber-10" return "vibe-status--processing"
if status == "transcribed": if status == "transcribed":
return "bg-green-1 text-green-10" return "vibe-status--transcribed"
if status == "failed": if status == "failed":
return "bg-red-1 text-red-10" return "vibe-status--failed"
return "bg-grey-2 text-grey-9" return "vibe-status--default"
def _metadata_row(label: str, value: str) -> None: def _metadata_row(label: str, value: str) -> None:
with ui.row().classes("w-full items-start justify-between q-gutter-x-md"): with ui.row().classes("w-full items-start justify-between q-gutter-x-md"):
ui.label(label).classes("text-caption text-grey-5 text-uppercase w-28") ui.label(label).classes("text-caption vibe-text-muted text-uppercase w-28")
ui.label(value).classes("text-body2 text-right text-grey-1 break-all") ui.label(value).classes("text-body2 text-right break-all")
def _render_source_section(source: Source) -> None: def _render_source_section(source: Source) -> None:
with ui.card().classes("w-full q-pa-md"): with ui.card().classes("w-full q-pa-md vibe-card"):
ui.label("Source").classes("text-subtitle1 text-weight-medium") ui.label("Source").classes("text-subtitle1 text-weight-medium")
ui.separator().classes("q-my-sm") ui.separator().classes("q-my-sm")
with ui.column().classes("w-full q-gutter-y-xs"): with ui.column().classes("w-full q-gutter-y-xs"):
@@ -49,12 +49,12 @@ def _render_source_section(source: Source) -> None:
def _render_revision_section(revision: Revision | None) -> None: def _render_revision_section(revision: Revision | None) -> None:
with ui.card().classes("w-full q-pa-md"): with ui.card().classes("w-full q-pa-md vibe-card"):
ui.label("Revision").classes("text-subtitle1 text-weight-medium") ui.label("Revision").classes("text-subtitle1 text-weight-medium")
ui.separator().classes("q-my-sm") ui.separator().classes("q-my-sm")
if revision is None: if revision is None:
ui.label("No revision exists for this source.").classes("text-body2 text-grey-3") ui.label("No revision exists for this source.").classes("text-body2 vibe-text-muted")
return return
render_revision_row(revision=revision, initially_expanded=True) render_revision_row(revision=revision, initially_expanded=True)
@@ -65,19 +65,19 @@ def render_job_detail(*, job: Job, source: Source | None, revision: Revision | N
logger.debug("Rendering job detail for job ID %s", job.id) logger.debug("Rendering job detail for job ID %s", job.id)
status_text = job.status.value status_text = job.status.value
with ui.column().classes("w-full max-w-4xl q-gutter-md"): with ui.column().classes("w-full max-w-4xl q-gutter-md"):
with ui.card().classes("w-full q-pa-lg"): with ui.card().classes("w-full q-pa-lg vibe-card"):
with ui.row().classes("w-full items-center justify-between q-gutter-md"): with ui.row().classes("w-full items-center justify-between q-gutter-md"):
with ui.column().classes("q-gutter-none"): with ui.column().classes("q-gutter-none"):
ui.label("Job overview").classes("text-h6 text-weight-bold") ui.label("Job overview").classes("text-h6 text-weight-bold")
ui.label(str(job.id)).classes("text-caption text-grey-5") ui.label(str(job.id)).classes("text-caption vibe-text-muted")
status_chip_classes = ( status_chip_classes = (
"q-px-sm q-py-xs rounded-borders " "q-px-sm q-py-xs rounded-borders "
"text-weight-medium text-capitalize " "vibe-status text-weight-medium text-capitalize "
f"{_status_chip_classes(status_text)}" f"{_status_chip_classes(status_text)}"
) )
ui.label(status_text).classes(status_chip_classes) ui.label(status_text).classes(status_chip_classes)
ui.separator().classes("q-my-md bg-blue-grey-7") ui.separator().classes("q-my-md vibe-separator")
with ui.column().classes("w-full q-gutter-y-xs"): with ui.column().classes("w-full q-gutter-y-xs"):
_metadata_row("Created", job.date_created.isoformat()) _metadata_row("Created", job.date_created.isoformat())
_metadata_row("Updated", job.date_updated.isoformat()) _metadata_row("Updated", job.date_updated.isoformat())
@@ -0,0 +1,55 @@
"""High-level placeholder content for a transcription workspace."""
from __future__ import annotations
from nicegui import ui
from transcription.ui.resources import read_css
def render_page_content(
*,
source_name: str = "document-placeholder.jpg",
raw_text: str = "AI transcription output will appear here.",
revised_text: str = "Human revision text will appear here.",
) -> None:
"""Render the primary editor workspace and supporting context sidebar."""
ui.add_css(read_css("components/page_content.css"))
with ui.element("div").classes("page-content"):
with ui.element("section").classes("page-content__editor"):
with ui.row().classes("page-content__heading"):
with ui.column().classes("gap-0"):
ui.label("Active source").classes("page-content__kicker")
ui.label("Page transcription").classes("page-content__title")
ui.badge("Page 1 of 1").classes("page-content__badge")
with ui.element("div").classes("page-content__workspace"):
with ui.element("section").classes("source-placeholder"):
with ui.row().classes("source-placeholder__toolbar"):
ui.label(source_name)
ui.icon("image", size="1.25rem")
with ui.column().classes("source-placeholder__body"):
ui.icon("description", size="4rem")
ui.label("Source preview")
with ui.column().classes("transcription-placeholder"):
with ui.element("section").classes("transcription-placeholder__section"):
ui.label("AI raw output").classes("transcription-placeholder__title")
ui.label(raw_text).classes("transcription-placeholder__text")
with ui.element("section").classes("transcription-placeholder__section"):
ui.label("Human revision").classes("transcription-placeholder__title")
ui.textarea(value=revised_text).props("outlined autogrow").classes("w-full")
with ui.element("aside").props('aria-label="Document context"').classes("page-content__sidebar"):
with ui.element("section").classes("page-content__sidebar-section"):
ui.label("People").classes("page-content__sidebar-title")
ui.label("Author · Placeholder Person")
ui.label("Recipient · Placeholder Person")
with ui.element("section").classes("page-content__sidebar-section"):
ui.label("AI processing").classes("page-content__sidebar-title")
ui.badge("Completed", color="positive")
ui.label("Provider · Placeholder provider")
ui.label("Model · Placeholder model")
@@ -0,0 +1,36 @@
"""High-level page header for document-oriented views."""
from __future__ import annotations
from collections.abc import Awaitable
from collections.abc import Callable
from nicegui import ui
from transcription.ui.resources import read_css
type PageHeaderAction = Callable[[], Awaitable[None] | None]
def render_page_header(
*,
eyebrow: str = "Letter · Placeholder Collection",
title: str = "Untitled archival document",
metadata: tuple[str, ...] = ("Date unknown", "Location unknown"),
on_details: PageHeaderAction | None = None,
on_review: PageHeaderAction | None = None,
) -> None:
"""Render document identity, metadata, and page-level actions."""
ui.add_css(read_css("components/page_header.css"))
with ui.element("section").classes("page-header"):
with ui.column().classes("page-header__identity"):
ui.label(eyebrow).classes("page-header__eyebrow")
ui.label(title).classes("page-header__title")
with ui.row().classes("page-header__metadata"):
for value in metadata:
ui.label(value)
with ui.row().classes("page-header__actions"):
ui.button("Document details", icon="info", on_click=on_details).props("outline no-caps")
ui.button("Mark reviewed", icon="task_alt", on_click=on_review).props("unelevated no-caps")
+9 -11
View File
@@ -9,8 +9,8 @@ from typing import Any
from nicegui import ui from nicegui import ui
from transcription.models import Job from transcription.db.models import Job
from transcription.models import Revision from transcription.db.models import Revision
type RevisionAction = Callable[[Revision], Awaitable[None] | None] type RevisionAction = Callable[[Revision], Awaitable[None] | None]
@@ -23,10 +23,10 @@ def render_original_transcription_card(*, job: Job, classes: str = "w-full") ->
model = job.model or "unknown" model = job.model or "unknown"
caption = f"{provider} | {model} | {_format_created_at(job.date_updated)}" caption = f"{provider} | {model} | {_format_created_at(job.date_updated)}"
card = ui.card().classes(f"{classes} q-pa-md bg-blue-grey-10") card = ui.card().classes(f"{classes} q-pa-md vibe-card")
with card, ui.column().classes("w-full q-gutter-y-sm"): with card, ui.column().classes("w-full q-gutter-y-sm"):
ui.label(header).classes("text-subtitle1 text-weight-medium") ui.label(header).classes("text-subtitle1 text-weight-medium")
ui.label(caption).classes("text-caption text-grey-5") ui.label(caption).classes("text-caption vibe-text-muted")
_metadata_row(label="Prompt", value=job.prompt_name or "unknown") _metadata_row(label="Prompt", value=job.prompt_name or "unknown")
_metadata_row(label="Updated", value=_format_created_at(job.date_updated)) _metadata_row(label="Updated", value=_format_created_at(job.date_updated))
@@ -35,7 +35,7 @@ def render_original_transcription_card(*, job: Job, classes: str = "w-full") ->
ui.markdown(job.text) ui.markdown(job.text)
if job.error_detail: if job.error_detail:
with ui.card().classes("w-full bg-red-1 text-red-10 q-pa-sm"): with ui.card().classes("w-full vibe-card--error q-pa-sm"):
ui.label("Failure detail").classes("text-caption text-uppercase") ui.label("Failure detail").classes("text-caption text-uppercase")
ui.label(job.error_detail).classes("text-body2") ui.label(job.error_detail).classes("text-body2")
@@ -53,15 +53,13 @@ def render_revision_row(
header = "Revision | User-authored" header = "Revision | User-authored"
caption = _format_created_at(revision.date_created) caption = _format_created_at(revision.date_created)
expansion = ui.expansion(value=initially_expanded, group="group").classes( expansion = ui.expansion(value=initially_expanded, group="group").classes(f"{classes} rounded-borders vibe-card")
f"{classes} rounded-borders bg-blue-grey-10"
)
with expansion, ui.column().classes("w-full q-gutter-y-sm q-pa-sm"): with expansion, ui.column().classes("w-full q-gutter-y-sm q-pa-sm"):
with expansion.add_slot("header"), ui.row().classes("w-full items-start justify-between q-gutter-md"): with expansion.add_slot("header"), ui.row().classes("w-full items-start justify-between q-gutter-md"):
with ui.column().classes("q-gutter-none"): with ui.column().classes("q-gutter-none"):
ui.label(header).classes("text-subtitle1 text-weight-medium") ui.label(header).classes("text-subtitle1 text-weight-medium")
ui.label(caption).classes("text-caption text-grey-5") ui.label(caption).classes("text-caption vibe-text-muted")
if on_delete is not None: if on_delete is not None:
with ui.dialog() as delete_dialog, ui.card().classes("q-pa-md"): with ui.dialog() as delete_dialog, ui.card().classes("q-pa-md"):
@@ -102,5 +100,5 @@ def _format_created_at(value: datetime) -> str:
def _metadata_row(*, label: str, value: str) -> None: def _metadata_row(*, label: str, value: str) -> None:
with ui.row().classes("w-md items-start justify-between q-gutter-x-md"): with ui.row().classes("w-md items-start justify-between q-gutter-x-md"):
ui.label(label).classes("text-caption text-grey-5 text-uppercase") ui.label(label).classes("text-caption vibe-text-muted text-uppercase")
ui.label(value).classes("text-body2 text-right text-grey-1 break-all") ui.label(value).classes("text-body2 text-right break-all")
+10 -13
View File
@@ -4,19 +4,18 @@ from __future__ import annotations
from uuid import UUID from uuid import UUID
from fastapi import Request
from nicegui import ui from nicegui import ui
from transcription.app_state import resolve_session_factory from transcription.db.models import Job
from transcription.models import Job from transcription.db.models import JobStatus
from transcription.models import JobStatus from transcription.db.models import Source
from transcription.models import Source
from transcription.services.jobs import JobService from transcription.services.jobs import JobService
from transcription.services.transcription import TranscriptionService from transcription.services.transcription import TranscriptionService
from transcription.ui.components.app_shell import render_navigation_header from transcription.ui.components.app_shell import render_navigation_header
from transcription.ui.components.error_presenter import show_error from transcription.ui.components.error_presenter import show_error
from transcription.ui.components.table.jobs import render_jobs_table from transcription.ui.components.table.jobs import render_jobs_table
from ...db.session import SessionFactoryDep
from ..components.document_panzoom import render_document_panzoom from ..components.document_panzoom import render_document_panzoom
from ..components.table.jobs import JobTableRow from ..components.table.jobs import JobTableRow
from ..components.transcript import render_original_transcription_card from ..components.transcript import render_original_transcription_card
@@ -27,8 +26,7 @@ def register_page() -> None: # noqa: PLR0915
"""Register jobs list and detail routes.""" """Register jobs list and detail routes."""
@ui.page("/jobs") @ui.page("/jobs")
async def jobs_page(request: Request) -> None: async def jobs_page(session_factory: SessionFactoryDep) -> None:
session_factory = resolve_session_factory(request.app.state)
jobs_service = JobService(session_factory=session_factory) jobs_service = JobService(session_factory=session_factory)
render_navigation_header(current_path="/jobs") render_navigation_header(current_path="/jobs")
@@ -51,8 +49,7 @@ def register_page() -> None: # noqa: PLR0915
await render_table() await render_table()
@ui.page("/jobs/{job_id}") @ui.page("/jobs/{job_id}")
async def job_detail_page(job_id: str, request: Request) -> None: # noqa: PLR0915 async def job_detail_page(job_id: str, session_factory: SessionFactoryDep) -> None: # noqa: PLR0915
session_factory = resolve_session_factory(request.app.state)
jobs_service = JobService(session_factory=session_factory) jobs_service = JobService(session_factory=session_factory)
transcription_service = TranscriptionService(session_factory=session_factory) transcription_service = TranscriptionService(session_factory=session_factory)
render_navigation_header(current_path="/jobs") render_navigation_header(current_path="/jobs")
@@ -76,7 +73,7 @@ def register_page() -> None: # noqa: PLR0915
if source is not None: if source is not None:
render_document_panzoom(source=source) render_document_panzoom(source=source)
else: else:
ui.label("No source preview is available for this job.").classes("text-body2 text-grey-3") ui.label("No source preview is available for this job.").classes("text-body2 vibe-text-muted")
with splitter.after, ui.column(align_items="stretch").classes("w-full h-full p-4 gap-3"): with splitter.after, ui.column(align_items="stretch").classes("w-full h-full p-4 gap-3"):
with ui.row(): with ui.row():
ui.button(icon="arrow_back", on_click=ui.navigate.back) ui.button(icon="arrow_back", on_click=ui.navigate.back)
@@ -84,7 +81,7 @@ def register_page() -> None: # noqa: PLR0915
ui.label(f"{job.id}").classes("text-h6 text-weight-bold") ui.label(f"{job.id}").classes("text-h6 text-weight-bold")
match job.status: match job.status:
case JobStatus.TRANSCRIBED: case JobStatus.TRANSCRIBED:
ui.chip(job.status.value.upper(), color="green", text_color="white").props("outline") ui.chip(job.status.value.upper(), color="positive", text_color="white").props("outline")
case _: case _:
ui.label(f"{job.status.value}").classes("text-subtitle1 text-weight-medium") ui.label(f"{job.status.value}").classes("text-subtitle1 text-weight-medium")
@@ -106,7 +103,7 @@ def register_page() -> None: # noqa: PLR0915
refreshed_job = await jobs_service.read_job(job_id=parsed_job_id) refreshed_job = await jobs_service.read_job(job_id=parsed_job_id)
refreshed_source = _resolve_primary_source(refreshed_job) refreshed_source = _resolve_primary_source(refreshed_job)
if refreshed_source is None: if refreshed_source is None:
ui.label("No source is available for revision editing.").classes("text-body2 text-grey-3") ui.label("No source is available for revision editing.").classes("text-body2 vibe-text-muted")
return return
current_revision = refreshed_source.revision current_revision = refreshed_source.revision
@@ -144,7 +141,7 @@ def register_page() -> None: # noqa: PLR0915
).props('unelevated color="primary"') ).props('unelevated color="primary"')
if current_revision is None: if current_revision is None:
ui.label("No revision exists for this source.").classes("text-body2 text-grey-3") ui.label("No revision exists for this source.").classes("text-body2 vibe-text-muted")
return return
render_revision_row( render_revision_row(
+2 -4
View File
@@ -5,8 +5,7 @@ from __future__ import annotations
from fastapi import Request from fastapi import Request
from nicegui import ui from nicegui import ui
from transcription.app_state import resolve_session_factory from transcription.db import session_scope
from transcription.db import get_session
from transcription.services.store import create_upload_job from transcription.services.store import create_upload_job
from transcription.ui.components.app_shell import render_navigation_header from transcription.ui.components.app_shell import render_navigation_header
from transcription.ui.components.upload import render_upload_widget from transcription.ui.components.upload import render_upload_widget
@@ -19,10 +18,9 @@ def register_page() -> None:
@ui.page("/upload", title="Upload Document") @ui.page("/upload", title="Upload Document")
def upload_page(request: Request) -> None: def upload_page(request: Request) -> None:
render_navigation_header(current_path="/upload") render_navigation_header(current_path="/upload")
session_factory = resolve_session_factory(request.app.state)
async def submit_upload(filename: str, file_bytes: bytes): async def submit_upload(filename: str, file_bytes: bytes):
async with get_session(session_factory=session_factory) as session: async with session_scope() as session:
return await create_upload_job( return await create_upload_job(
filename=filename, filename=filename,
file_bytes=file_bytes, file_bytes=file_bytes,
+19
View File
@@ -0,0 +1,19 @@
"""Package resource helpers for UI presentation assets."""
from __future__ import annotations
from functools import cache
from importlib.resources import files
from pathlib import PurePosixPath
@cache
def read_css(relative_path: str) -> str:
"""Read and cache a CSS resource relative to ``ui/static``."""
resource_path = PurePosixPath(relative_path)
if resource_path.is_absolute() or ".." in resource_path.parts or resource_path.suffix != ".css":
msg = f"Invalid CSS resource path: {relative_path}"
raise ValueError(msg)
resource = files("transcription.ui").joinpath("static", *resource_path.parts)
return resource.read_text(encoding="utf-8")
-30
View File
@@ -1,30 +0,0 @@
:root {
/* Soft blue-night palette tokens */
--ctp-rosewater: #f2dde5;
--ctp-flamingo: #edcfd8;
--ctp-pink: #dcc7de;
--ctp-mauve: #a9bde5;
--ctp-red: #d98a9a;
--ctp-maroon: #d39aa5;
--ctp-peach: #d7af8c;
--ctp-yellow: #e2c083;
--ctp-green: #86c8ad;
--ctp-teal: #77bfbe;
--ctp-sky: #7ebdda;
--ctp-sapphire: #74aed0;
--ctp-blue: #92b5f5;
--ctp-lavender: #6f97e8;
--ctp-text: #d8e2f5;
--ctp-subtext1: #bfcae0;
--ctp-subtext0: #a9b6cf;
--ctp-overlay2: #95a3bf;
--ctp-overlay1: #7c8ca9;
--ctp-overlay0: #657490;
--ctp-surface2: #4d5f7c;
--ctp-surface1: #394a65;
--ctp-surface0: #2a3954;
--ctp-base: #1f2b42;
--ctp-mantle: #1a2538;
--ctp-crust: #141e30;
}
@@ -0,0 +1,86 @@
.app-shell {
min-height: 64px;
padding: 0.75rem 2rem;
border-bottom: 1px solid var(--theme-border);
color: var(--theme-text);
background: var(--theme-surface-raised);
}
.app-shell__inner {
width: 100%;
display: grid;
grid-template-columns: minmax(180px, 1fr) auto minmax(180px, 1fr);
align-items: center;
gap: 1.5rem;
}
.app-shell__brand,
.app-shell__actions {
align-items: center;
}
.app-shell__brand {
gap: 0.75rem;
color: var(--theme-text);
font-family: Georgia, serif;
font-weight: 700;
}
.app-shell__brand-mark {
width: 34px;
height: 34px;
display: grid;
place-items: center;
color: var(--theme-inverse-text);
background: var(--theme-primary);
font-family: "Trebuchet MS", sans-serif;
font-size: 0.72rem;
}
.app-shell__nav {
display: flex;
align-items: center;
gap: 0.5rem;
}
.app-shell__nav-item {
min-height: 40px;
color: var(--theme-text-muted);
}
.app-shell__nav-item--active {
color: var(--theme-primary-hover);
border-bottom: 3px solid var(--theme-primary);
}
.app-shell__actions {
justify-content: flex-end;
gap: 0.75rem;
}
.app-shell__save-state {
color: var(--theme-text-muted);
font-size: 0.82rem;
font-weight: 700;
}
@media (max-width: 700px) {
.app-shell {
padding-inline: 0.75rem;
}
.app-shell__inner {
grid-template-columns: 1fr auto;
}
.app-shell__nav {
grid-column: 1 / -1;
grid-row: 2;
justify-content: center;
}
.app-shell__brand-name,
.app-shell__save-state {
display: none;
}
}
@@ -0,0 +1,138 @@
.page-content {
width: 100%;
display: grid;
grid-template-columns: minmax(0, 1fr) minmax(260px, 320px);
align-items: start;
border: 1px solid var(--theme-border);
color: var(--theme-text);
background: var(--theme-surface-raised);
box-shadow: var(--theme-shadow);
}
.page-content__editor {
min-width: 0;
padding: 1.5rem;
}
.page-content__heading {
width: 100%;
align-items: center;
justify-content: space-between;
margin-bottom: 1rem;
}
.page-content__kicker {
color: var(--theme-text-muted);
font-size: 0.72rem;
font-weight: 800;
text-transform: uppercase;
}
.page-content__title,
.page-content__sidebar-title {
color: var(--theme-text);
font-family: Georgia, serif;
font-weight: 700;
}
.page-content__title {
font-size: 1.35rem;
}
.page-content__badge {
color: var(--theme-text);
background: var(--theme-surface-muted);
}
.page-content__workspace {
display: grid;
grid-template-columns: minmax(280px, 0.85fr) minmax(320px, 1.15fr);
gap: 1.5rem;
}
.source-placeholder {
min-height: 440px;
display: grid;
grid-template-rows: auto 1fr;
border: 1px solid var(--theme-viewer-border);
background: var(--theme-viewer);
}
.source-placeholder__toolbar {
width: 100%;
min-height: 48px;
align-items: center;
justify-content: space-between;
padding: 0.5rem 0.75rem;
color: var(--theme-inverse-text);
}
.source-placeholder__body {
align-items: center;
justify-content: center;
color: var(--theme-viewer-muted);
}
.transcription-placeholder {
min-width: 0;
gap: 1rem;
}
.transcription-placeholder__section {
min-width: 0;
}
.transcription-placeholder__title {
margin-bottom: 0.5rem;
font-weight: 700;
}
.transcription-placeholder__text {
min-height: 160px;
padding: 1rem;
border: 1px solid var(--theme-border);
background: var(--theme-surface);
font-family: Georgia, serif;
}
.page-content__sidebar {
min-width: 0;
border-left: 1px solid var(--theme-border);
background: var(--theme-surface);
}
.page-content__sidebar-section {
display: grid;
gap: 0.75rem;
padding: 1.5rem;
border-bottom: 1px solid var(--theme-border);
}
.page-content__sidebar-title {
font-size: 1.05rem;
}
@media (max-width: 1000px) {
.page-content {
grid-template-columns: 1fr;
}
.page-content__sidebar {
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
border-top: 1px solid var(--theme-border);
border-left: 0;
}
}
@media (max-width: 760px) {
.page-content__workspace,
.page-content__sidebar {
grid-template-columns: 1fr;
}
.page-content__editor,
.page-content__sidebar-section {
padding: 1rem;
}
}
@@ -0,0 +1,60 @@
.page-header {
width: 100%;
display: flex;
align-items: flex-end;
justify-content: space-between;
gap: 2rem;
padding: 2rem 0 1.5rem;
}
.page-header__identity {
gap: 0.25rem;
}
.page-header__eyebrow {
color: var(--theme-text-muted);
font-size: 0.72rem;
font-weight: 800;
text-transform: uppercase;
}
.page-header__title {
color: var(--theme-text);
font-family: Georgia, serif;
font-size: 2rem;
font-weight: 700;
line-height: 1.15;
}
.page-header__metadata {
gap: 0.75rem;
color: var(--theme-text-muted);
font-size: 0.92rem;
}
.page-header__metadata > * + *::before {
margin-right: 0.75rem;
content: "·";
}
.page-header__actions {
flex-wrap: wrap;
justify-content: flex-end;
gap: 0.75rem;
}
@media (max-width: 700px) {
.page-header {
align-items: flex-start;
flex-direction: column;
}
.page-header__title {
font-size: 1.65rem;
}
.page-header__actions,
.page-header__actions .q-btn {
width: 100%;
}
}
+105
View File
@@ -0,0 +1,105 @@
:root {
--palette-carbon-black: #1c2321;
--palette-cool-steel: #7d98a1;
--palette-blue-slate: #5e6572;
--palette-powder-blue: #a9b4c2;
--palette-platinum: #eef1ef;
--theme-text: var(--palette-carbon-black);
--theme-text-muted: var(--palette-blue-slate);
--theme-page: var(--palette-platinum);
--theme-surface: color-mix(in srgb, var(--palette-platinum) 88%, var(--palette-powder-blue));
--theme-surface-raised: var(--palette-platinum);
--theme-surface-muted: color-mix(in srgb, var(--palette-platinum) 68%, var(--palette-powder-blue));
--theme-border: var(--palette-powder-blue);
--theme-primary: var(--palette-blue-slate);
--theme-primary-hover: var(--palette-carbon-black);
--theme-secondary: var(--palette-cool-steel);
--theme-focus: var(--palette-cool-steel);
--theme-inverse-text: var(--palette-platinum);
--theme-viewer: var(--palette-carbon-black);
--theme-viewer-border: var(--palette-blue-slate);
--theme-viewer-muted: var(--palette-powder-blue);
--theme-shadow: 0 10px 28px color-mix(in srgb, var(--palette-carbon-black) 14%, transparent);
--q-primary: var(--palette-blue-slate);
--q-secondary: var(--palette-cool-steel);
--q-accent: var(--palette-powder-blue);
--q-dark: var(--palette-carbon-black);
--q-dark-page: var(--palette-carbon-black);
--q-positive: var(--palette-cool-steel);
--q-negative: var(--palette-carbon-black);
--q-info: var(--palette-cool-steel);
--q-warning: var(--palette-powder-blue);
}
body,
.q-layout,
.q-page-container {
color: var(--theme-text);
background: var(--theme-page);
}
body {
font-family: "Aptos", "Trebuchet MS", sans-serif;
}
.q-card,
.vibe-card {
border: 1px solid var(--theme-border);
color: var(--theme-text);
background: var(--theme-surface-raised);
box-shadow: none;
}
.vibe-card--error {
border-color: var(--palette-carbon-black);
color: var(--theme-inverse-text);
background: var(--palette-carbon-black);
}
.vibe-text-muted {
color: var(--theme-text-muted);
}
.vibe-separator {
background: var(--theme-border);
}
.vibe-status {
border: 1px solid currentColor;
}
.vibe-status--queued {
color: var(--palette-blue-slate);
background: var(--palette-platinum);
}
.vibe-status--processing {
color: var(--palette-carbon-black);
background: var(--palette-powder-blue);
}
.vibe-status--transcribed {
color: var(--palette-carbon-black);
background: var(--palette-cool-steel);
}
.vibe-status--failed {
color: var(--palette-platinum);
background: var(--palette-carbon-black);
}
.vibe-status--default {
color: var(--palette-blue-slate);
background: var(--theme-surface-muted);
}
button:focus-visible,
a:focus-visible,
textarea:focus-visible,
input:focus-visible,
[tabindex="0"]:focus-visible {
outline: 3px solid var(--theme-focus);
outline-offset: 2px;
}
+2 -2
View File
@@ -14,7 +14,7 @@ from uuid import UUID
from sqlalchemy.ext.asyncio import async_sessionmaker from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
from transcription.db import get_session from transcription.db import session_scope
from transcription.errors import AppError from transcription.errors import AppError
from transcription.errors import classify_unexpected_error from transcription.errors import classify_unexpected_error
@@ -178,7 +178,7 @@ async def process_next_queued_job(
) )
if session is None: if session is None:
async with get_session(session_factory=session_factory) as local_session: async with session_scope(session_factory=session_factory) as local_session:
return await process_next_queued_job_workflow(services=services, session=local_session) return await process_next_queued_job_workflow(services=services, session=local_session)
return await process_next_queued_job_workflow(services=services, session=session) return await process_next_queued_job_workflow(services=services, session=session)
+12 -8
View File
@@ -13,11 +13,12 @@ from sqlmodel.pool import StaticPool
from transcription.config import Settings from transcription.config import Settings
from transcription.config import get_settings from transcription.config import get_settings
from transcription.db.engine import get_database_url
from transcription.db.engine import get_engine
from transcription.db.operations import create_all from transcription.db.operations import create_all
from transcription.db.runtime import dispose_database_runtime from transcription.db.session import dispose_session_factory
from transcription.db.runtime import get_engine from transcription.db.session import get_session_factory
from transcription.db.runtime import get_session from transcription.db.session import session_scope
from transcription.db.runtime import get_session_factory
from transcription.services.documents import DocumentService from transcription.services.documents import DocumentService
from transcription.services.jobs import JobService from transcription.services.jobs import JobService
@@ -39,23 +40,26 @@ def session():
async def default_settings(): async def default_settings():
"""Provide default settings for tests.""" """Provide default settings for tests."""
settings = get_settings(database_url="sqlite:///:memory:") settings = get_settings(database_url="sqlite:///:memory:")
await create_all(engine=get_engine(settings=settings)) db_url = get_database_url(settings)
await create_all(engine=get_engine(database_url=db_url))
return settings return settings
@pytest_asyncio.fixture @pytest_asyncio.fixture
async def async_session(default_settings: Settings): async def async_session(default_settings: Settings):
"""Provide a clean asynchronous database session for async tests.""" """Provide a clean asynchronous database session for async tests."""
async with get_session(settings=default_settings) as async_session: db_url = get_database_url(default_settings)
async with session_scope(database_url=db_url) as async_session:
yield async_session yield async_session
await dispose_database_runtime() await dispose_session_factory(db_url)
@pytest.fixture @pytest.fixture
def default_session_factory(default_settings: Settings): def default_session_factory(default_settings: Settings):
"""Provide a base fixture for tests that require database access.""" """Provide a base fixture for tests that require database access."""
session_factory = get_session_factory(settings=default_settings) db_url = get_database_url(default_settings)
session_factory = get_session_factory(database_url=db_url)
return session_factory return session_factory
+2 -2
View File
@@ -5,8 +5,8 @@ from pathlib import Path
import pytest import pytest
from transcription.config import Settings from transcription.config import Settings
from transcription.models import Job from transcription.db.models import Job
from transcription.models import JobStatus from transcription.db.models import JobStatus
from transcription.providers.base import TranscriptionResult from transcription.providers.base import TranscriptionResult
from transcription.services.store import create_upload_job from transcription.services.store import create_upload_job
from transcription.worker import process_next_queued_job from transcription.worker import process_next_queued_job
+4 -4
View File
@@ -2,10 +2,10 @@ from uuid import uuid4
import pytest import pytest
from transcription.models import Document from transcription.db.models import Document
from transcription.models import Job from transcription.db.models import Job
from transcription.models import JobStatus from transcription.db.models import JobStatus
from transcription.models import Source from transcription.db.models import Source
from transcription.services.documents import DocumentService from transcription.services.documents import DocumentService
from transcription.services.jobs import JobService from transcription.services.jobs import JobService
+4 -4
View File
@@ -4,10 +4,10 @@ from uuid import uuid4
import pytest import pytest
from transcription.models import Document from transcription.db.models import Document
from transcription.models import Job from transcription.db.models import Job
from transcription.models import JobStatus from transcription.db.models import JobStatus
from transcription.models import Source from transcription.db.models import Source
from transcription.services.documents import DocumentService from transcription.services.documents import DocumentService
from transcription.services.jobs import JobService from transcription.services.jobs import JobService
from transcription.services.transcription import TranscriptionService from transcription.services.transcription import TranscriptionService
+4 -4
View File
@@ -6,10 +6,10 @@ from uuid import uuid4
import pytest import pytest
from transcription.config import Settings from transcription.config import Settings
from transcription.models import Document from transcription.db.models import Document
from transcription.models import Job from transcription.db.models import Job
from transcription.models import JobStatus from transcription.db.models import JobStatus
from transcription.models import Source from transcription.db.models import Source
from transcription.services import ServiceBundle from transcription.services import ServiceBundle
from transcription.services.workflows import process_queued_job from transcription.services.workflows import process_queued_job
+2 -2
View File
@@ -27,7 +27,7 @@ class TestAppLifespan:
"""Startup initializes logging, schema, directories, and worker resources.""" """Startup initializes logging, schema, directories, and worker resources."""
calls = [] calls = []
monkeypatch.setattr("transcription.app.configure_logging", lambda: calls.append("logging")) monkeypatch.setattr("transcription.app.configure_logging", lambda _settings: calls.append("logging"))
async def _create_all(**_kwargs): async def _create_all(**_kwargs):
calls.append("schema") calls.append("schema")
@@ -80,7 +80,7 @@ class TestAppLifespan:
"""Shutdown signals and stops worker resources cleanly.""" """Shutdown signals and stops worker resources cleanly."""
calls = [] calls = []
monkeypatch.setattr("transcription.app.configure_logging", lambda: calls.append("logging")) monkeypatch.setattr("transcription.app.configure_logging", lambda _settings: calls.append("logging"))
async def _create_all(**_kwargs): async def _create_all(**_kwargs):
calls.append("schema") calls.append("schema")
+25
View File
@@ -7,6 +7,7 @@ from pydantic import ValidationError
from transcription.config import Provider from transcription.config import Provider
from transcription.config import Settings from transcription.config import Settings
from transcription.config import parse_cli_settings
def _make_settings(**overrides) -> Settings: def _make_settings(**overrides) -> Settings:
@@ -31,6 +32,30 @@ class TestSettingsLoading:
with pytest.raises(ValidationError): with pytest.raises(ValidationError):
Settings(_env_file=None) Settings(_env_file=None)
def test_ignores_process_cli_arguments(self, monkeypatch):
"""Ordinary settings construction does not consume tooling arguments."""
monkeypatch.setattr("sys.argv", ["pytest", "--rootdir=/tmp/project"])
settings = _make_settings()
assert settings.port == 8000
def test_explicit_cli_parser_reads_arguments(self):
"""The executable settings boundary accepts application CLI flags."""
settings = parse_cli_settings(
[
"--openrouter-api-key",
"test-key",
"--port",
"8123",
"--reload",
]
)
assert settings.openrouter_api_key == "test-key"
assert settings.port == 8123
assert settings.reload is True
class TestProviderSettings: class TestProviderSettings:
"""Verify provider enum defaults and validation.""" """Verify provider enum defaults and validation."""
+5 -4
View File
@@ -4,17 +4,18 @@ import pytest
from sqlalchemy import inspect from sqlalchemy import inspect
from transcription.config import Settings from transcription.config import Settings
from transcription.config import SqliteSettings
from transcription.db import create_all from transcription.db import create_all
from transcription.db import dispose_database_runtime from transcription.db import dispose_database_runtime
from transcription.db import get_session
from transcription.db import initialize_database_runtime from transcription.db import initialize_database_runtime
from transcription.db import session_scope
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_create_all_creates_expected_tables(tmp_path): async def test_create_all_creates_expected_tables(tmp_path):
settings = Settings( settings = Settings(
openrouter_api_key="test-key", openrouter_api_key="test-key",
database_url=f"sqlite:///{tmp_path / 'schema.db'}", database=SqliteSettings(path=str(tmp_path / "schema.db")),
environment="test", environment="test",
) )
runtime = initialize_database_runtime(settings=settings) runtime = initialize_database_runtime(settings=settings)
@@ -36,13 +37,13 @@ async def test_create_all_creates_expected_tables(tmp_path):
async def test_get_session_yields_async_session(tmp_path): async def test_get_session_yields_async_session(tmp_path):
settings = Settings( settings = Settings(
openrouter_api_key="test-key", openrouter_api_key="test-key",
database_url=f"sqlite:///{tmp_path / 'session.db'}", database=SqliteSettings(path=str(tmp_path / "session.db")),
environment="test", environment="test",
) )
initialize_database_runtime(settings=settings) initialize_database_runtime(settings=settings)
try: try:
async with get_session(settings=settings) as session: async with session_scope(settings=settings) as session:
assert session is not None assert session is not None
finally: finally:
await dispose_database_runtime() await dispose_database_runtime()
+58
View File
@@ -0,0 +1,58 @@
"""Tests for the executable application entry point."""
from types import SimpleNamespace
import pytest
from transcription import __main__ as entrypoint
@pytest.mark.unit
def test_main_passes_constructed_app_to_uvicorn(monkeypatch):
"""Non-reload execution keeps the parsed settings instance in the app."""
settings = SimpleNamespace(host="127.0.0.1", port=8123, log_level="debug", reload=False)
application = object()
captured = {}
def create_app(*, settings: object) -> object:
assert settings is expected_settings
return application
expected_settings = settings
monkeypatch.setattr(entrypoint, "parse_cli_settings", lambda: settings)
monkeypatch.setattr(entrypoint, "create_app", create_app)
monkeypatch.setattr(
entrypoint.uvicorn,
"run",
lambda app, **kwargs: captured.update(application=app, **kwargs),
)
entrypoint.main()
assert captured == {
"application": application,
"factory": False,
"host": "127.0.0.1",
"port": 8123,
"log_level": "debug",
"reload": False,
}
@pytest.mark.unit
def test_main_uses_cli_factory_for_reload(monkeypatch):
"""Reload execution gives Uvicorn an importable CLI-aware factory."""
settings = SimpleNamespace(host="127.0.0.1", port=8123, log_level="info", reload=True)
captured = {}
monkeypatch.setattr(entrypoint, "parse_cli_settings", lambda: settings)
monkeypatch.setattr(
entrypoint.uvicorn,
"run",
lambda app, **kwargs: captured.update(application=app, **kwargs),
)
entrypoint.main()
assert captured["application"] == "transcription.__main__:create_cli_app"
assert captured["factory"] is True
assert captured["reload"] is True
+5 -5
View File
@@ -5,11 +5,11 @@ from uuid import UUID
import pytest import pytest
from sqlalchemy.exc import IntegrityError from sqlalchemy.exc import IntegrityError
from transcription.models import Document from transcription.db.models import Document
from transcription.models import Job from transcription.db.models import Job
from transcription.models import JobStatus from transcription.db.models import JobStatus
from transcription.models import Revision from transcription.db.models import Revision
from transcription.models import Source from transcription.db.models import Source
def _make_document(**overrides) -> Document: def _make_document(**overrides) -> Document:
+38
View File
@@ -0,0 +1,38 @@
"""Tests for global UI theme registration."""
import re
import pytest
from fastapi import FastAPI
from transcription.ui import register_pages
from transcription.ui.resources import read_css
@pytest.mark.unit
def test_page_registration_uses_vibescribe_theme(monkeypatch):
"""Global UI registration loads the standalone VibeScribe theme in light mode."""
registered_css: list[str] = []
run_options: dict[str, object] = {}
monkeypatch.setattr("transcription.ui.ui.add_css", lambda css, **_kwargs: registered_css.append(css))
monkeypatch.setattr("transcription.ui.register_upload_page", lambda: None)
monkeypatch.setattr("transcription.ui.register_jobs_page", lambda: None)
monkeypatch.setattr(
"transcription.ui.ui.run_with",
lambda _app, **options: run_options.update(options),
)
register_pages(FastAPI())
theme_css = read_css("theme.css")
assert registered_css == [theme_css]
assert set(re.findall(r"#[0-9a-fA-F]{6}", theme_css)) == {
"#1c2321",
"#7d98a1",
"#5e6572",
"#a9b4c2",
"#eef1ef",
}
assert "--q-primary" in theme_css
assert run_options["dark"] is False
+12 -12
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import asyncio import asyncio
from collections.abc import Callable from collections.abc import Callable
from collections.abc import Generator
from pathlib import Path from pathlib import Path
from uuid import UUID from uuid import UUID
@@ -14,32 +15,31 @@ from sqlmodel import delete
from transcription.app import create_app from transcription.app import create_app
from transcription.config import Settings from transcription.config import Settings
from transcription.config import _settings from transcription.config import SqliteSettings
from transcription.db import create_all from transcription.db import create_all
from transcription.db import get_session
from transcription.db import initialize_database_runtime from transcription.db import initialize_database_runtime
from transcription.models import Document from transcription.db import session_scope
from transcription.models import Job from transcription.db.models import Document
from transcription.models import JobStatus from transcription.db.models import Job
from transcription.models import Revision from transcription.db.models import JobStatus
from transcription.models import Source from transcription.db.models import Revision
from transcription.db.models import Source
RevisionSeed = str RevisionSeed = str
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def app_client(tmp_path_factory: pytest.TempPathFactory) -> tuple[FastAPI, TestClient]: def app_client(tmp_path_factory: pytest.TempPathFactory) -> Generator[tuple[FastAPI, TestClient]]:
"""Provide a real application and test client backed by in-memory SQLite.""" """Provide a real application and test client backed by in-memory SQLite."""
tmp_path = tmp_path_factory.mktemp("ui") tmp_path = tmp_path_factory.mktemp("ui")
settings = Settings( settings = Settings(
openrouter_api_key="test-key", openrouter_api_key="test-key",
database_url="sqlite:///:memory:", database=SqliteSettings(path=":memory:"),
environment="test", environment="test",
bootstrap_schema_on_startup=True, bootstrap_schema_on_startup=True,
upload_dir=tmp_path / "uploads", upload_dir=tmp_path / "uploads",
prompt_dir=tmp_path / "prompts", prompt_dir=tmp_path / "prompts",
) )
_settings.set(settings)
app = create_app() app = create_app()
app.state.runtime = initialize_database_runtime(settings=settings) app.state.runtime = initialize_database_runtime(settings=settings)
@@ -54,7 +54,7 @@ def clear_ui_database(app_client: tuple[FastAPI, TestClient]) -> None:
app, _ = app_client app, _ = app_client
async def _clear() -> None: async def _clear() -> None:
async with get_session(session_factory=app.state.runtime.session_factory) as session: async with session_scope() as session:
await session.exec(delete(Revision)) await session.exec(delete(Revision))
await session.exec(delete(Source)) await session.exec(delete(Source))
await session.exec(delete(Job)) await session.exec(delete(Job))
@@ -80,7 +80,7 @@ def seed_job(app_client: tuple[FastAPI, TestClient]) -> Callable[..., UUID]:
source_file: Path | None = None, source_file: Path | None = None,
) -> UUID: ) -> UUID:
async def _insert() -> UUID: async def _insert() -> UUID:
async with get_session(session_factory=app.state.runtime.session_factory) as session: async with session_scope() as session:
stored_path = app.state.settings.upload_dir / filename stored_path = app.state.settings.upload_dir / filename
stored_path.parent.mkdir(parents=True, exist_ok=True) stored_path.parent.mkdir(parents=True, exist_ok=True)
source_path = source_file or fixtures_dir / "small_png.png" source_path = source_file or fixtures_dir / "small_png.png"
+1 -1
View File
@@ -5,7 +5,7 @@ from uuid import uuid4
import pytest import pytest
from transcription.models import JobStatus from transcription.db.models import JobStatus
@pytest.mark.integration @pytest.mark.integration
+1
View File
@@ -29,6 +29,7 @@ class TestPageRendering:
response = client.get("/ui/upload") response = client.get("/ui/upload")
assert response.status_code == 200 assert response.status_code == 200
assert "VibeScribe" in response.text
assert "Upload Document" in response.text assert "Upload Document" in response.text
assert "Select document file" in response.text assert "Select document file" in response.text
assert "Upload" in response.text assert "Upload" in response.text
Generated
+459 -388
View File
File diff suppressed because it is too large Load Diff