generated from john/python-template
145 lines
4.7 KiB
Markdown
145 lines
4.7 KiB
Markdown
# Transcription
|
|
|
|
Historical document transcription system for family-history documents.
|
|
|
|
The app lets you upload a document image/PDF, queues a background transcription job, and then shows job status and results in a web UI.
|
|
|
|
## What the app does
|
|
|
|
- Upload document files (`.jpg`, `.jpeg`, `.png`, `.tif`, `.tiff`, `.pdf`)
|
|
- Persist document + job records in SQLite
|
|
- Process jobs in a background worker (`queued -> processing -> transcribed/failed`)
|
|
- Store transcript text (or failure detail)
|
|
- Show status and results in the NiceGUI interface
|
|
|
|
## Quick start
|
|
|
|
### 1) Install dependencies
|
|
|
|
```bash
|
|
uv sync
|
|
```
|
|
|
|
### 2) Configure environment
|
|
|
|
Create a `.env` file in the project root with the required OpenRouter API key:
|
|
|
|
```env
|
|
OPENROUTER_API_KEY=your_openrouter_api_key
|
|
```
|
|
|
|
Settings are read from CLI arguments first, then environment variables, then `.env`, then the defaults below.
|
|
|
|
### Configuration Source Precedence
|
|
|
|
When the same setting is provided in multiple places, the value is chosen in this order (highest priority first):
|
|
|
|
1. CLI arguments (for example `--port 9999`)
|
|
2. Settings constructor arguments (used mainly in tests)
|
|
3. Environment variables
|
|
4. `.env` file values
|
|
5. Model defaults in code
|
|
|
|
Practical examples:
|
|
|
|
- `--port 9999` overrides both `PORT=8000` in the shell and `PORT=7000` in `.env`.
|
|
- `DATABASE__PATH=prod.db` in the shell overrides `DATABASE__PATH=dev.db` in `.env`.
|
|
|
|
#### Server and runtime
|
|
|
|
| Environment variable | Default | Description |
|
|
| --- | --- | --- |
|
|
| `HOST` | `0.0.0.0` | Address on which the server listens. |
|
|
| `PORT` | `8000` | Server port. |
|
|
| `LOG_LEVEL` | `info` | Uvicorn and application log level. |
|
|
| `RELOAD` | `false` | Restart the development server when source files change. |
|
|
| `ENVIRONMENT` | `development` | Runtime environment: `development`, `test`, or `production`. |
|
|
|
|
#### Provider
|
|
|
|
| Environment variable | Default | Description |
|
|
| --- | --- | --- |
|
|
| `PROVIDER` | `openrouter` | Transcription provider. |
|
|
| `OPENROUTER_API_KEY` | Required | OpenRouter API key. |
|
|
| `PROVIDER_MODEL` | Provider default | Optional model override. |
|
|
| `OPENROUTER_HTTP_REFERER` | Unset | Optional OpenRouter attribution URL. |
|
|
| `OPENROUTER_APP_TITLE` | Unset | Optional OpenRouter attribution title. |
|
|
|
|
#### Database and files
|
|
|
|
Use nested env vars for database settings (recommended):
|
|
|
|
```env
|
|
DATABASE__DRIVER=sqlite
|
|
DATABASE__PATH=app.db
|
|
# BOOTSTRAP_SCHEMA_ON_STARTUP=true
|
|
SQLITE_CHECK_SAME_THREAD=false
|
|
UPLOAD_DIR=./uploads
|
|
PROMPT_DIR=./prompts
|
|
```
|
|
|
|
For PostgreSQL:
|
|
|
|
```env
|
|
DATABASE__DRIVER=postgres
|
|
DATABASE__HOST=localhost
|
|
DATABASE__PORT=5432
|
|
DATABASE__DATABASE=transcription
|
|
DATABASE__USER=postgres
|
|
DATABASE__PASSWORD=change-me
|
|
```
|
|
|
|
This uses Pydantic nested settings (`env_nested_delimiter='__'`) and avoids JSON blobs in `.env`. A top-level `DATABASE={...}` JSON value is still supported as a fallback, and nested keys such as `DATABASE__PATH` take precedence over conflicting JSON keys.
|
|
|
|
`BOOTSTRAP_SCHEMA_ON_STARTUP` creates missing tables when the app starts. When unset, it is enabled in `development` and `test`, and disabled in `production`; set it explicitly to override that policy. `SQLITE_CHECK_SAME_THREAD` defaults to `false`.
|
|
|
|
#### Worker
|
|
|
|
```env
|
|
WORKER_MAX_RETRIES=0
|
|
WORKER_RETRY_BACKOFF_SECONDS=0
|
|
WORKER_PROVIDER_TIMEOUT_SECONDS=20
|
|
WORKER_MIN_TRANSCRIPTION_CHARS=0
|
|
WORKER_MIN_TRANSCRIPTION_LINES=0
|
|
WORKER_FAIL_ON_FINISH_REASON_LENGTH=false
|
|
```
|
|
|
|
### 3) Run the app
|
|
|
|
```bash
|
|
uv run python -m transcription --port 9999 --reload --database.driver sqlite --bootstrap-schema-on-startup
|
|
```
|
|
|
|
This starts the development server with SQLite, creates missing tables, and enables automatic reload. Run `uv run python -m transcription --help` for all CLI options; CLI names use kebab case and nested database options use dot notation, such as `--database.path ./data/transcription.db`.
|
|
|
|
### 4) Open in browser
|
|
|
|
- GUI: [http://localhost:9999/ui](http://localhost:9999/ui)
|
|
- Health check: [http://localhost:9999/healthz](http://localhost:9999/healthz)
|
|
|
|
Replace `localhost` with the server's hostname or IP address when connecting from another machine.
|
|
|
|
## How to navigate the GUI
|
|
|
|
- **Upload page** (`/ui`)
|
|
- Select a supported file to upload.
|
|
- The app creates a queued transcription job.
|
|
- Use the **View jobs** link to inspect progress.
|
|
|
|
- **Jobs page** (`/ui/jobs`)
|
|
- See all jobs and their status.
|
|
- Use **Refresh** to reload current states.
|
|
- Open a specific job to see details.
|
|
|
|
- **Job detail page** (`/ui/jobs/{job_id}`)
|
|
- Shows job metadata and status.
|
|
- Displays transcript text when successful.
|
|
- Displays failure detail when transcription fails.
|
|
|
|
## Prompt artifacts
|
|
|
|
Prompt files are stored in `prompts/` and loaded from `PROMPT_DIR` (default: `./prompts`).
|
|
|
|
The canonical MVP prompt is:
|
|
- `prompts/transcribe_document.md`
|