V6.0 Phase 3 - Add cloudflare tunnel. Remote access to app via the tunnel now working.
Quality Gate / gate (push) Failing after 49s

This commit is contained in:
Jim Lancaster
2026-08-25 13:05:47 -05:00
parent faa30fd27b
commit 234476ba6d
10 changed files with 136 additions and 9 deletions
+1
View File
@@ -50,4 +50,5 @@ POSTGRES_USER=transcription
POSTGRES_PASSWORD=replace-with-strong-password POSTGRES_PASSWORD=replace-with-strong-password
# --- cloudflare tunnel --- # --- cloudflare tunnel ---
# Required for token-based tunnel startup.
CLOUDFLARE_TUNNEL_TOKEN=replace-with-cloudflare-tunnel-token CLOUDFLARE_TUNNEL_TOKEN=replace-with-cloudflare-tunnel-token
+3
View File
@@ -27,3 +27,6 @@ data/photos/*
data-migration-test/* data-migration-test/*
.migration-bundle/* .migration-bundle/*
# Cloudflare tunnel local runtime files
deploy/cloudflared/config.yml
deploy/cloudflared/credentials.json
+7 -1
View File
@@ -137,7 +137,13 @@ Services:
- `app`: FastAPI + NiceGUI runtime (`RUN_EMBEDDED_WORKER=false`) - `app`: FastAPI + NiceGUI runtime (`RUN_EMBEDDED_WORKER=false`)
- `worker`: standalone queue processor (`python -m transcription.worker_service`) - `worker`: standalone queue processor (`python -m transcription.worker_service`)
- `postgres`: primary datastore - `postgres`: primary datastore
- `cloudflared`: tunnel client using `CLOUDFLARE_TUNNEL_TOKEN` - `cloudflared`: tunnel client using mounted ingress config + `CLOUDFLARE_TUNNEL_TOKEN`
Cloudflare setup files:
1. `copy deploy\cloudflared\config.yml.example deploy\cloudflared\config.yml`
2. set `CLOUDFLARE_TUNNEL_TOKEN` in `.env.production`
3. update ingress hostnames in `deploy\cloudflared\config.yml`
## How to navigate the GUI ## How to navigate the GUI
+14
View File
@@ -0,0 +1,14 @@
ingress:
# Primary transcription app endpoint.
- hostname: transcription.example.com
service: http://app:8000
# Optional: generic remote access endpoints for other internal services.
# Replace hostnames and targets for your LAN.
- hostname: homeassistant.example.com
service: http://192.168.1.50:8123
- hostname: pihole.example.com
service: http://192.168.1.60:80
# Required catch-all.
- service: http_status:404
+3 -3
View File
@@ -54,17 +54,17 @@ services:
timeout: 5s timeout: 5s
retries: 10 retries: 10
start_period: 10s start_period: 10s
ports:
- "5432:5432"
cloudflared: cloudflared:
image: cloudflare/cloudflared:2026.8.0 image: cloudflare/cloudflared:2026.8.0
env_file: env_file:
- .env.production - .env.production
command: tunnel --no-autoupdate run --token ${CLOUDFLARE_TUNNEL_TOKEN} command: tunnel --no-autoupdate --config /etc/cloudflared/config.yml run --token ${CLOUDFLARE_TUNNEL_TOKEN}
depends_on: depends_on:
app: app:
condition: service_healthy condition: service_healthy
volumes:
- ./deploy/cloudflared/config.yml:/etc/cloudflared/config.yml:ro
restart: unless-stopped restart: unless-stopped
volumes: volumes:
+60
View File
@@ -0,0 +1,60 @@
# Cloudflare Tunnel and Access Setup (V6.0 Phase 3)
This guide defines the repository-supported setup for exposing app and selected LAN services through Cloudflare Tunnel with Cloudflare Access protection.
## 1. Files used by this deployment
1. `deploy/cloudflared/config.yml` (local copy from `config.yml.example`)
2. `.env.production` (`CLOUDFLARE_TUNNEL_TOKEN`)
3. `docker-compose.production.yml` (`cloudflared` service reads token + mounts config)
Do not commit `config.yml` or `.env.production`.
## 2. Configure cloudflared
1. Copy `deploy/cloudflared/config.yml.example` to `deploy/cloudflared/config.yml`.
2. Update hostname -> service mappings in `ingress`.
3. Keep the final catch-all ingress `http_status:404`.
4. Set `CLOUDFLARE_TUNNEL_TOKEN` in `.env.production`.
Example app route:
- `transcription.example.com` -> `http://app:8000`
Optional generic remote-access routes:
- `homeassistant.example.com` -> `http://<home-assistant-lan-ip>:8123`
- `pihole.example.com` -> `http://<pihole-lan-ip>:80`
## 3. Cloudflare Access policy baseline
Create one Access app policy per exposed hostname:
1. Include: your allowed identities/groups only.
2. Exclude: none by default.
3. Require: identity provider login (and MFA if available).
Recommended baseline:
- App endpoint (`transcription.*`): your admin identity set.
- Other internal endpoints (`homeassistant.*`, `pihole.*`, etc.): explicit least-privilege groups.
## 4. Startup
Start production stack:
```bash
docker compose --env-file .env.production -f docker-compose.production.yml up -d --build
```
Validate tunnel container:
```bash
docker compose --env-file .env.production -f docker-compose.production.yml logs cloudflared
```
## 5. Security notes
- Keep `postgres` and other internal-only services off public hostnames unless required.
- Use distinct hostnames per service; avoid path-based multiplexing for unrelated admin surfaces.
- Rotate `CLOUDFLARE_TUNNEL_TOKEN` and Access policy memberships on a regular schedule.
+8
View File
@@ -25,6 +25,7 @@ This runbook is the operational checklist for releasing and monitoring the trans
- `/healthz` responds `200` - `/healthz` responds `200`
- if `RUN_EMBEDDED_WORKER=true`, `worker.state` is `running` - if `RUN_EMBEDDED_WORKER=true`, `worker.state` is `running`
- if `RUN_EMBEDDED_WORKER=false`, validate `worker` container is healthy/running in Compose - if `RUN_EMBEDDED_WORKER=false`, validate `worker` container is healthy/running in Compose
- validate `cloudflared` logs show active tunnel routes and no ingress errors
3. Execute one smoke workflow: 3. Execute one smoke workflow:
- create a document/job with at least one source - create a document/job with at least one source
- verify terminal job outcome updates - verify terminal job outcome updates
@@ -86,6 +87,13 @@ This runbook is the operational checklist for releasing and monitoring the trans
2. Confirm failures are category-aligned (`external`/`timeout`/`internal`). 2. Confirm failures are category-aligned (`external`/`timeout`/`internal`).
3. Triage whether issue is source quality, provider, or runtime regression. 3. Triage whether issue is source quality, provider, or runtime regression.
### Cloudflare ingress/access failure
1. Check `cloudflared` container logs for ingress parse, DNS, or auth failures.
2. Confirm `deploy/cloudflared/config.yml` tunnel UUID and hostname mappings are correct.
3. Confirm `deploy/cloudflared/credentials.json` matches the tunnel configured in Cloudflare.
4. Confirm Cloudflare Access app policy includes the intended identity/group for that hostname.
## 6. Dependency upgrade policy ## 6. Dependency upgrade policy
Dependencies are declared in `pyproject.toml` and resolved through the committed Dependencies are declared in `pyproject.toml` and resolved through the committed
+5
View File
@@ -49,6 +49,11 @@ Implemented workflow references:
2. Apply Cloudflare Access policies for identity-gated remote access. 2. Apply Cloudflare Access policies for identity-gated remote access.
3. Keep non-required administrative/internal surfaces LAN-only unless explicitly approved. 3. Keep non-required administrative/internal surfaces LAN-only unless explicitly approved.
Implemented workflow references:
- `deploy/cloudflared/config.yml.example`
- `docs/cloudflare_tunnel_access.md`
## 3.4 Backup, restore, rollback ## 3.4 Backup, restore, rollback
1. Define backup schedule, retention, and artifact naming/versioning. 1. Define backup schedule, retention, and artifact naming/versioning.
+20 -5
View File
@@ -26,6 +26,18 @@ from .base import ServiceBase
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def _utc_now_naive() -> datetime:
"""Return current UTC as naive datetime for DB timestamp columns."""
return datetime.now(UTC).replace(tzinfo=None)
def _as_naive_utc(value: datetime) -> datetime:
"""Normalize datetimes to naive UTC for DB comparisons/binds."""
if value.tzinfo is None:
return value
return value.astimezone(UTC).replace(tzinfo=None)
class JobDeleteBlockedError(AppError): class JobDeleteBlockedError(AppError):
"""Raised when a job delete operation is blocked by lifecycle policy.""" """Raised when a job delete operation is blocked by lifecycle policy."""
@@ -201,7 +213,7 @@ class JobService(ServiceBase):
await self._finalize(session=_session, caller_session=session, refresh=(job,)) await self._finalize(session=_session, caller_session=session, refresh=(job,))
return job return job
now = datetime.now(UTC) now = _utc_now_naive()
queued_job_id = ( queued_job_id = (
select(col(Job.id)) select(col(Job.id))
.where(col(Job.status) == JobStatus.QUEUED) .where(col(Job.status) == JobStatus.QUEUED)
@@ -239,12 +251,15 @@ class JobService(ServiceBase):
``stale_before`` are considered stale and re-queued. ``stale_before`` are considered stale and re-queued.
""" """
async with self._session_scope(session) as _session: async with self._session_scope(session) as _session:
query = select(Job).where(Job.status == JobStatus.PROCESSING).where(Job.date_updated < stale_before) normalized_stale_before = _as_naive_utc(stale_before)
query = (
select(Job).where(Job.status == JobStatus.PROCESSING).where(Job.date_updated < normalized_stale_before)
)
stale_jobs = (await _session.exec(query)).all() stale_jobs = (await _session.exec(query)).all()
if not stale_jobs: if not stale_jobs:
return 0 return 0
now = datetime.now(UTC) now = _utc_now_naive()
for job in stale_jobs: for job in stale_jobs:
job.status = JobStatus.QUEUED job.status = JobStatus.QUEUED
job.date_updated = now job.date_updated = now
@@ -352,7 +367,7 @@ class JobService(ServiceBase):
suggestion="Use resubmit for reprocessing needs, or leave the terminal job unchanged.", suggestion="Use resubmit for reprocessing needs, or leave the terminal job unchanged.",
) )
now = datetime.now(UTC) now = _utc_now_naive()
job.status = JobStatus.FAILED job.status = JobStatus.FAILED
job.date_updated = now job.date_updated = now
@@ -398,7 +413,7 @@ class JobService(ServiceBase):
suggestion="Only failed or cancelled sources can be resubmitted.", suggestion="Only failed or cancelled sources can be resubmitted.",
) )
now = datetime.now(UTC) now = _utc_now_naive()
for job_source in candidates: for job_source in candidates:
job_source.status = JobSourceStatus.PENDING job_source.status = JobSourceStatus.PENDING
+15
View File
@@ -20,9 +20,24 @@ from transcription.services.jobs import JobDeleteBlockedError
from transcription.services.jobs import JobNotFoundError from transcription.services.jobs import JobNotFoundError
from transcription.services.jobs import JobResubmitBlockedError from transcription.services.jobs import JobResubmitBlockedError
from transcription.services.jobs import JobService from transcription.services.jobs import JobService
from transcription.services.jobs import _as_naive_utc
from transcription.services.jobs import _utc_now_naive
from transcription.services.sources import SourceService from transcription.services.sources import SourceService
def test_utc_now_naive_returns_naive_datetime():
now = _utc_now_naive()
assert now.tzinfo is None
def test_as_naive_utc_normalizes_timezone_aware_datetime():
aware = datetime.now(UTC)
normalized = _as_naive_utc(aware)
assert normalized.tzinfo is None
assert normalized == aware.replace(tzinfo=None)
class TestJobService: class TestJobService:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_create_and_read_job(self, job_service: JobService, document_service: DocumentService): async def test_create_and_read_job(self, job_service: JobService, document_service: DocumentService):