diff --git a/.github/instructions/services.instructions.md b/.github/instructions/services.instructions.md index 4e19c57..a7f6543 100644 --- a/.github/instructions/services.instructions.md +++ b/.github/instructions/services.instructions.md @@ -173,6 +173,16 @@ pages before it. Enforced by `tests/integration/test_pipeline_atomicity.py`; per durability is separately enforced by `tests/services/test_workflows_reliability.py::TestWorkflowReliability::test_transcribed_page_is_committed_before_next_provider_call_finishes`. +### Stale-reclaim safety + +- `WORKER_STALE_JOB_SECONDS` must remain **greater than** `WORKER_PROVIDER_TIMEOUT_SECONDS`; stale + recovery must not be able to fire before one provider call can legitimately finish. +- Long-running multi-page orchestration must refresh job liveness explicitly between intermediate + page commits. Do not rely on incidental row updates or provider metadata writes to keep + `Job.date_updated` fresh. +- Enforced by `tests/test_config.py` and + `tests/services/test_workflows_reliability.py::TestWorkflowReliability::test_intermediate_page_commit_advances_job_liveness_timestamp`. + ## Contract Alignment - Treat `docs/` as the active architecture and requirements baseline. diff --git a/docs/invariant/ai_evidence_and_provenance.md b/docs/invariant/ai_evidence_and_provenance.md index 88c1ea8..5d844f4 100644 --- a/docs/invariant/ai_evidence_and_provenance.md +++ b/docs/invariant/ai_evidence_and_provenance.md @@ -123,6 +123,10 @@ Evaluation should: 5. Preserve the exact model, endpoint or route, parameters, prompt, source digest, and scoring method for every comparison. 6. Treat model rankings as corpus- and version-specific, not permanent declarations of a universal “best” model. +The deterministic scorer for these comparisons lives in `src/transcription/benchmarking.py`; it is +retained as evaluation-policy infrastructure even though application runtime paths do not call it +directly. + Benchmark material containing family records remains private application data unless explicitly approved for publication. ## 6. Ownership and Change Policy diff --git a/src/transcription/benchmarking.py b/src/transcription/benchmarking.py index c2c3908..2b8bfec 100644 --- a/src/transcription/benchmarking.py +++ b/src/transcription/benchmarking.py @@ -1,4 +1,11 @@ -"""Private-corpus benchmark contracts and deterministic text scoring.""" +"""Private-corpus benchmark contracts and deterministic text scoring. + +This module is retained as the implementation of the evaluation policy in +`docs/invariant/ai_evidence_and_provenance.md` §5. Application runtime paths do +not call it directly, but preserved execution attempts and manually reviewed +references need a deterministic scorer that remains importable for tests and +operator tooling. +""" from __future__ import annotations