generated from john/python-template
This commit is contained in:
@@ -173,6 +173,16 @@ pages before it. Enforced by `tests/integration/test_pipeline_atomicity.py`; per
|
||||
durability is separately enforced by
|
||||
`tests/services/test_workflows_reliability.py::TestWorkflowReliability::test_transcribed_page_is_committed_before_next_provider_call_finishes`.
|
||||
|
||||
### Stale-reclaim safety
|
||||
|
||||
- `WORKER_STALE_JOB_SECONDS` must remain **greater than** `WORKER_PROVIDER_TIMEOUT_SECONDS`; stale
|
||||
recovery must not be able to fire before one provider call can legitimately finish.
|
||||
- Long-running multi-page orchestration must refresh job liveness explicitly between intermediate
|
||||
page commits. Do not rely on incidental row updates or provider metadata writes to keep
|
||||
`Job.date_updated` fresh.
|
||||
- Enforced by `tests/test_config.py` and
|
||||
`tests/services/test_workflows_reliability.py::TestWorkflowReliability::test_intermediate_page_commit_advances_job_liveness_timestamp`.
|
||||
|
||||
## Contract Alignment
|
||||
|
||||
- Treat `docs/` as the active architecture and requirements baseline.
|
||||
|
||||
@@ -123,6 +123,10 @@ Evaluation should:
|
||||
5. Preserve the exact model, endpoint or route, parameters, prompt, source digest, and scoring method for every comparison.
|
||||
6. Treat model rankings as corpus- and version-specific, not permanent declarations of a universal “best” model.
|
||||
|
||||
The deterministic scorer for these comparisons lives in `src/transcription/benchmarking.py`; it is
|
||||
retained as evaluation-policy infrastructure even though application runtime paths do not call it
|
||||
directly.
|
||||
|
||||
Benchmark material containing family records remains private application data unless explicitly approved for publication.
|
||||
|
||||
## 6. Ownership and Change Policy
|
||||
|
||||
@@ -1,4 +1,11 @@
|
||||
"""Private-corpus benchmark contracts and deterministic text scoring."""
|
||||
"""Private-corpus benchmark contracts and deterministic text scoring.
|
||||
|
||||
This module is retained as the implementation of the evaluation policy in
|
||||
`docs/invariant/ai_evidence_and_provenance.md` §5. Application runtime paths do
|
||||
not call it directly, but preserved execution attempts and manually reviewed
|
||||
references need a deterministic scorer that remains importable for tests and
|
||||
operator tooling.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
Reference in New Issue
Block a user