generated from john/python-template
This commit is contained in:
@@ -173,6 +173,16 @@ pages before it. Enforced by `tests/integration/test_pipeline_atomicity.py`; per
|
|||||||
durability is separately enforced by
|
durability is separately enforced by
|
||||||
`tests/services/test_workflows_reliability.py::TestWorkflowReliability::test_transcribed_page_is_committed_before_next_provider_call_finishes`.
|
`tests/services/test_workflows_reliability.py::TestWorkflowReliability::test_transcribed_page_is_committed_before_next_provider_call_finishes`.
|
||||||
|
|
||||||
|
### Stale-reclaim safety
|
||||||
|
|
||||||
|
- `WORKER_STALE_JOB_SECONDS` must remain **greater than** `WORKER_PROVIDER_TIMEOUT_SECONDS`; stale
|
||||||
|
recovery must not be able to fire before one provider call can legitimately finish.
|
||||||
|
- Long-running multi-page orchestration must refresh job liveness explicitly between intermediate
|
||||||
|
page commits. Do not rely on incidental row updates or provider metadata writes to keep
|
||||||
|
`Job.date_updated` fresh.
|
||||||
|
- Enforced by `tests/test_config.py` and
|
||||||
|
`tests/services/test_workflows_reliability.py::TestWorkflowReliability::test_intermediate_page_commit_advances_job_liveness_timestamp`.
|
||||||
|
|
||||||
## Contract Alignment
|
## Contract Alignment
|
||||||
|
|
||||||
- Treat `docs/` as the active architecture and requirements baseline.
|
- Treat `docs/` as the active architecture and requirements baseline.
|
||||||
|
|||||||
@@ -123,6 +123,10 @@ Evaluation should:
|
|||||||
5. Preserve the exact model, endpoint or route, parameters, prompt, source digest, and scoring method for every comparison.
|
5. Preserve the exact model, endpoint or route, parameters, prompt, source digest, and scoring method for every comparison.
|
||||||
6. Treat model rankings as corpus- and version-specific, not permanent declarations of a universal “best” model.
|
6. Treat model rankings as corpus- and version-specific, not permanent declarations of a universal “best” model.
|
||||||
|
|
||||||
|
The deterministic scorer for these comparisons lives in `src/transcription/benchmarking.py`; it is
|
||||||
|
retained as evaluation-policy infrastructure even though application runtime paths do not call it
|
||||||
|
directly.
|
||||||
|
|
||||||
Benchmark material containing family records remains private application data unless explicitly approved for publication.
|
Benchmark material containing family records remains private application data unless explicitly approved for publication.
|
||||||
|
|
||||||
## 6. Ownership and Change Policy
|
## 6. Ownership and Change Policy
|
||||||
|
|||||||
@@ -1,4 +1,11 @@
|
|||||||
"""Private-corpus benchmark contracts and deterministic text scoring."""
|
"""Private-corpus benchmark contracts and deterministic text scoring.
|
||||||
|
|
||||||
|
This module is retained as the implementation of the evaluation policy in
|
||||||
|
`docs/invariant/ai_evidence_and_provenance.md` §5. Application runtime paths do
|
||||||
|
not call it directly, but preserved execution attempts and manually reviewed
|
||||||
|
references need a deterministic scorer that remains importable for tests and
|
||||||
|
operator tooling.
|
||||||
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user