---
description: Authoring rules for the test suite, including markers, async discipline, and guard-test design.
applyTo: 'tests/**/*.py'
---

# Tests

Primary references:

- `AGENTS.md` (Change Protocol — failing test first)
- `docs/index.md` and `docs/invariant/*`
- `.github/skills/test-effectiveness-auditor/skill.md` (periodic audit of this suite)

The suite is not only regression protection here — it is where several architectural rules are
*defined*. `tests/test_service_boundaries.py`, `tests/test_ui_boundaries.py`,
`tests/test_provider_boundaries.py`, `tests/test_model_contract_guards.py`, and
`tests/test_meta_contract_guards.py` are the enforcement layer named in the `AGENTS.md` authority
order. A weak test in this repository does not merely fail to catch a bug; it can silently repeal a
documented invariant.

The baseline is green. `uv run pytest -q -m "not external"` must report zero failures and zero
errors, and there is no tolerated set of known-failing tests.

## Write the Failing Test First

For any behavioral fix, write the test before the fix and confirm it fails *for the reason you
expect*. A test that passes against the broken code proves nothing, and several defects in this
repository were subtle enough that a test written afterward would have done exactly that. If the
new test passes immediately, you have not reproduced the defect yet.

## Runner Configuration

Configured in `pyproject.toml`; do not work around these:

- `--strict-markers` — an unregistered marker is an error. Register new markers in
  `[tool.pytest.ini_options] markers` with a description rather than inventing one at the call site.
- `asyncio_mode = "strict"` — every async test needs an explicit `@pytest.mark.asyncio`, and async
  fixtures use `@pytest_asyncio.fixture`. There is no implicit promotion.
- `filterwarnings = ["error:coroutine .* was never awaited:RuntimeWarning"]` — an un-awaited
  coroutine is an error, not a warning. This usually means a mock replaced an async callable with a
  sync one, or an `await` was dropped. Fix the call; never silence the warning.

## Markers and Layout

- `unit` — pure logic, no framework or database.
- `integration` — touches framework, database, or multi-component contracts.
- `external` — calls live services; slow and credential-dependent.

`external` tests must also carry their own `skipif` so the suite stays green without credentials
(see `tests/services/test_transcription_external.py`). Local and documented runs use
`-m "not external"`; CI intentionally runs unfiltered, which is equivalent because those tests skip
themselves. Never let an unmarked test reach the network.

Place tests by the layer under test: `tests/services/`, `tests/ui/`, `tests/api/`,
`tests/providers/`, `tests/integration/`, with cross-cutting guards at the top level.

## Fixtures and Isolation

- Prefer the shared fixtures in `tests/conftest.py` (`default_settings`, `async_session`,
  `default_session_factory`, and the per-aggregate service fixtures) over building settings or
  engines by hand.
- `Settings` is isolated suite-wide by the session-scoped autouse fixture in `conftest.py`, because
  `env_file` resolves against the working directory. Tests that need env-file loading pass
  `_env_file=` explicitly; tests asserting declared defaults need nothing. Do not reintroduce
  reliance on a developer's local env file. Guarded by `tests/test_config_isolation.py`.
- Database fixtures refuse to run against anything but the per-test path, and that refusal is
  deliberate. Never relax it to point a destructive fixture at a real database.
- Tests must not leave artifacts outside `tmp_path`.

## Assertion Strength

Assert on the domain effect, not on the fact that code ran.

- Prefer persisted state, status transitions, error categories, and evidence records over
  "no exception raised", "not None", or a bare status code.
- **Read committed state through a separate session.** Asserting against the same session that
  performed the write can pass on unflushed in-memory state and prove nothing about durability.
  This is how the atomicity guarantees in `tests/services/test_workflows_reliability.py` and
  `tests/integration/test_pipeline_atomicity.py` are made real.
- Critical paths need negative-path coverage — timeouts, provider failures, validation errors,
  cancellation. Happy-path-only coverage of a critical module is a gap, not a suite.
- Avoid count-threshold assertions as a proxy for correctness. A test asserting "at least N items
  were discovered" passes indefinitely while the thing it was meant to protect rots; assert on a
  specific known member instead.

## Guard Tests

Structural guards carry extra obligations, because they are cited as proof that a rule holds.

- **Guard the guard.** Every scanning guard needs a companion assertion that the scan actually found
  something, following the existing `test_*_are_discovered` pattern. A guard that silently scans an
  empty set passes forever.
- **Scope must match the claim.** A guard's name and docstring must describe only what it actually
  verifies. A test covering one function while appearing to enforce a repo-wide rule is worse than
  no test, because it stops anyone from writing the real one.
- **Prove non-vacuity by injected fault.** Temporarily introduce the violation, confirm the guard
  fails with a comprehensible message, then revert. Do this whenever you add or materially change a
  guard. Revert with an explicit edit if the file has uncommitted changes — `git checkout --` will
  discard them.
- **Prefer structural analysis to substring matching.** AST inspection of imports and definitions is
  resistant to false negatives; a bare-name search across the repository is not, since an unrelated
  mention anywhere makes dead code look reachable.
- Failure messages should name the offending file, symbol, and the remedy. These fire for people who
  did not write the guard.
- Any new file under `.github/**` must be added to `ACTIVE_CONTRACT_FILES` in
  `tests/test_meta_contract_guards.py`, or the completeness guard fails by design.

## Redundancy

Duplicate coverage across layers costs runtime and dilutes signal. Pick the canonical layer for a
behavior — unit for logic, integration for wiring — and let the other layer assert only what is
unique to it. Retire tests superseded by a stronger guard instead of accumulating both, and record
deliberate retentions with a rationale rather than leaving them unexplained.

## Contract Sync Rule

When a test encodes or relaxes a documented rule, update the corresponding instruction file or
`docs/*` page in the same change. When a guard test is the enforcement for a rule stated in
`AGENTS.md` or an instruction file, cite the test by name there so the link survives refactoring.