john/prompts
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

pytest naming convention

Browse Source
This commit is contained in:
John Lancaster
2026-06-20 20:02:03 -05:00
parent 3c5efc6018
commit 7f672b9c8f
3 changed files with 161 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/skills/pytest-scaffolding/SKILL.md
+42 -86
View File
@@ -1,6 +1,6 @@
---
name: pytest-scaffolding
description: "Scaffold a maintainable, hierarchical pytest suite with fast defaults and clear escalation paths for FastAPI and SQLAlchemy tests. Use when creating or reorganizing tests, defining fixture/marker boundaries, or making test strategy progressively discoverable."
description: "Reference hub for pytest suite structure, naming, markers, and stack-specific testing patterns. Use when you need best-practice guidance and source links for core pytest, FastAPI testing, and SQLAlchemy testing."
argument-hint: "Target scope plus stack details (pure Python, FastAPI, SQLAlchemy sync, SQLAlchemy async, or mixed)"
x-personal-mcp:
id: pytest-scaffolding
@@ -16,111 +16,67 @@ x-personal-mcp:
# Pytest Scaffolding
Create test scaffolding that stays fast for daily work and scales safely as dependencies increase.
This skill is a collection of best-practice references and source-documentation links for building and maintaining pytest suites.
This skill is optimized for progressive discoverability:
1. Start with the shortest path in this file.
2. Load exactly one deeper reference only when a decision requires it.
3. Continue only as far as needed for the current task.
Use it to quickly find the right guidance for:
1. Baseline pytest structure and marker strategy.
2. Naming conventions and test hierarchy organization.
3. FastAPI route, dependency override, and lifespan testing patterns.
4. SQLAlchemy transaction and session testing patterns.
Repository defaults:
- `uv run pytest` is the canonical invocation.
- pytest settings live in `pyproject.toml` under `[tool.pytest.ini_options]`.
- strict marker checking is expected (`--strict-markers`).
## Discovery Ladder
## Reference Map
### Level 0: Scope And Stack Triage (always)
Collect:
1. Target scope (repo, package, module).
2. Stack shape (pure Python, FastAPI, SQLAlchemy sync, SQLAlchemy async, or mixed).
3. Speed target (what must stay instant).
4. CI gate policy (which marker groups block merge).
Use this map to open only the reference that matches your immediate need.
If any are missing, ask concise clarifying questions before scaffolding.
- Core pytest practices and command patterns: [pytest-docs.md](./references/pytest-docs.md)
- Naming conventions and hierarchy organization: [naming-and-organization.md](./references/naming-and-organization.md)
- FastAPI-specific testing patterns: [fastapi-testing.md](./references/fastapi-testing.md)
- SQLAlchemy-specific testing patterns: [sqlalchemy-testing.md](./references/sqlalchemy-testing.md)
### Level 1: Core pytest scaffold (default)
Use this for all stacks first:
1. Mirror `src/` into `tests/` with one starter file per core module.
2. Classify test intent by cost:
- `unit`: no DB/network/filesystem side effects.
- `integration`: framework, DB, or multi-layer contracts.
- `smoke`: thin critical-path checks.
3. Scaffold each new module with:
- one happy-path test,
- one failure/edge test,
- TODO anchors for deeper assertions.
4. Keep fixtures layered:
- global lightweight fixtures in `tests/conftest.py`,
- domain fixtures in subtree `conftest.py` only when needed.
5. Register markers early: `unit`, `integration`, `smoke`, `slow`, `external`.
6. Validate in order:
- `uv run pytest --collect-only -q`
- `uv run pytest -m unit -q`
- `uv run pytest -q` when dependencies are available.
## Baseline Best Practices
Load next reference only if needed:
- Baseline details and rationale: [pytest-docs.md](./references/pytest-docs.md)
These are stable defaults regardless of stack:
### Level 2: FastAPI branch (only for HTTP/dependency/lifespan concerns)
Escalate here when testing API routes, dependency injection boundaries, or app lifespan behavior.
1. Mirror `src/` into `tests/` so ownership and coverage are obvious.
2. Keep fixtures explicit and layered (`tests/conftest.py` globally, subtree `conftest.py` for domain-specific fixtures).
3. Register markers up front (`unit`, `integration`, `smoke`, `slow`, `external`) and keep strict marker checks enabled.
4. Separate fast feedback (`-m unit`) from broader integration/external lanes.
5. Validate structure early with collection checks before expanding assertions.
Apply these defaults:
1. Prefer `TestClient` with sync `def` tests for route behavior.
2. Use `AsyncClient` + `@pytest.mark.anyio` only when test logic must await other async work.
3. Prefer `app.dependency_overrides` over patching internals.
4. Reset dependency overrides in teardown after every test/fixture.
5. For startup/shutdown semantics:
- use `TestClient` as context manager, or
- use `LifespanManager` with async client.
## Stack-Specific Guidance
Marker intent in FastAPI-heavy suites:
- `unit`: service logic without HTTP/DB.
- `integration`: route + DI + DB contract checks.
- `smoke`: one request per critical user path.
- For FastAPI, prefer dependency overrides and clear lifecycle handling; see [fastapi-testing.md](./references/fastapi-testing.md).
- For SQLAlchemy, prefer transaction-safe session fixtures and explicit async loading strategy; see [sqlalchemy-testing.md](./references/sqlalchemy-testing.md).
- For naming and tree organization, use the conventions in [naming-and-organization.md](./references/naming-and-organization.md).
Reference: [fastapi-testing.md](./references/fastapi-testing.md)
## Source Documentation Entry Points
### Level 3: SQLAlchemy branch (only for DB transaction/session design)
Escalate here when session lifecycle, transaction isolation, or async ORM behavior matters.
Primary upstream docs are curated in each reference page. Start with:
Apply these defaults:
1. Create engine once per test session.
2. Open connection + outer transaction per test.
3. Bind session with `join_transaction_mode="create_savepoint"`.
4. Allow code under test to call `commit()` safely; rollback outer transaction at test end.
5. Keep unit tests DB-free; DB tests belong under `integration`.
1. Pytest good practices: [pytest docs](https://docs.pytest.org/en/stable/explanation/goodpractices.html)
2. Pytest fixtures: [fixture how-to](https://docs.pytest.org/en/stable/how-to/fixtures.html)
3. Pytest markers: [marker examples](https://docs.pytest.org/en/stable/example/markers.html)
4. FastAPI testing: [FastAPI testing tutorial](https://fastapi.tiangolo.com/tutorial/testing/)
5. SQLAlchemy transaction testing: [SQLAlchemy external transaction pattern](https://docs.sqlalchemy.org/en/20/orm/session_transaction.html#joining-a-session-into-an-external-transaction-such-as-for-test-suites)
Async additions:
- use async fixtures and `@pytest.mark.anyio`.
- set `expire_on_commit=False` for `AsyncSession`.
- avoid implicit lazy IO; use eager loading (`selectinload`) or explicit refresh.
## Quick Validation Commands
SQLite in-memory with threaded test clients:
- use `StaticPool` when required by thread/connection sharing.
Use these commands to check structure and execution lanes:
Reference: [sqlalchemy-testing.md](./references/sqlalchemy-testing.md)
## Branching Logic Summary
- If pure logic can be faked cleanly, keep in `unit`.
- If framework/DB contract is the behavior under test, use `integration`.
- If external service credentials/network is required, gate behind `external`.
- If suite slows down, split by marker before broadening fixture scope.
- If async relationship access raises `MissingGreenlet`, switch to eager loading strategy.
## Completion Checks
A scaffold pass is complete when all are true:
1. Core source areas map to clear test modules.
2. Fast path (`-m unit`) is deterministic and quick.
3. Integration and external paths are isolated by fixtures and markers.
4. No unregistered-marker failures occur.
5. Structure is understandable without extra oral context.
6. Clear TODO extension points exist for deeper assertions.
1. `uv run pytest --collect-only -q`
2. `uv run pytest -m unit -q`
3. `uv run pytest -m "not external" -q`
4. `uv run pytest -q`
## Output Contract
When this skill is applied, return:
1. Proposed test tree diff.
2. Marker and fixture plan.
3. Exact fast-path and full-path commands.
4. Which reference level was loaded and why.
5. Risks or open questions before expanding assertions.
1. Which references were consulted.
2. Recommended structure, naming, fixture, and marker decisions.
3. Exact validation commands.
4. Relevant source-doc links for any non-trivial recommendation.
5. Risks, assumptions, or open questions.
docs/skills/pytest-scaffolding/references/naming-and-organization.md
+118
View File
@@ -0,0 +1,118 @@
# Pytest Naming Conventions and Test Organization
!!! info "Primary sources"
- [Good integration practices](https://docs.pytest.org/en/stable/explanation/goodpractices.html)
- [Changing standard (Python) test discovery](https://docs.pytest.org/en/stable/example/pythoncollection.html)
- [How to use fixtures](https://docs.pytest.org/en/stable/how-to/fixtures.html)
- [How to parametrize fixtures and test functions](https://docs.pytest.org/en/stable/how-to/parametrize.html)
- [Marker examples](https://docs.pytest.org/en/stable/example/markers.html)
## Agent Quick Path
Use this when creating or reorganizing test modules so naming and hierarchy stay predictable.
1. Mirror the product domain structure in `tests/` so ownership is obvious.
2. Encode broad context in module and class names (`test_*.py`, `Test*`).
3. Keep leaf test names short and behavior-focused (`test_*`).
4. Use `class Test<Subject>:` only for grouping related scenarios.
5. Place fixtures in the nearest `conftest.py` needed by scope.
6. Separate expensive tests with markers first, directories second.
## Naming Conventions
### File and directory naming
- Use lowercase snake_case for test file names: `test_user_service.py`.
- Keep directories domain-oriented and stable over time: `tests/orders/`, `tests/billing/`.
- Prefer descriptive test names over internal ticket numbers or implementation details.
### Test function naming
- Prefer hierarchical naming: put broad context in folder/module/class, and keep the function name focused on the final assertion.
- Keep pytest discovery prefixes intact:
- modules start with `test_`
- classes start with `Test`
- functions start with `test_`
- Start with user-visible behavior or contract, not private helper names.
Recommended pattern:
- module: `test_<subject>.py`
- class: `Test<Operation>` or `Test<Scenario>`
- function: `test_<expected_outcome>`
Examples:
- Flat (still valid): `test_create_order_rejects_invalid_currency`
- Class-context: `TestOrder -> TestCreate -> test_rejects_invalid_currency`
- Module-context: `test_order.py -> TestCreate -> test_rejects_invalid_currency`
- Module + class context can similarly shorten:
- `test_token.py -> TestRefresh -> test_rotates_session_id`
- `test_user_list.py -> TestListUsers -> test_returns_empty_for_new_tenant`
### Test class naming
- Use `class Test<SubjectOrScenario>:` for scenario grouping and context reduction.
- Keep class names noun-focused (`TestOrderService`) rather than action-focused.
- Avoid xUnit style setup inheritance when fixtures can express dependencies directly.
## Hierarchy and Organization Patterns
Two patterns work well; choose one and apply it consistently.
### Pattern A: Source-mirror hierarchy (default for product code ownership)
```text
src/
app/
orders/service.py
billing/invoice.py
tests/
app/
orders/test_service.py
billing/test_invoice.py
```
Use this when teams own modules by source path and want direct test-to-source mapping.
### Pattern B: Cost-lane hierarchy (default for CI policy clarity)
```text
tests/
unit/
orders/test_service.py
integration/
api/test_orders.py
persistence/test_order_repository.py
smoke/
test_health.py
```
Use this when CI gating is based on cost lanes and marker filtering.
### Hybrid rule (recommended)
- Keep a source-mirror tree for local ownership.
- Add markers (`unit`, `integration`, `smoke`, `external`) for runtime policy.
- Avoid duplicating both trees unless the repository already requires it.
## Fixture Placement Strategy
- Put universal lightweight fixtures in `tests/conftest.py`.
- Put domain fixtures in subtree `conftest.py` files close to where they are used.
- Keep fixtures composable and explicit; avoid large fixture "god objects".
- Use `yield` fixtures for teardown so cleanup is always paired with setup.
## Parametrize and ID Naming
- Use `pytest.mark.parametrize` for behavior matrices instead of copy/paste tests.
- Provide explicit `ids=` labels when case names are not obvious.
- Keep IDs business-meaningful (`"expired-token"`, `"zero-balance"`) so failures are readable.
## Collection and Structure Checks
Use these checks after introducing new test files or renaming modules:
- `uv run pytest --collect-only -q`
- `uv run pytest -m unit -q`
- `uv run pytest -m "not external" -q`
If collection surprises appear, verify file names, marker registration, and directory placement first.
## Common Anti-Patterns
- Mixed naming styles (`testFoo.py`, `test_foo.py`, `foo_test.py`) in one repository.
- Deep fixture chains that hide setup behavior.
- Test names that encode implementation details instead of behavior.
- Moving slow tests into `unit` directories without marker updates.
- Sharing mutable module-level state across tests.
docs/skills/pytest-scaffolding/references/pytest-docs.md
+1
View File
@@ -19,6 +19,7 @@ Use this file when you need fast pytest scaffolding defaults without framework-s
Load other references only when needed:
- FastAPI routes/dependency injection/lifespan: `fastapi-testing.md`
- SQLAlchemy sessions/transactions/DB fixtures: `sqlalchemy-testing.md`
- Naming conventions and test hierarchy: `naming-and-organization.md`
## Practical Guidance For This Skill
- Use src-aligned test layout and keep test discovery conventional.