diff --git a/docs/skills/pytest-scaffolding/SKILL.md b/docs/skills/pytest-scaffolding/SKILL.md index f548960..c6d0909 100644 --- a/docs/skills/pytest-scaffolding/SKILL.md +++ b/docs/skills/pytest-scaffolding/SKILL.md @@ -1,6 +1,6 @@ --- name: pytest-scaffolding -description: "Scaffold a maintainable, hierarchical pytest suite with fast defaults and clear escalation paths for FastAPI and SQLAlchemy tests. Use when creating or reorganizing tests, defining fixture/marker boundaries, or making test strategy progressively discoverable." +description: "Reference hub for pytest suite structure, naming, markers, and stack-specific testing patterns. Use when you need best-practice guidance and source links for core pytest, FastAPI testing, and SQLAlchemy testing." argument-hint: "Target scope plus stack details (pure Python, FastAPI, SQLAlchemy sync, SQLAlchemy async, or mixed)" x-personal-mcp: id: pytest-scaffolding @@ -16,111 +16,67 @@ x-personal-mcp: # Pytest Scaffolding -Create test scaffolding that stays fast for daily work and scales safely as dependencies increase. +This skill is a collection of best-practice references and source-documentation links for building and maintaining pytest suites. -This skill is optimized for progressive discoverability: -1. Start with the shortest path in this file. -2. Load exactly one deeper reference only when a decision requires it. -3. Continue only as far as needed for the current task. +Use it to quickly find the right guidance for: +1. Baseline pytest structure and marker strategy. +2. Naming conventions and test hierarchy organization. +3. FastAPI route, dependency override, and lifespan testing patterns. +4. SQLAlchemy transaction and session testing patterns. Repository defaults: - `uv run pytest` is the canonical invocation. - pytest settings live in `pyproject.toml` under `[tool.pytest.ini_options]`. - strict marker checking is expected (`--strict-markers`). -## Discovery Ladder +## Reference Map -### Level 0: Scope And Stack Triage (always) -Collect: -1. Target scope (repo, package, module). -2. Stack shape (pure Python, FastAPI, SQLAlchemy sync, SQLAlchemy async, or mixed). -3. Speed target (what must stay instant). -4. CI gate policy (which marker groups block merge). +Use this map to open only the reference that matches your immediate need. -If any are missing, ask concise clarifying questions before scaffolding. +- Core pytest practices and command patterns: [pytest-docs.md](./references/pytest-docs.md) +- Naming conventions and hierarchy organization: [naming-and-organization.md](./references/naming-and-organization.md) +- FastAPI-specific testing patterns: [fastapi-testing.md](./references/fastapi-testing.md) +- SQLAlchemy-specific testing patterns: [sqlalchemy-testing.md](./references/sqlalchemy-testing.md) -### Level 1: Core pytest scaffold (default) -Use this for all stacks first: -1. Mirror `src/` into `tests/` with one starter file per core module. -2. Classify test intent by cost: - - `unit`: no DB/network/filesystem side effects. - - `integration`: framework, DB, or multi-layer contracts. - - `smoke`: thin critical-path checks. -3. Scaffold each new module with: - - one happy-path test, - - one failure/edge test, - - TODO anchors for deeper assertions. -4. Keep fixtures layered: - - global lightweight fixtures in `tests/conftest.py`, - - domain fixtures in subtree `conftest.py` only when needed. -5. Register markers early: `unit`, `integration`, `smoke`, `slow`, `external`. -6. Validate in order: - - `uv run pytest --collect-only -q` - - `uv run pytest -m unit -q` - - `uv run pytest -q` when dependencies are available. +## Baseline Best Practices -Load next reference only if needed: -- Baseline details and rationale: [pytest-docs.md](./references/pytest-docs.md) +These are stable defaults regardless of stack: -### Level 2: FastAPI branch (only for HTTP/dependency/lifespan concerns) -Escalate here when testing API routes, dependency injection boundaries, or app lifespan behavior. +1. Mirror `src/` into `tests/` so ownership and coverage are obvious. +2. Keep fixtures explicit and layered (`tests/conftest.py` globally, subtree `conftest.py` for domain-specific fixtures). +3. Register markers up front (`unit`, `integration`, `smoke`, `slow`, `external`) and keep strict marker checks enabled. +4. Separate fast feedback (`-m unit`) from broader integration/external lanes. +5. Validate structure early with collection checks before expanding assertions. -Apply these defaults: -1. Prefer `TestClient` with sync `def` tests for route behavior. -2. Use `AsyncClient` + `@pytest.mark.anyio` only when test logic must await other async work. -3. Prefer `app.dependency_overrides` over patching internals. -4. Reset dependency overrides in teardown after every test/fixture. -5. For startup/shutdown semantics: - - use `TestClient` as context manager, or - - use `LifespanManager` with async client. +## Stack-Specific Guidance -Marker intent in FastAPI-heavy suites: -- `unit`: service logic without HTTP/DB. -- `integration`: route + DI + DB contract checks. -- `smoke`: one request per critical user path. +- For FastAPI, prefer dependency overrides and clear lifecycle handling; see [fastapi-testing.md](./references/fastapi-testing.md). +- For SQLAlchemy, prefer transaction-safe session fixtures and explicit async loading strategy; see [sqlalchemy-testing.md](./references/sqlalchemy-testing.md). +- For naming and tree organization, use the conventions in [naming-and-organization.md](./references/naming-and-organization.md). -Reference: [fastapi-testing.md](./references/fastapi-testing.md) +## Source Documentation Entry Points -### Level 3: SQLAlchemy branch (only for DB transaction/session design) -Escalate here when session lifecycle, transaction isolation, or async ORM behavior matters. +Primary upstream docs are curated in each reference page. Start with: -Apply these defaults: -1. Create engine once per test session. -2. Open connection + outer transaction per test. -3. Bind session with `join_transaction_mode="create_savepoint"`. -4. Allow code under test to call `commit()` safely; rollback outer transaction at test end. -5. Keep unit tests DB-free; DB tests belong under `integration`. +1. Pytest good practices: [pytest docs](https://docs.pytest.org/en/stable/explanation/goodpractices.html) +2. Pytest fixtures: [fixture how-to](https://docs.pytest.org/en/stable/how-to/fixtures.html) +3. Pytest markers: [marker examples](https://docs.pytest.org/en/stable/example/markers.html) +4. FastAPI testing: [FastAPI testing tutorial](https://fastapi.tiangolo.com/tutorial/testing/) +5. SQLAlchemy transaction testing: [SQLAlchemy external transaction pattern](https://docs.sqlalchemy.org/en/20/orm/session_transaction.html#joining-a-session-into-an-external-transaction-such-as-for-test-suites) -Async additions: -- use async fixtures and `@pytest.mark.anyio`. -- set `expire_on_commit=False` for `AsyncSession`. -- avoid implicit lazy IO; use eager loading (`selectinload`) or explicit refresh. +## Quick Validation Commands -SQLite in-memory with threaded test clients: -- use `StaticPool` when required by thread/connection sharing. +Use these commands to check structure and execution lanes: -Reference: [sqlalchemy-testing.md](./references/sqlalchemy-testing.md) - -## Branching Logic Summary -- If pure logic can be faked cleanly, keep in `unit`. -- If framework/DB contract is the behavior under test, use `integration`. -- If external service credentials/network is required, gate behind `external`. -- If suite slows down, split by marker before broadening fixture scope. -- If async relationship access raises `MissingGreenlet`, switch to eager loading strategy. - -## Completion Checks -A scaffold pass is complete when all are true: -1. Core source areas map to clear test modules. -2. Fast path (`-m unit`) is deterministic and quick. -3. Integration and external paths are isolated by fixtures and markers. -4. No unregistered-marker failures occur. -5. Structure is understandable without extra oral context. -6. Clear TODO extension points exist for deeper assertions. +1. `uv run pytest --collect-only -q` +2. `uv run pytest -m unit -q` +3. `uv run pytest -m "not external" -q` +4. `uv run pytest -q` ## Output Contract When this skill is applied, return: -1. Proposed test tree diff. -2. Marker and fixture plan. -3. Exact fast-path and full-path commands. -4. Which reference level was loaded and why. -5. Risks or open questions before expanding assertions. +1. Which references were consulted. +2. Recommended structure, naming, fixture, and marker decisions. +3. Exact validation commands. +4. Relevant source-doc links for any non-trivial recommendation. +5. Risks, assumptions, or open questions. diff --git a/docs/skills/pytest-scaffolding/references/naming-and-organization.md b/docs/skills/pytest-scaffolding/references/naming-and-organization.md new file mode 100644 index 0000000..3142818 --- /dev/null +++ b/docs/skills/pytest-scaffolding/references/naming-and-organization.md @@ -0,0 +1,118 @@ +# Pytest Naming Conventions and Test Organization + +!!! info "Primary sources" + - [Good integration practices](https://docs.pytest.org/en/stable/explanation/goodpractices.html) + - [Changing standard (Python) test discovery](https://docs.pytest.org/en/stable/example/pythoncollection.html) + - [How to use fixtures](https://docs.pytest.org/en/stable/how-to/fixtures.html) + - [How to parametrize fixtures and test functions](https://docs.pytest.org/en/stable/how-to/parametrize.html) + - [Marker examples](https://docs.pytest.org/en/stable/example/markers.html) + +## Agent Quick Path +Use this when creating or reorganizing test modules so naming and hierarchy stay predictable. + +1. Mirror the product domain structure in `tests/` so ownership is obvious. +2. Encode broad context in module and class names (`test_*.py`, `Test*`). +3. Keep leaf test names short and behavior-focused (`test_*`). +4. Use `class Test:` only for grouping related scenarios. +5. Place fixtures in the nearest `conftest.py` needed by scope. +6. Separate expensive tests with markers first, directories second. + +## Naming Conventions + +### File and directory naming +- Use lowercase snake_case for test file names: `test_user_service.py`. +- Keep directories domain-oriented and stable over time: `tests/orders/`, `tests/billing/`. +- Prefer descriptive test names over internal ticket numbers or implementation details. + +### Test function naming +- Prefer hierarchical naming: put broad context in folder/module/class, and keep the function name focused on the final assertion. +- Keep pytest discovery prefixes intact: + - modules start with `test_` + - classes start with `Test` + - functions start with `test_` +- Start with user-visible behavior or contract, not private helper names. + +Recommended pattern: +- module: `test_.py` +- class: `Test` or `Test` +- function: `test_` + +Examples: +- Flat (still valid): `test_create_order_rejects_invalid_currency` +- Class-context: `TestOrder -> TestCreate -> test_rejects_invalid_currency` +- Module-context: `test_order.py -> TestCreate -> test_rejects_invalid_currency` +- Module + class context can similarly shorten: + - `test_token.py -> TestRefresh -> test_rotates_session_id` + - `test_user_list.py -> TestListUsers -> test_returns_empty_for_new_tenant` + +### Test class naming +- Use `class Test:` for scenario grouping and context reduction. +- Keep class names noun-focused (`TestOrderService`) rather than action-focused. +- Avoid xUnit style setup inheritance when fixtures can express dependencies directly. + +## Hierarchy and Organization Patterns + +Two patterns work well; choose one and apply it consistently. + +### Pattern A: Source-mirror hierarchy (default for product code ownership) + +```text +src/ + app/ + orders/service.py + billing/invoice.py + +tests/ + app/ + orders/test_service.py + billing/test_invoice.py +``` + +Use this when teams own modules by source path and want direct test-to-source mapping. + +### Pattern B: Cost-lane hierarchy (default for CI policy clarity) + +```text +tests/ + unit/ + orders/test_service.py + integration/ + api/test_orders.py + persistence/test_order_repository.py + smoke/ + test_health.py +``` + +Use this when CI gating is based on cost lanes and marker filtering. + +### Hybrid rule (recommended) +- Keep a source-mirror tree for local ownership. +- Add markers (`unit`, `integration`, `smoke`, `external`) for runtime policy. +- Avoid duplicating both trees unless the repository already requires it. + +## Fixture Placement Strategy +- Put universal lightweight fixtures in `tests/conftest.py`. +- Put domain fixtures in subtree `conftest.py` files close to where they are used. +- Keep fixtures composable and explicit; avoid large fixture "god objects". +- Use `yield` fixtures for teardown so cleanup is always paired with setup. + +## Parametrize and ID Naming +- Use `pytest.mark.parametrize` for behavior matrices instead of copy/paste tests. +- Provide explicit `ids=` labels when case names are not obvious. +- Keep IDs business-meaningful (`"expired-token"`, `"zero-balance"`) so failures are readable. + +## Collection and Structure Checks +Use these checks after introducing new test files or renaming modules: + +- `uv run pytest --collect-only -q` +- `uv run pytest -m unit -q` +- `uv run pytest -m "not external" -q` + +If collection surprises appear, verify file names, marker registration, and directory placement first. + +## Common Anti-Patterns +- Mixed naming styles (`testFoo.py`, `test_foo.py`, `foo_test.py`) in one repository. +- Deep fixture chains that hide setup behavior. +- Test names that encode implementation details instead of behavior. +- Moving slow tests into `unit` directories without marker updates. +- Sharing mutable module-level state across tests. \ No newline at end of file diff --git a/docs/skills/pytest-scaffolding/references/pytest-docs.md b/docs/skills/pytest-scaffolding/references/pytest-docs.md index 21c54a7..e12913b 100644 --- a/docs/skills/pytest-scaffolding/references/pytest-docs.md +++ b/docs/skills/pytest-scaffolding/references/pytest-docs.md @@ -19,6 +19,7 @@ Use this file when you need fast pytest scaffolding defaults without framework-s Load other references only when needed: - FastAPI routes/dependency injection/lifespan: `fastapi-testing.md` - SQLAlchemy sessions/transactions/DB fixtures: `sqlalchemy-testing.md` +- Naming conventions and test hierarchy: `naming-and-organization.md` ## Practical Guidance For This Skill - Use src-aligned test layout and keep test discovery conventional.