testing page

2026-06-20 20:34:10 -05:00
parent f8e0c14d46
commit 82b50fb63b
1 changed files with 177 additions and 1 deletions
@@ -1 +1,177 @@
 # Testing
+
+This page defines an initial, high-level test structure for this repository that is easy to grow over time while keeping local feedback fast.
+
+Primary guidance sources:
+- [Pytest scaffolding skill](./skills/pytest-scaffolding/SKILL.md)
+- [FastAPI testing reference](./skills/pytest-scaffolding/references/fastapi-testing.md)
+- [FastAPI + uv + Docker skill](./skills/fastapi-uv-docker/SKILL.md)
+
+## Goals
+
+1. Keep the default developer test loop fast and deterministic.
+2. Separate cheap tests from expensive tests with clear markers.
+3. Mirror production code layout so new tests are easy to place.
+4. Make FastAPI dependency and lifespan testing explicit.
+5. Leave clear extension points for database and external integrations.
+
+## Naming And Layout Conventions
+
+Use these naming rules consistently:
+- test files: `test_<subject>.py`
+- test functions: `test_<behavior>_<expected_result>()`
+- test classes (optional): `Test<Subject>`
+- fixtures: `<scope>_<resource>` (for example `session_engine`, `api_client`)
+
+Target end-state test tree:
+
+```text
+tests/
+	conftest.py
+	unit/
+		test_main.py
+		test_mcp.py
+		catalog/
+			test_server.py
+		skills/
+			test_document_loader.py
+			test_document_loader_references.py
+		web/
+			test_config.py
+			test_docs_mount.py
+	integration/
+		api/
+			test_health_endpoint.py
+			test_app_routes.py
+		catalog/
+			test_prompt_loader_and_catalog.py
+			test_step6_tool_fallback.py
+		startup/
+			test_lifespan.py
+	smoke/
+		test_service_boot_and_health.py
+	fixtures/
+		__init__.py
+		factories.py
+		payloads.py
+	resources/
+		prompts/
+		skills/
+		catalog/
+		configs/
+```
+
+Mapping rule:
+- mirror `src/personal_mcp/` into `tests/unit/` first.
+- add integration modules only where contracts exist (FastAPI routes, DB behavior, external adapters).
+
+Current to target mapping in this repo:
+- `tests/test_document_loader_references.py` -> `tests/unit/skills/test_document_loader_references.py`
+- `tests/test_prompt_loader_and_catalog.py` -> `tests/integration/catalog/test_prompt_loader_and_catalog.py`
+- `tests/test_step6_tool_fallback.py` -> `tests/integration/catalog/test_step6_tool_fallback.py`
+
+## Marker Strategy
+
+Use a small marker vocabulary from day one:
+- `unit`: pure logic, no DB/network/filesystem side effects.
+- `integration`: framework wiring and/or DB contract checks.
+- `smoke`: thin checks for critical paths.
+- `slow`: expensive tests that should not run in every local loop.
+- `external`: real third-party calls; typically excluded in CI by default.
+
+Fast-path expectation:
+- local default loop should prioritize `unit` (and optionally `smoke`).
+
+## Fixture Layering
+
+1. Keep lightweight, global fixtures in `tests/conftest.py`.
+2. Add subtree `conftest.py` files only when a test group needs dedicated setup.
+3. Prefer fixture composition over large monolithic fixtures.
+4. Always clean up FastAPI dependency overrides in fixture teardown.
+
+Suggested early fixtures:
+- `settings_override`
+- `api_client`
+- `tmp_workspace`
+
+Optional integration fixtures (add when needed):
+- `session_engine` / `db_session`
+- `async_engine` / `async_session`
+
+## FastAPI Testing Structure
+
+Default route tests:
+- use `fastapi.testclient.TestClient` for standard route behavior tests.
+- keep tests as sync `def` unless async behavior must be asserted.
+
+Async route tests:
+- use `httpx.AsyncClient` with `ASGITransport` and `@pytest.mark.anyio`.
+
+Dependency testing:
+- prefer `app.dependency_overrides` for `Depends(...)` seams.
+- reset `app.dependency_overrides` after each test/fixture.
+
+Lifespan behavior:
+- use `TestClient(app)` as a context manager for startup/shutdown checks.
+
+## Test Granularity By Layer
+
+`unit/`:
+- parser and loader behavior
+- catalog indexing and filtering logic
+- utility functions and pure transformations
+
+`integration/api/`:
+- endpoint request/response contracts
+- dependency override behavior
+- error mapping and status code assertions
+
+`integration/db/` (future-ready):
+- transaction boundaries
+- commit/rollback semantics
+- async session lifecycle behavior
+
+`smoke/`:
+- one request/assertion path per critical workflow
+
+## Command Baseline
+
+Canonical invocation in this repository:
+
+```bash
+uv run pytest
+```
+
+Recommended execution sequence:
+
+```bash
+uv run pytest --collect-only -q
+uv run pytest -m unit -q
+uv run pytest -m "unit or smoke" -q
+uv run pytest -q
+```
+
+## Extension Plan
+
+When adding a new feature:
+1. Add or update corresponding `unit` tests first.
+2. Add an `integration` test only for real boundary/contracts.
+3. Add/adjust fixtures at the narrowest useful scope.
+4. Add a `smoke` test only for user-critical paths.
+
+When the suite grows:
+1. Split slower groups behind `slow` and/or `external` markers.
+2. Keep `unit` runtime bounded for rapid local feedback.
+3. Promote shared setup into fixtures only after repeated duplication.
+
+## Rollout Order
+
+Use this order to reach the end-state structure with minimal disruption:
+1. Create `tests/unit/` and `tests/integration/` subtrees.
+2. Move the 3 existing top-level tests into the mapped target paths.
+3. Add `tests/unit` modules mirroring uncovered source modules.
+4. Add API and lifespan integration tests under `tests/integration/api/` and `tests/integration/startup/`.
+5. Add one smoke test for boot + health path.
+6. Add marker registration in `pyproject.toml` as marker usage expands.
+
+This gives a concrete, stable final organization that remains easy to extend as new modules and boundaries are added.