154 lines
8.9 KiB
Markdown
154 lines
8.9 KiB
Markdown
# Testing Database Targets and Data
|
|
|
|
Use the same application database construction path in production and tests. Tests select a different URL and bind their request-session dependency to a test-scoped transaction; they do not replace repositories, services, or SQLAlchemy mechanics with mocks.
|
|
|
|
## Decision Table
|
|
|
|
| Test need | Database target | Isolation approach | What it proves |
|
|
|---|---|---|---|
|
|
| Fast, serial application tests | `sqlite+aiosqlite://` | Per-test engine or outer transaction | ORM mappings and ordinary application behavior |
|
|
| Async code using multiple simultaneous sessions | Named SQLite shared-cache URL or temporary SQLite file | Per-test schema or cleanup strategy | Concurrent-session behavior without a database server |
|
|
| PostgreSQL-specific behavior | Dedicated PostgreSQL test database | Per-test outer transaction and SAVEPOINT | SQL, constraints, types, locking, and migrations that SQLite cannot represent |
|
|
|
|
SQLite is a useful fast target, not a drop-in PostgreSQL substitute. Keep a small PostgreSQL integration suite for PostgreSQL-specific queries, extensions, row locking, JSON semantics, collations, isolation, and migration validation.
|
|
|
|
## One Construction Path
|
|
|
|
Make the application factory accept a database URL or settings object, and keep engine and session-factory construction in one function. The only test-specific inputs should be the URL and, for request tests, the session dependency override.
|
|
|
|
```python
|
|
from sqlalchemy.ext.asyncio import AsyncEngine, async_sessionmaker, create_async_engine
|
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
|
|
|
|
|
def create_database(
|
|
database_url: str,
|
|
) -> tuple[AsyncEngine, async_sessionmaker[AsyncSession]]:
|
|
engine = create_async_engine(database_url)
|
|
session_factory = async_sessionmaker(
|
|
engine,
|
|
class_=AsyncSession,
|
|
expire_on_commit=False,
|
|
)
|
|
return engine, session_factory
|
|
```
|
|
|
|
Production passes its `postgresql+asyncpg://...` URL to `create_database()`. A local SQLite run passes `sqlite+aiosqlite:///./app.db`. Tests pass a dedicated test URL to the same function. Do not create an engine during module import: that makes it easy for tests to accidentally retain the production URL before an override is applied.
|
|
|
|
Use migrations to provision an integration database when migrations are part of the release contract. `metadata.create_all()` is appropriate for focused ORM tests only when it accurately represents the schema under test. Import all table models before creating metadata; [SQLModel documents that model-registration order matters](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/#import-table-models).
|
|
|
|
## Transactional Async Fixture
|
|
|
|
For tests that exercise code which calls `commit()`, start an outer transaction on one test connection. Bind the test `AsyncSession` to that connection and use `join_transaction_mode="create_savepoint"`. SQLAlchemy documents this as its test-suite pattern: session commits resolve a SAVEPOINT while fixture teardown rolls back the outer transaction.
|
|
|
|
```python
|
|
from collections.abc import AsyncGeneratorr
|
|
|
|
import pytest_asyncio
|
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
|
|
|
|
|
@pytest_asyncio.fixture
|
|
async def session(test_engine: AsyncEngine) -> AsyncGenerator[AsyncSession]:
|
|
async with test_engine.connect() as connection:
|
|
transaction = await connection.begin()
|
|
test_session = AsyncSession(
|
|
bind=connection,
|
|
expire_on_commit=False,
|
|
join_transaction_mode="create_savepoint",
|
|
)
|
|
try:
|
|
yield test_session
|
|
finally:
|
|
await test_session.close()
|
|
await transaction.rollback()
|
|
```
|
|
|
|
Use the test session through the normal FastAPI dependency seam, and always remove the override after the test. [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/) are an application-level dictionary, so leaving one installed leaks test state.
|
|
|
|
```python
|
|
import pytest
|
|
from fastapi import FastAPI
|
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
|
|
|
|
|
@pytest.fixture
|
|
def app_with_test_session(
|
|
app: FastAPI,
|
|
session: AsyncSession,
|
|
) -> FastAPI:
|
|
async def get_test_session() -> AsyncGenerator[AsyncSession]:
|
|
yield session
|
|
|
|
app.dependency_overrides[get_session] = get_test_session
|
|
try:
|
|
yield app
|
|
finally:
|
|
app.dependency_overrides.clear()
|
|
```
|
|
|
|
This fixture is deliberately serial: one mutable `AsyncSession` must not serve concurrent tasks. A test that verifies concurrently active sessions should create independent sessions from a factory and use a database target that supports independent connections.
|
|
|
|
## SQLite Targets
|
|
|
|
### Serial in-memory tests
|
|
|
|
Use `sqlite+aiosqlite://` for a fresh in-memory database when the test runs all database work serially. SQLAlchemy's `aiosqlite` dialect uses a single-connection `StaticPool` for this target, so all sessions share one SQLite transaction state. One session's rollback can discard another session's uncommitted work.
|
|
|
|
Create the schema and dispose the engine deterministically:
|
|
|
|
```python
|
|
import pytest_asyncio
|
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
|
|
|
|
|
@pytest_asyncio.fixture
|
|
async def test_engine() -> AsyncGenerator[AsyncEngine]:
|
|
engine, _ = create_database("sqlite+aiosqlite://")
|
|
async with engine.begin() as connection:
|
|
await connection.run_sync(SQLModel.metadata.create_all)
|
|
try:
|
|
yield engine
|
|
finally:
|
|
await engine.dispose()
|
|
```
|
|
|
|
### Concurrent in-memory tests
|
|
|
|
Do not use the default `:memory:` target for tests that have multiple active sessions or tasks. Use a named shared-cache database instead, with a name unique to the test process:
|
|
|
|
```text
|
|
sqlite+aiosqlite:///file:test-suite?mode=memory&cache=shared&uri=true
|
|
```
|
|
|
|
This lets connections share the same in-memory database while retaining independent transaction state. A temporary file URL such as `sqlite+aiosqlite:////tmp/test.db` is often simpler when test isolation or cleanup tooling already manages files.
|
|
|
|
For both SQLite forms, enable and test the constraints your application depends on. SQLite foreign-key enforcement is disabled by default, and its transaction behavior has driver-specific differences. Keep PostgreSQL integration coverage for behavior that SQLite cannot faithfully model.
|
|
|
|
## Test Data Practices
|
|
|
|
- Build only the data a test needs, through named factory functions or pytest fixtures rather than a large global seed.
|
|
- Give each fixture a domain meaning, such as `active_account`, `expired_subscription`, or `admin_user`; avoid opaque rows with unexplained defaults.
|
|
- Set values relevant to the assertion explicitly, including timestamps, permissions, statuses, and unique identifiers. Use fixed clocks or injected clock values instead of the wall clock.
|
|
- Construct object graphs through relationships, then `await session.flush()` before reading generated identifiers or passing foreign keys onward. `flush()` exercises database constraints without ending the test transaction.
|
|
- Seed prerequisite data before creating a client request. Let the endpoint own the mutation being asserted; do not pre-insert the row that the endpoint is supposed to create.
|
|
- Use `commit()` in fixture setup only when the test specifically needs to prove post-commit behavior. With the transactional fixture, this remains isolated through the outer rollback.
|
|
- Keep shared reference data immutable and explicit. If it must be reused for performance, load it once into a dedicated test database and reset all mutable tables between tests; never depend on test order.
|
|
- Include both valid and constraint-breaking graphs where a behavior depends on foreign keys, uniqueness, nullability, or cascading deletes. SQLite-only tests should not be the sole evidence for PostgreSQL constraints.
|
|
|
|
## Completion Checks
|
|
|
|
- A test run cannot reach the production URL; production credentials are absent from the test environment.
|
|
- Production PostgreSQL, local SQLite, and in-memory SQLite all use the same engine/session-factory construction path.
|
|
- Every test owns its override, connection, transaction, session, and engine cleanup.
|
|
- Test data is deterministic, minimal, and expresses the scenario under test.
|
|
- PostgreSQL integration tests cover every PostgreSQL-specific contract and run against migrations where migrations are shipped.
|
|
|
|
## Sources
|
|
|
|
- [SQLAlchemy: joining a session into an external transaction](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html#joining-a-session-into-an-external-transaction-such-as-for-test-suites)
|
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
|
- [SQLAlchemy SQLite dialect and async in-memory pooling](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#using-a-memory-database-with-multiple-coroutines)
|
|
- [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
|
|
- [SQLModel testing with FastAPI](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/) |