176 lines
11 KiB
Markdown
176 lines
11 KiB
Markdown
# Testing Database Targets and Data
|
|
|
|
Use the same engine and session primitives in production and tests. Tests select a different URL and, when transaction isolation is required, bind a test session factory to one test-owned connection and outer transaction. They do not replace repositories, services, or SQLAlchemy mechanics with mocks.
|
|
|
|
## Decision Table
|
|
|
|
| Test need | Database target | Isolation approach | What it proves |
|
|
|---|---|---|---|
|
|
| Fast, serial application tests | `sqlite+aiosqlite://` | Per-test engine or connection-bound session factory over an outer transaction | ORM mappings and ordinary application behavior |
|
|
| Async code using multiple simultaneous sessions | Named SQLite shared-cache URL or temporary SQLite file | Per-test schema or cleanup strategy | Concurrent-session behavior without a database server |
|
|
| PostgreSQL-specific behavior | Dedicated PostgreSQL test database | Per-test outer transaction and SAVEPOINT | SQL, constraints, types, locking, and migrations that SQLite cannot represent |
|
|
|
|
SQLite is a useful fast target, not a drop-in PostgreSQL substitute. Keep a small PostgreSQL integration suite for PostgreSQL-specific queries, extensions, row locking, JSON semantics, collations, isolation, and migration validation.
|
|
|
|
## Shared Construction Primitives
|
|
|
|
Make the application factory accept a database URL or settings object. Production, workers, and ordinary integration tests enter the same [`database_scope()`](engine.md#one-engine-context-manager). Tests enter the lower-level `engine_scope()` only when they need direct engine or connection ownership for schema setup, an outer transaction, or engine-specific assertions:
|
|
|
|
```python
|
|
from collections.abc import AsyncGenerator
|
|
|
|
import pytest_asyncio
|
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
|
|
|
from .engine import engine_scope
|
|
|
|
|
|
@pytest_asyncio.fixture
|
|
async def test_engine(database_url: str) -> AsyncGenerator[AsyncEngine]:
|
|
async with engine_scope(database_url) as engine:
|
|
yield engine
|
|
```
|
|
|
|
Production passes its `postgresql+asyncpg://...` URL to `database_scope()`. A local SQLite run passes `sqlite+aiosqlite:///./app.db`. Tests pass a dedicated test URL to `database_scope()` or `engine_scope()` and receive deterministic disposal when the context exits. Do not create an engine during module import: that makes it easy for tests to retain the production URL before an override is applied.
|
|
|
|
Use migrations to provision an integration database when migrations are part of the release contract. `metadata.create_all()` is appropriate for focused ORM tests only when it accurately represents the schema under test. Import all table models before creating metadata; [SQLModel documents that model-registration order matters](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/#import-table-models).
|
|
|
|
## Transactional Async Fixture
|
|
|
|
For tests that exercise code which commits, start an outer transaction on one test connection. Bind a test `SessionFactory` to that connection with `join_transaction_mode="create_savepoint"`. SQLAlchemy documents this as its test-suite pattern: sessions created by the factory resolve their commits through SAVEPOINTs while fixture teardown rolls back the outer transaction.
|
|
|
|
```python
|
|
from collections.abc import AsyncGenerator
|
|
|
|
import pytest_asyncio
|
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
|
from sqlalchemy.ext.asyncio import async_sessionmaker
|
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
|
|
|
from .session import SessionFactory
|
|
|
|
|
|
@pytest_asyncio.fixture
|
|
async def session_factory(test_engine: AsyncEngine) -> AsyncGenerator[SessionFactory]:
|
|
async with test_engine.connect() as connection:
|
|
transaction = await connection.begin()
|
|
factory = async_sessionmaker(
|
|
bind=connection,
|
|
class_=AsyncSession,
|
|
expire_on_commit=False,
|
|
join_transaction_mode="create_savepoint",
|
|
)
|
|
try:
|
|
yield factory
|
|
finally:
|
|
await transaction.rollback()
|
|
```
|
|
|
|
Each factory call still creates a distinct `AsyncSession`, matching [session factory mechanics](session.md#session-factory-mechanics). The factory belongs to the fixture's engine and outer transaction and must not escape either scope.
|
|
|
|
For service tests that need to pass a caller-owned active session into `transaction_scope(session=...)`, derive that session from the same factory:
|
|
|
|
```python
|
|
@pytest_asyncio.fixture
|
|
async def session(session_factory: SessionFactory) -> AsyncGenerator[AsyncSession]:
|
|
async with session_factory() as test_session:
|
|
await test_session.begin()
|
|
yield test_session
|
|
```
|
|
|
|
The explicit `begin()` satisfies the supplied-session contract from [session management](session.md#one-optional-ownership-helper). Session closure rolls back unfinished work; the outer connection transaction remains the final isolation boundary even if application code commits its SAVEPOINT.
|
|
|
|
For FastAPI request tests, override `get_session_factory`, not only `get_session`. Both `SessionDep` and `TransactionSessionDep` then retain their production ownership behavior while receiving the test-bound factory. Always remove the override after the test because [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/) are stored in a mutable application-level dictionary.
|
|
|
|
```python
|
|
from collections.abc import Generator
|
|
|
|
import pytest
|
|
from fastapi import FastAPI
|
|
|
|
from .fastapi import get_session_factory
|
|
from .session import SessionFactory
|
|
|
|
|
|
@pytest.fixture
|
|
def app_with_test_database(app: FastAPI, session_factory: SessionFactory) -> Generator[FastAPI]:
|
|
def get_test_session_factory() -> SessionFactory:
|
|
return session_factory
|
|
|
|
app.dependency_overrides[get_session_factory] = get_test_session_factory
|
|
try:
|
|
yield app
|
|
finally:
|
|
app.dependency_overrides.pop(get_session_factory, None)
|
|
```
|
|
|
|
Construct `app` with test settings before lifespan starts so startup cannot resolve the production URL. The override changes request session creation; it does not prevent lifespan from entering its configured `database_scope()`.
|
|
|
|
The connection-bound factory is deliberately serial even though it creates distinct sessions: those sessions still share one connection and outer transaction. A test that verifies concurrently active sessions must use independent connections and a database target that supports them.
|
|
|
|
## SQLite Targets
|
|
|
|
### Serial in-memory tests
|
|
|
|
Use `sqlite+aiosqlite://` for a fresh in-memory database when the test runs all database work serially. SQLAlchemy's `aiosqlite` dialect uses a single-connection `StaticPool` for this target, so all sessions share one SQLite transaction state. One session's rollback can discard another session's uncommitted work.
|
|
|
|
Create the schema and dispose the engine deterministically:
|
|
|
|
```python
|
|
from collections.abc import AsyncGenerator
|
|
|
|
import pytest_asyncio
|
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
|
from sqlmodel import SQLModel
|
|
|
|
from .engine import engine_scope
|
|
|
|
|
|
@pytest_asyncio.fixture
|
|
async def test_engine() -> AsyncGenerator[AsyncEngine]:
|
|
async with engine_scope("sqlite+aiosqlite://") as engine:
|
|
async with engine.begin() as connection:
|
|
await connection.run_sync(SQLModel.metadata.create_all)
|
|
yield engine
|
|
```
|
|
|
|
### Concurrent in-memory tests
|
|
|
|
Do not use the default `:memory:` target for tests that have multiple active sessions or tasks. Use a named shared-cache database instead, with a name unique to the test process:
|
|
|
|
```text
|
|
sqlite+aiosqlite:///file:test-suite?mode=memory&cache=shared&uri=true
|
|
```
|
|
|
|
This lets connections share the same in-memory database while retaining independent transaction state. A temporary file URL such as `sqlite+aiosqlite:////tmp/test.db` is often simpler when test isolation or cleanup tooling already manages files.
|
|
|
|
For both SQLite forms, enable and test the constraints your application depends on. SQLite foreign-key enforcement is disabled by default, and its transaction behavior has driver-specific differences. Keep PostgreSQL integration coverage for behavior that SQLite cannot faithfully model.
|
|
|
|
## Test Data Practices
|
|
|
|
- Build only the data a test needs, through named factory functions or pytest fixtures rather than a large global seed.
|
|
- Give each fixture a domain meaning, such as `active_account`, `expired_subscription`, or `admin_user`; avoid opaque rows with unexplained defaults.
|
|
- Set values relevant to the assertion explicitly, including timestamps, permissions, statuses, and unique identifiers. Use fixed clocks or injected clock values instead of the wall clock.
|
|
- Construct object graphs through relationships, then `await session.flush()` before reading generated identifiers or passing foreign keys onward. `flush()` exercises database constraints without ending the test transaction.
|
|
- Seed prerequisite data before creating a client request. Let the endpoint own the mutation being asserted; do not pre-insert the row that the endpoint is supposed to create.
|
|
- Use `commit()` in fixture setup only when the test specifically needs to prove post-commit behavior. With the transactional fixture, this remains isolated through the outer rollback.
|
|
- Keep shared reference data immutable and explicit. If it must be reused for performance, load it once into a dedicated test database and reset all mutable tables between tests; never depend on test order.
|
|
- Include both valid and constraint-breaking graphs where a behavior depends on foreign keys, uniqueness, nullability, or cascading deletes. SQLite-only tests should not be the sole evidence for PostgreSQL constraints.
|
|
|
|
## Completion Checks
|
|
|
|
- A test run cannot reach the production URL; production credentials are absent from the test environment.
|
|
- Production PostgreSQL, local SQLite, and in-memory SQLite all use `database_scope()` unless a test explicitly needs lower-level engine or connection ownership.
|
|
- Every test owns its override, session factory, connection, transaction, session, and engine cleanup.
|
|
- Request tests override `get_session_factory`, preserving both read-session and transactional-session dependency behavior.
|
|
- Test data is deterministic, minimal, and expresses the scenario under test.
|
|
- PostgreSQL integration tests cover every PostgreSQL-specific contract and run against migrations where migrations are shipped.
|
|
|
|
## Sources
|
|
|
|
- [SQLAlchemy: joining a session into an external transaction](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html#joining-a-session-into-an-external-transaction-such-as-for-test-suites)
|
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
|
- [SQLAlchemy SQLite dialect and async in-memory pooling](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#using-a-memory-database-with-multiple-coroutines)
|
|
- [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
|
|
- [SQLModel testing with FastAPI](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/)
|
|
- [pytest-asyncio fixtures](https://pytest-asyncio.readthedocs.io/en/stable/how-to-guides/index.html) |