Async SQLAlchemy Engine

!!! info "Primary sources" - Python functools.cache - SQLAlchemy connections - SQLAlchemy asyncio extension - SQLAlchemy pooling and multiprocessing - FastAPI lifespan events

Engine Ownership Model

Create one async engine per process per database URL and keep engine construction independent from FastAPI.

SQLAlchemy guidance: the engine is intended as a long-lived, concurrent registry over pooled DB connections, not a per-request object.
A cached function provides stable process-local engine identity without making framework state the only way to obtain it.
FastAPI lifespan starts and stops that independently defined resource; it does not contain the construction policy.

!!! tip "Practical rule" - Exactly one create_async_engine(...) call in the cached engine factory. - Zero create_async_engine(...) calls in request handlers. - Zero calls to the cached factory from repository code.

Cached Engine Factory

Use functools.cache on a synchronous factory. Creating an AsyncEngine configures the dialect and pool; it does not need to await a database connection.

from functools import cache

from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine


@cache
def get_engine(database_url: str) -> AsyncEngine:
    return create_async_engine(
        database_url,
        pool_pre_ping=True,
    )


async def dispose_engine(database_url: str) -> None:
    engine = get_engine(database_url)
    try:
        await engine.dispose()
    finally:
        get_engine.cache_clear()


async def refresh_engine(database_url: str) -> AsyncEngine:
    await dispose_engine(database_url)
    return get_engine(database_url)

The database URL is an explicit, hashable cache key. Calls with the same URL return the same engine; a different URL receives a different engine. If engine options vary at runtime, make them explicit hashable arguments too.

Resolve settings at the composition boundary and call get_engine(settings.database_url). Do not hide settings lookup or engine creation inside feature code.

Thin FastAPI Lifespan Wrapper

The lifespan context manager only connects the cached resource to FastAPI ownership:

from collections.abc import AsyncIterator
from contextlib import asynccontextmanager

from fastapi import FastAPI


@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
    database_url = app.state.settings.database_url
    engine = get_engine(database_url)
    app.state.engine = engine

    try:
        yield
    finally:
        await dispose_engine(database_url)


app = FastAPI(lifespan=lifespan)

dispose() closes checked-in connections and replaces the pool, but it does not remove the Python object from functools.cache. dispose_engine() clears the cache even if driver cleanup raises, preventing a later lifespan run or test from retrieving that engine instance.

This simple cleanup assumes one configured database URL per process. If a process intentionally owns several cached engines, use a small registry with per-key removal instead of clearing the whole cache. For a fixed engine, try/finally is sufficient; use AsyncExitStack when lifespan composes multiple conditional or dynamically acquired resources.

When directly testing engine construction or lifespan behavior:

Call get_engine.cache_clear() before the test to remove process-local state.
Dispose any engine the test creates.
Clear the cache again during teardown, even when the test fails.

Driver URLs (Project Requirement: asyncpg + aiosqlite)

Use SQLAlchemy async driver URLs:

PostgreSQL: postgresql+asyncpg://user:pass@host:5432/dbname
SQLite: sqlite+aiosqlite:///./app.db

!!! warning "Driver compatibility" - Do not mix sync drivers, for example psycopg2, with create_async_engine(). - Keep URL construction centralized in settings/config, not in feature modules.

Pooling Defaults and Tuning

Default behavior is usually correct first:

Async engines use async-compatible pooling (AsyncAdaptedQueuePool) by default.
Start with defaults, then tune from observed load (pool_size, max_overflow, pool_timeout, pool_recycle).
Enable pool_pre_ping=True for safer stale-connection handling in long-running services.

When to switch pool strategy:

NullPool if you explicitly need no pooling (special environments, some tests, or strict cross-loop constraints).
Keep in mind this increases connect/disconnect churn.

Disposal Semantics

engine.dispose() replaces/disposes the pool, but only checked-in connections are immediately closed.

Rules:

Dispose when the app is shutting down.
Dispose before reusing an engine across event loops.
In forked child-process initialization, use engine.dispose(close=False) (sync API guidance) so child processes do not touch parent-held connections.

Avoid relying on garbage collection for engine cleanup in async code.

Event Loop and Process Boundaries

Do not share pooled connections across boundaries:

Multiple event loops: do not reuse the same pooled async engine across loops unless you intentionally disable pooling (NullPool) or dispose before handoff.
Multiprocessing/fork: pooled connections must not be inherited for active use across process boundaries.

This prevents broken socket state and cross-process connection corruption.

What Not to Do

Create an engine inside every request dependency.
Create/dispose engines inside repository methods.
Call get_engine() from repositories instead of injecting their engine or session dependency.
Keep engine creation as a hidden side effect of import-time module globals.
Dispose a cached engine without clearing the cache during final teardown.
Use deprecated FastAPI startup/shutdown events together with lifespan.

Engine Design Checklist

One engine per process per DB URL.
Engine created by one cached, framework-independent factory.
Lifespan only retrieves, exposes, disposes, and uncaches the engine.
Async driver URL matches backend (asyncpg or aiosqlite).
Pooling strategy is explicit for non-default needs.
No request-path engine creation.
Tests dispose engines and clear cached state deterministically.

6.6 KiB Raw Blame History