improvements
This commit is contained in:
@@ -1,6 +1,7 @@
|
||||
# Async SQLAlchemy Engine
|
||||
|
||||
!!! info "Primary sources"
|
||||
- [Python `functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
|
||||
- [SQLAlchemy connections](https://docs.sqlalchemy.org/en/21/core/connections.html)
|
||||
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||
- [SQLAlchemy pooling and multiprocessing](https://docs.sqlalchemy.org/en/21/core/pooling.html#pooling-multiprocessing)
|
||||
@@ -10,55 +11,89 @@
|
||||
|
||||
## Engine Ownership Model
|
||||
|
||||
Create one async engine per process per database URL and keep it for the app lifetime.
|
||||
Create one async engine per process per database URL and keep engine construction independent from FastAPI.
|
||||
|
||||
- SQLAlchemy guidance: the engine is intended as a long-lived, concurrent registry over pooled DB connections, not a per-request object.
|
||||
- In FastAPI, app startup and shutdown ownership belongs in lifespan.
|
||||
- Use `FastAPI(lifespan=...)` (not startup/shutdown events) for modern lifecycle wiring.
|
||||
- A cached function provides stable process-local engine identity without making framework state the only way to obtain it.
|
||||
- FastAPI lifespan starts and stops that independently defined resource; it does not contain the construction policy.
|
||||
|
||||
!!! tip "Practical rule"
|
||||
- Exactly one `create_async_engine(...)` call in app bootstrap code.
|
||||
- Exactly one `create_async_engine(...)` call in the cached engine factory.
|
||||
- Zero `create_async_engine(...)` calls in request handlers.
|
||||
- Zero calls to the cached factory from repository code.
|
||||
|
||||
---
|
||||
|
||||
## Canonical Lifespan Pattern (AsyncExitStack)
|
||||
## Cached Engine Factory
|
||||
|
||||
Use `@asynccontextmanager` + `AsyncExitStack` to make teardown deterministic and composable.
|
||||
Use [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) on a synchronous factory. Creating an `AsyncEngine` configures the dialect and pool; it does not need to await a database connection.
|
||||
|
||||
```python
|
||||
from contextlib import AsyncExitStack, asynccontextmanager
|
||||
from functools import cache
|
||||
|
||||
from fastapi import FastAPI
|
||||
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
|
||||
|
||||
|
||||
@cache
|
||||
def get_engine(database_url: str) -> AsyncEngine:
|
||||
return create_async_engine(
|
||||
database_url,
|
||||
pool_pre_ping=True,
|
||||
)
|
||||
|
||||
|
||||
async def dispose_engine(database_url: str) -> None:
|
||||
engine = get_engine(database_url)
|
||||
try:
|
||||
await engine.dispose()
|
||||
finally:
|
||||
get_engine.cache_clear()
|
||||
|
||||
|
||||
async def refresh_engine(database_url: str) -> AsyncEngine:
|
||||
await dispose_engine(database_url)
|
||||
return get_engine(database_url)
|
||||
```
|
||||
|
||||
The database URL is an explicit, hashable cache key. Calls with the same URL return the same engine; a different URL receives a different engine. If engine options vary at runtime, make them explicit hashable arguments too.
|
||||
|
||||
Resolve settings at the composition boundary and call `get_engine(settings.database_url)`. Do not hide settings lookup or engine creation inside feature code.
|
||||
|
||||
## Thin FastAPI Lifespan Wrapper
|
||||
|
||||
The lifespan context manager only connects the cached resource to FastAPI ownership:
|
||||
|
||||
```python
|
||||
from collections.abc import AsyncIterator
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
async with AsyncExitStack() as stack:
|
||||
engine: AsyncEngine = create_async_engine(
|
||||
app.state.settings.database_url,
|
||||
pool_pre_ping=True,
|
||||
# Optional examples:
|
||||
# echo=app.state.settings.sql_echo,
|
||||
# pool_size=10,
|
||||
# max_overflow=20,
|
||||
)
|
||||
app.state.engine = engine
|
||||
|
||||
# Ensure engine disposal always runs at shutdown.
|
||||
stack.push_async_callback(engine.dispose)
|
||||
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
||||
database_url = app.state.settings.database_url
|
||||
engine = get_engine(database_url)
|
||||
app.state.engine = engine
|
||||
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
await dispose_engine(database_url)
|
||||
|
||||
|
||||
app = FastAPI(lifespan=lifespan)
|
||||
```
|
||||
|
||||
Why this pattern:
|
||||
- FastAPI executes code before `yield` at startup and after `yield` at shutdown.
|
||||
- `AsyncExitStack` lets you register multiple async cleanups in one place while preserving order.
|
||||
- Explicit disposal (directly awaited or via `AsyncExitStack` callback) avoids event-loop-closed warnings when objects fall out of scope.
|
||||
`dispose()` closes checked-in connections and replaces the pool, but it does not remove the Python object from `functools.cache`. `dispose_engine()` clears the cache even if driver cleanup raises, preventing a later lifespan run or test from retrieving that engine instance.
|
||||
|
||||
This simple cleanup assumes one configured database URL per process. If a process intentionally owns several cached engines, use a small registry with per-key removal instead of clearing the whole cache. For a fixed engine, `try/finally` is sufficient; use `AsyncExitStack` when lifespan composes multiple conditional or dynamically acquired resources.
|
||||
|
||||
When directly testing engine construction or lifespan behavior:
|
||||
|
||||
- Call `get_engine.cache_clear()` before the test to remove process-local state.
|
||||
- Dispose any engine the test creates.
|
||||
- Clear the cache again during teardown, even when the test fails.
|
||||
|
||||
---
|
||||
|
||||
@@ -118,7 +153,9 @@ This prevents broken socket state and cross-process connection corruption.
|
||||
|
||||
- Create an engine inside every request dependency.
|
||||
- Create/dispose engines inside repository methods.
|
||||
- Call `get_engine()` from repositories instead of injecting their engine or session dependency.
|
||||
- Keep engine creation as a hidden side effect of import-time module globals.
|
||||
- Dispose a cached engine without clearing the cache during final teardown.
|
||||
- Use deprecated FastAPI startup/shutdown events together with lifespan.
|
||||
|
||||
---
|
||||
@@ -126,8 +163,9 @@ This prevents broken socket state and cross-process connection corruption.
|
||||
## Engine Design Checklist
|
||||
|
||||
- One engine per process per DB URL.
|
||||
- Engine created in lifespan startup.
|
||||
- Engine disposed in lifespan shutdown.
|
||||
- Engine created by one cached, framework-independent factory.
|
||||
- Lifespan only retrieves, exposes, disposes, and uncaches the engine.
|
||||
- Async driver URL matches backend (`asyncpg` or `aiosqlite`).
|
||||
- Pooling strategy is explicit for non-default needs.
|
||||
- No request-path engine creation.
|
||||
- Tests dispose engines and clear cached state deterministically.
|
||||
|
||||
Reference in New Issue
Block a user