363 lines
18 KiB
Markdown
363 lines
18 KiB
Markdown
# Async SQLAlchemy Session Management
|
|
|
|
!!! info "Primary sources"
|
|
- [Python `asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager)
|
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
|
- [SQLAlchemy session basics](https://docs.sqlalchemy.org/en/21/orm/session_basics.html)
|
|
|
|
---
|
|
|
|
## Purpose
|
|
|
|
Define one canonical session model for SQLAlchemy asyncio:
|
|
|
|
- configure one shared session factory,
|
|
- create one AsyncSession per task or unit of work,
|
|
- never share one AsyncSession across concurrent tasks.
|
|
|
|
---
|
|
|
|
## Scope and Non-Goals
|
|
|
|
- In scope: session factory creation, task scoping, and transaction demarcation.
|
|
- Out of scope: framework dependency wiring, ORM model design, query optimization strategy, and schema migration tooling.
|
|
|
|
---
|
|
|
|
## Rules
|
|
|
|
- Create one `async_sessionmaker` inside each app-owned engine scope.
|
|
- Resolve the configured `async_sessionmaker` at the application composition boundary and inject it where standalone operations begin.
|
|
- Use a fresh AsyncSession for each task or explicit unit of work.
|
|
- Pass an `AsyncSession` directly to data-access functions.
|
|
- Require lower-level data-access functions to receive an `AsyncSession`; they must not create sessions or control transactions.
|
|
- Treat a supplied session as an explicit declaration that the caller owns an active transaction.
|
|
- Borrow a caller-provided session without beginning, closing, committing, or rolling it back.
|
|
- Do not share AsyncSession across `asyncio.gather()` or parallel tasks.
|
|
- Prefer direct dependency injection over global scoped-session patterns in new code.
|
|
- Use explicit transaction boundaries (`async with session.begin():`) for writes.
|
|
- When a complete operation accepts an optional session, borrow the caller's active transaction or own the complete session-and-transaction scope.
|
|
- Use `begin_nested()` directly and only when partial rollback through a database SAVEPOINT is required.
|
|
|
|
---
|
|
|
|
## Sessions and Transactions
|
|
|
|
A session and a transaction solve related but different problems:
|
|
|
|
| Concept | Responsibility | Typical lifetime |
|
|
| --- | --- | --- |
|
|
| `AsyncSession` | Provides the ORM workspace: executes queries, tracks loaded and changed objects in its identity map, and flushes pending changes. It also coordinates access to a database connection. | One task or explicit unit of work. |
|
|
| Transaction | Defines the atomic database boundary: all work inside it commits together on success or rolls back together on failure. | One complete operation that must have a single outcome. |
|
|
|
|
A transaction belongs to a session; it is not an alternative to one. The session is the interface used by application and data-access code, while the transaction determines when that work becomes permanent. A session may coordinate sequential transactions during its lifetime, although short-lived application scopes commonly use one session for one transaction.
|
|
|
|
Use a session without a helper-owned commit boundary for independent reads or lower-level functions that must participate in whatever transaction their caller controls:
|
|
|
|
```python
|
|
async with session_factory() as session:
|
|
item = await find_item(session, item_id)
|
|
```
|
|
|
|
Use an explicit transaction for writes, read-modify-write operations, or several statements that must succeed or fail as one unit:
|
|
|
|
```python
|
|
async with session_factory.begin() as session:
|
|
order = await create_order(session, order_data)
|
|
await reserve_inventory(session, order)
|
|
```
|
|
|
|
SQLAlchemy sessions use [autobegin](https://docs.sqlalchemy.org/en/21/orm/session_basics.html#auto-begin), so the first database operation normally starts a transaction even for a read. Therefore, “session-only” means that the surrounding helper owns only session lifetime and does not promise to commit; it does not mean that no database transaction exists. Closing such a session releases its resources and rolls back any unfinished transaction. An explicit `begin()` is valuable when application code must make the atomic boundary and commit ownership visible.
|
|
|
|
For most read-only operations, a session context is sufficient. Use an explicit transaction for reads when they need a defined consistency boundary, participate in a larger atomic operation, or use locking such as `SELECT ... FOR UPDATE`.
|
|
|
|
---
|
|
|
|
## Session Factory Mechanics
|
|
|
|
An `async_sessionmaker[AsyncSession]` is a reusable configuration object and callable session producer. It stores how sessions should be created, including the engine binding and options such as `expire_on_commit=False`. It is not itself a session, connection, or transaction, and calling it does not make a shared global `AsyncSession`.
|
|
|
|
Create it once from the application-owned engine and inject it into application services and dependencies:
|
|
|
|
```python
|
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
|
from sqlalchemy.ext.asyncio import async_sessionmaker
|
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
|
|
|
type SessionFactory = async_sessionmaker[AsyncSession]
|
|
|
|
|
|
def create_session_factory(engine: AsyncEngine) -> SessionFactory:
|
|
return async_sessionmaker(
|
|
bind=engine,
|
|
class_=AsyncSession,
|
|
expire_on_commit=False,
|
|
)
|
|
```
|
|
|
|
The maker is cheap configuration and has no independent connection pool or async cleanup method. Its bound engine owns the pool, so construct the maker inside that engine's lifecycle and do not retain it after the engine scope exits. A global cache adds no value when the composition root creates both resources once.
|
|
|
|
Each call to `session_factory()` creates a distinct `AsyncSession`. The caller that invokes the factory owns that session lifetime and must close it, normally with `async with`:
|
|
|
|
```python
|
|
async with session_factory() as session:
|
|
...
|
|
```
|
|
|
|
The factory can be shared across operations and tasks. Sessions produced by it cannot be shared across concurrent tasks.
|
|
|
|
Passing the factory directly has three useful consequences:
|
|
|
|
- Lower layers do not resolve settings or global resources.
|
|
- Tests can inject a test factory directly.
|
|
- Transaction ownership remains independent of engine construction.
|
|
|
|
---
|
|
|
|
## Minimal Scope Model
|
|
|
|
Most applications need only these three forms:
|
|
|
|
1. `session_factory()` for a standalone read or other session-only conversation.
|
|
2. One `transaction_scope()` helper for a complete operation that may either own a transaction or join its caller's transaction.
|
|
3. `session.begin_nested()` at the exact call site that needs partial rollback through a SAVEPOINT.
|
|
|
|
Do not add a general `atomic_scope()` abstraction. The word "atomic" does not reveal whether the scope joins an outer transaction, creates and commits a root transaction, or creates a SAVEPOINT. Those behaviors have different failure and ownership semantics and should remain visible.
|
|
|
|
### One optional-ownership helper
|
|
|
|
```python
|
|
from collections.abc import AsyncGenerator
|
|
from contextlib import asynccontextmanager
|
|
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
from sqlalchemy.ext.asyncio import async_sessionmaker
|
|
|
|
type SessionFactory = async_sessionmaker[AsyncSession]
|
|
|
|
|
|
@asynccontextmanager
|
|
async def transaction_scope(
|
|
session_factory: SessionFactory,
|
|
*,
|
|
session: AsyncSession | None = None,
|
|
) -> AsyncGenerator[AsyncSession]:
|
|
if session is not None:
|
|
if not session.in_transaction():
|
|
raise RuntimeError("A supplied session must have an active transaction")
|
|
|
|
yield session
|
|
return
|
|
|
|
async with session_factory.begin() as owned_session:
|
|
yield owned_session
|
|
```
|
|
|
|
The explicit branch is preferable to compressing both paths through `nullcontext()` or a mode-driven helper. It makes the ownership transition obvious and keeps type narrowing straightforward. Its runtime cost is negligible compared with database I/O.
|
|
|
|
The two paths have deliberately different responsibilities:
|
|
|
|
| Input | Session owner | Transaction owner | Successful exit | Exceptional exit |
|
|
| --- | --- | --- | --- | --- |
|
|
| `session=None` | Helper | Helper | Flush, commit, then close | Roll back, then close |
|
|
| Existing `session` | Caller | Caller | Yield control back to caller | Propagate to caller without cleanup |
|
|
|
|
[`async_sessionmaker.begin()`](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.async_sessionmaker.begin) is the right primitive for the owned path because it creates the session and root transaction together, commits on successful exit, rolls back on exceptional exit, and closes the session. It is equivalent in ownership terms to nesting `session_factory()` and `owned_session.begin()` context managers.
|
|
|
|
Do not call `session.begin()` when a session is supplied. A supplied session means the caller has already chosen the transaction boundary. Silently beginning a transaction would make commit ownership depend on hidden branch behavior and would fail when the session was already active.
|
|
|
|
### Autobegin and the defensive check
|
|
|
|
The `session.in_transaction()` check catches the obvious contract violation of passing an unused session without an outer transaction. It does not prove that the caller intentionally opened a transaction.
|
|
|
|
SQLAlchemy's [autobegin](https://docs.sqlalchemy.org/en/21/orm/session_basics.html#auto-begin) behavior starts transactional state after operations such as `execute()`, `add()`, or modifying a persistent object. A preceding read can therefore make `in_transaction()` return `True`. The real ownership signal is the API call itself: passing `session=` declares that the caller owns the active transaction.
|
|
|
|
Applications that require mechanical enforcement can construct sessions with `autobegin=False`, but then every database conversation, including reads and every post-commit reuse, must begin explicitly. That stricter policy is valid but is not the minimalist default.
|
|
|
|
---
|
|
|
|
## Function and Service Boundaries
|
|
|
|
Lower-level functions should require a session and contain only data-access behavior:
|
|
|
|
```python
|
|
from sqlalchemy import select
|
|
|
|
|
|
async def find_item(session: AsyncSession, item_id: int) -> Item | None:
|
|
statement = select(Item).where(Item.id == item_id)
|
|
return await session.scalar(statement)
|
|
|
|
|
|
async def insert_order(session: AsyncSession, payload: OrderCreate) -> Order:
|
|
order = Order.model_validate(payload)
|
|
session.add(order)
|
|
await session.flush()
|
|
return order
|
|
```
|
|
|
|
These functions do not create, close, commit, roll back, or nest transactions. This keeps them composable and makes transaction behavior a property of the calling use case rather than the query helper.
|
|
|
|
A complete write operation may accept an optional session and use `transaction_scope()`:
|
|
|
|
```python
|
|
async def create_order(
|
|
session_factory: SessionFactory,
|
|
payload: OrderCreate,
|
|
*,
|
|
session: AsyncSession | None = None,
|
|
) -> Order:
|
|
async with transaction_scope(
|
|
session_factory,
|
|
session=session,
|
|
) as active_session:
|
|
return await insert_order(active_session, payload)
|
|
```
|
|
|
|
The standalone call owns and commits its work:
|
|
|
|
```python
|
|
order = await create_order(session_factory, payload)
|
|
```
|
|
|
|
A larger use case owns one transaction and passes the same session through every operation:
|
|
|
|
```python
|
|
async with transaction_scope(session_factory) as session:
|
|
order = await create_order(
|
|
session_factory,
|
|
payload,
|
|
session=session,
|
|
)
|
|
await reserve_inventory(session, order)
|
|
await create_audit_entry(session, order)
|
|
```
|
|
|
|
The inner `create_order()` scope joins the existing transaction; it does not commit and does not create a SAVEPOINT. If inventory reservation or audit creation fails, the outer scope rolls back all three operations together. This is ordinary service composition, not a nested database transaction.
|
|
|
|
For standalone reads, use the factory directly rather than routing through a transaction-owning helper:
|
|
|
|
```python
|
|
async with session_factory() as session:
|
|
item = await find_item(session, item_id)
|
|
```
|
|
|
|
The session context closes the session and rolls back any unfinished autobegun transaction. It does not commit. If a public read operation supports a caller-supplied session, keep the small borrow-or-create branch in that operation; do not disguise it as transaction ownership.
|
|
|
|
Application service objects that represent standalone operations may store the immutable session factory, but they must not store a mutable session:
|
|
|
|
```python
|
|
class ItemService:
|
|
def __init__(self, session_factory: SessionFactory) -> None:
|
|
self.session_factory = session_factory
|
|
|
|
async def find(self, item_id: int) -> Item | None:
|
|
async with self.session_factory() as session:
|
|
return await find_item(session, item_id)
|
|
```
|
|
|
|
Code that already owns a transaction should call the session-required function directly. Repositories should normally remain in that session-required layer; the service or use-case boundary owns standalone session creation. This avoids optional-session APIs spreading into every data-access function.
|
|
|
|
---
|
|
|
|
## SAVEPOINTs and Partial Failure
|
|
|
|
Use [`begin_nested()`](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html#using-savepoint) only when failure inside one portion of an operation should roll back that portion while preserving the outer transaction:
|
|
|
|
```python
|
|
async with transaction_scope(session_factory) as session:
|
|
order = await insert_order(session, payload)
|
|
|
|
try:
|
|
async with session.begin_nested():
|
|
await apply_optional_discount(session, order)
|
|
except DiscountError:
|
|
pass
|
|
|
|
await reserve_inventory(session, order)
|
|
```
|
|
|
|
Important SAVEPOINT semantics:
|
|
|
|
- `begin_nested()` starts a root transaction if one is not already active, so call it inside a visible outer transaction when that ownership matters.
|
|
- Entering `begin_nested()` unconditionally flushes pending session state, regardless of the `autoflush` setting.
|
|
- Successful exit releases the SAVEPOINT; it does not commit the outer transaction.
|
|
- Exceptional exit rolls back to the SAVEPOINT and leaves the outer transaction active.
|
|
- In SQLAlchemy 2.x, `session.commit()` commits the outermost transaction. Never call it to release a SAVEPOINT; let the nested context manager manage its transaction handle.
|
|
|
|
Do not create a SAVEPOINT merely because one service calls another. SAVEPOINTs add database work and alter flush and error-recovery behavior. Use them only for explicit partial-failure requirements such as skipping one conflicting row while retaining the rest of a batch.
|
|
|
|
---
|
|
|
|
## Framework Integration
|
|
|
|
Keep framework adapters outside these session primitives. See [FastAPI database integration](fastapi.md) for lifespan ownership, `Annotated` dependency aliases, and read-versus-write request sessions.
|
|
|
|
---
|
|
|
|
## Configuration Guidance
|
|
|
|
- `expire_on_commit=False` is commonly preferred in asyncio applications to reduce accidental post-commit reload behavior.
|
|
- `AsyncSession.refresh()` is preferred over broad expiration patterns when state refresh is needed.
|
|
- [`async_sessionmaker.begin()`](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.async_sessionmaker.begin) is a concise option when one scope must create a session, begin a transaction, commit on success, roll back on failure, and close. Do not use it when borrowing a caller's session.
|
|
|
|
## SQLModel Alignment
|
|
|
|
- Use SQLModel as the default model and statement layer while keeping the same session ownership model: one `async_sessionmaker`, one `AsyncSession` per task or unit of work.
|
|
- SQLModel does not replace SQLAlchemy async lifecycle primitives; it provides model declaration, validation, and typing ergonomics on top of them.
|
|
- Do not mix ad hoc session construction with the canonical session factory.
|
|
|
|
---
|
|
|
|
## Concurrency Rules
|
|
|
|
- One session per concurrent task.
|
|
- If work fans out into parallel tasks, each task receives its own AsyncSession.
|
|
- Pass sessions explicitly to service functions; avoid mutable global session state.
|
|
|
|
---
|
|
|
|
## Anti-Patterns
|
|
|
|
- A singleton/global AsyncSession reused across tasks or operations.
|
|
- Sharing one AsyncSession across parallel tasks.
|
|
- Passing an application-global AsyncSession to a repository constructor.
|
|
- Creating a new `async_sessionmaker` in each operation.
|
|
- Retaining a session factory after its bound engine scope exits.
|
|
- Calling the session factory inside low-level access functions such as `find_item()`.
|
|
- Hidden session creation in lower access functions with no caller control.
|
|
- Closing or committing a session supplied by the caller.
|
|
- Starting a new transaction inside a helper that may receive a session already in a transaction.
|
|
- Silently starting or committing a transaction on a supplied session.
|
|
- Treating `in_transaction()` as proof that a caller intentionally owns the transaction.
|
|
- Creating a SAVEPOINT for ordinary nested service calls.
|
|
- Hiding root transaction, joined transaction, and SAVEPOINT behavior behind one mode-driven `atomic_scope()` helper.
|
|
- Calling `session.commit()` inside a SAVEPOINT scope.
|
|
- Mixing commit/rollback ownership across layers without a declared boundary.
|
|
|
|
---
|
|
|
|
## Operational Checks
|
|
|
|
- Exactly one `async_sessionmaker` is configured inside each application engine scope.
|
|
- The session factory does not outlive its bound engine.
|
|
- Application operations receive sessions from one canonical session factory.
|
|
- No code path creates AsyncSession in module import side effects.
|
|
- Concurrent jobs and operations each create task-local sessions.
|
|
|
|
---
|
|
|
|
## Testing Checks
|
|
|
|
- Service constructors accept a test session factory without framework startup.
|
|
- Session-taking access functions accept a transaction-scoped test session directly.
|
|
- Transaction-scope tests verify supplied sessions require an active transaction and remain caller-owned.
|
|
- Transaction-scope tests verify owned transactions commit on success, roll back on failure, and close their sessions.
|
|
- Composition tests verify nested service calls join one outer transaction without committing it.
|
|
- SAVEPOINT tests verify local rollback preserves the outer transaction and successful exit does not commit it.
|
|
- Tests that depend on SAVEPOINT timing account for `begin_nested()` flushing pending state on entry.
|
|
- Rollback behavior is verified for failed write units.
|
|
- Parallel-task tests verify no shared AsyncSession instances.
|
|
- Lifecycle tests confirm the session factory is initialized and teardown-safe.
|
|
|