john/prompts
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

engine/session updates

Browse Source
This commit is contained in:
John Lancaster
2026-07-31 22:15:51 -05:00
parent cd11ea8255
commit 0dc06f72ca
12 changed files with 767 additions and 457 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/skills/async-fastapi-sqlmodel/references/session.md
+154 -176
View File
@@ -1,42 +1,43 @@
# Async SQLAlchemy Session Management
!!! info "Primary sources"
- [Python `functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
- [Python `asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager)
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
- [SQLAlchemy session basics](https://docs.sqlalchemy.org/en/21/orm/session_basics.html)
- [FastAPI dependencies with yield](https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/)
---
## Purpose
Define one canonical session model for FastAPI + SQLAlchemy asyncio:
Define one canonical session model for SQLAlchemy asyncio:
- configure one shared session factory,
- create one AsyncSession per request or per unit-of-work,
- create one AsyncSession per task or unit of work,
- never share one AsyncSession across concurrent tasks.
---
## Scope and Non-Goals
- In scope: session factory creation, FastAPI dependency wiring, request/task scoping, transaction demarcation.
- Out of scope: ORM model design, query optimization strategy, schema migration tooling.
- In scope: session factory creation, task scoping, and transaction demarcation.
- Out of scope: framework dependency wiring, ORM model design, query optimization strategy, and schema migration tooling.
---
## Rules
- Create one cached `async_sessionmaker` per app-owned AsyncEngine.
- Let repositories resolve the cached maker by database URL.
- Use a fresh AsyncSession for each request or explicit unit-of-work.
- Create one `async_sessionmaker` inside each app-owned engine scope.
- Resolve the configured `async_sessionmaker` at the application composition boundary and inject it where standalone operations begin.
- Use a fresh AsyncSession for each task or explicit unit of work.
- Pass an `AsyncSession` directly to data-access functions.
- Borrow a caller-provided session without closing or committing it.
- Require lower-level data-access functions to receive an `AsyncSession`; they must not create sessions or control transactions.
- Treat a supplied session as an explicit declaration that the caller owns an active transaction.
- Borrow a caller-provided session without beginning, closing, committing, or rolling it back.
- Do not share AsyncSession across `asyncio.gather()` or parallel tasks.
- Prefer direct dependency injection over global scoped-session patterns in new code.
- Use explicit transaction boundaries (`async with session.begin():`) for writes.
- When a use case accepts an optional session, borrow only an active caller-owned transaction or own the complete session-and-transaction scope.
- When a complete operation accepts an optional session, borrow the caller's active transaction or own the complete session-and-transaction scope.
- Use `begin_nested()` directly and only when partial rollback through a database SAVEPOINT is required.
---
@@ -46,7 +47,7 @@ A session and a transaction solve related but different problems:
| Concept | Responsibility | Typical lifetime |
| --- | --- | --- |
| `AsyncSession` | Provides the ORM workspace: executes queries, tracks loaded and changed objects in its identity map, and flushes pending changes. It also coordinates access to a database connection. | One request, task, or explicit unit of work. |
| `AsyncSession` | Provides the ORM workspace: executes queries, tracks loaded and changed objects in its identity map, and flushes pending changes. It also coordinates access to a database connection. | One task or explicit unit of work. |
| Transaction | Defines the atomic database boundary: all work inside it commits together on success or rolls back together on failure. | One complete operation that must have a single outcome. |
A transaction belongs to a session; it is not an alternative to one. The session is the interface used by application and data-access code, while the transaction determines when that work becomes permanent. A session may coordinate sequential transactions during its lifetime, although short-lived application scopes commonly use one session for one transaction.
@@ -76,33 +77,25 @@ For most read-only operations, a session context is sufficient. Use an explicit
An `async_sessionmaker[AsyncSession]` is a reusable configuration object and callable session producer. It stores how sessions should be created, including the engine binding and options such as `expire_on_commit=False`. It is not itself a session, connection, or transaction, and calling it does not make a shared global `AsyncSession`.
Cache it by the application-owned engine so repeated composition calls return the same maker:
Create it once from the application-owned engine and inject it into application services and dependencies:
```python
from functools import cache
from sqlalchemy.ext.asyncio import AsyncEngine
from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlmodel.ext.asyncio.session import AsyncSession
from .engine import dispose_engine
from .engine import get_engine
type SessionFactory = async_sessionmaker[AsyncSession]
@cache
def get_session_factory(database_url: str) -> async_sessionmaker[AsyncSession]:
def create_session_factory(engine: AsyncEngine) -> SessionFactory:
return async_sessionmaker(
bind=get_engine(database_url),
bind=engine,
class_=AsyncSession,
expire_on_commit=False,
)
async def dispose_session_factory(database_url: str) -> None:
get_session_factory.cache_clear()
await dispose_engine(database_url)
```
`functools.cache` caches by argument equality and requires hashable arguments. The database URL is an explicit string key shared with the cached engine factory. The cache retains the returned maker until `get_session_factory.cache_clear()` runs. Cache the synchronous maker function, never an async function and never a produced `AsyncSession`.
The maker is cheap configuration and has no independent connection pool or async cleanup method. Its bound engine owns the pool, so construct the maker inside that engine's lifecycle and do not retain it after the engine scope exits. A global cache adds no value when the composition root creates both resources once.
Each call to `session_factory()` creates a distinct `AsyncSession`. The caller that invokes the factory owns that session lifetime and must close it, normally with `async with`:
@@ -111,123 +104,84 @@ async with session_factory() as session:
...
```
The factory can be shared across requests and tasks. Sessions produced by it cannot be shared across concurrent tasks.
The factory can be shared across operations and tasks. Sessions produced by it cannot be shared across concurrent tasks.
An `async_sessionmaker` has no connection pool or async `dispose()` method of its own. `dispose_session_factory()` means "invalidate the cached maker, then dispose its engine." Clearing the maker first ensures no subsequent composition call can retrieve a maker bound to the engine being shut down.
Passing the factory directly has three useful consequences:
Use the helper when shutting down or replacing the database resources:
```python
await dispose_session_factory(database_url)
```
Otherwise, a later call can return a maker that still references the old engine object. This matters in lifespan tests, application restarts within one process, and test suites that replace engines.
- Lower layers do not resolve settings or global resources.
- Tests can inject a test factory directly.
- Transaction ownership remains independent of engine construction.
---
## Optional Session Ownership
## Minimal Scope Model
A small [`asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager) can make repository methods composable. It borrows an existing session when supplied; otherwise it creates and closes one from a supplied factory:
Most applications need only these three forms:
1. `session_factory()` for a standalone read or other session-only conversation.
2. One `transaction_scope()` helper for a complete operation that may either own a transaction or join its caller's transaction.
3. `session.begin_nested()` at the exact call site that needs partial rollback through a SAVEPOINT.
Do not add a general `atomic_scope()` abstraction. The word "atomic" does not reveal whether the scope joins an outer transaction, creates and commits a root transaction, or creates a SAVEPOINT. Those behaviors have different failure and ownership semantics and should remain visible.
### One optional-ownership helper
```python
from collections.abc import AsyncGeneratorr
from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.ext.asyncio import async_sessionmaker
@asynccontextmanager
async def session_scope(
*,
database_url: str,
session: AsyncSession | None = None,
) -> AsyncGenerator[AsyncSession]:
if session is not None:
yield session
return
type SessionFactory = async_sessionmaker[AsyncSession]
async with get_session_factory(database_url)() as owned_session:
yield owned_session
```
The branch is intentionally explicit. Python's [`nullcontext`](https://docs.python.org/3/library/contextlib.html#contextlib.nullcontext) can express the same borrow-or-own idea, but the branch keeps ownership and typing obvious.
This helper manages session lifetime only:
- It does not close, commit, or roll back a supplied session; the caller owns it.
- It closes a session that it creates. Closing releases resources and rolls back an unfinished transaction; it does not commit.
- It does not start a transaction. Put `session.begin()` at the use-case boundary.
- A supplied session wins; the cached factory is not resolved.
- Otherwise, `database_url` selects the cached factory returned by `get_session_factory()`.
Do not turn this into an implicit unit-of-work helper that sometimes commits. Whether work joins an existing transaction or creates a new one must remain visible to the caller.
---
## Optional Transaction Ownership
Use a separate context manager when a service or use-case function must support both a caller-owned transaction and a standalone transaction. A supplied session must already be inside a transaction; otherwise the helper creates a session and transaction together with `async_sessionmaker.begin()`:
```python
@asynccontextmanager
async def transaction_scope(
session_factory: SessionFactory,
*,
database_url: str,
session: AsyncSession | None = None,
) -> AsyncGenerator[AsyncSession]:
if session is not None:
if not session.in_transaction():
raise RuntimeError("A supplied session must have an active transaction")
yield session
return
session_factory = get_session_factory(database_url)
async with session_factory.begin() as owned_session:
yield owned_session
```
Here, `begin()` is intentionally called on the [`async_sessionmaker`](https://docs.sqlalchemy.org/en/20/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.async_sessionmaker.begin), not on an existing `AsyncSession`. The related APIs have different ownership semantics:
The explicit branch is preferable to compressing both paths through `nullcontext()` or a mode-driven helper. It makes the ownership transition obvious and keeps type narrowing straightforward. Its runtime cost is negligible compared with database I/O.
- `session_factory()` creates a session whose lifetime the surrounding code must manage; it does not commit automatically.
- `session_factory.begin()` creates a new session and transaction together, commits on successful exit or rolls back on exceptional exit, and then closes the session.
- `session.begin()` manages a transaction on an existing session but does not own or close that session.
The two paths have deliberately different responsibilities:
The factory form is equivalent in ownership terms to creating a session and then entering that session's transaction:
| Input | Session owner | Transaction owner | Successful exit | Exceptional exit |
| --- | --- | --- | --- | --- |
| `session=None` | Helper | Helper | Flush, commit, then close | Roll back, then close |
| Existing `session` | Caller | Caller | Yield control back to caller | Propagate to caller without cleanup |
```python
async with session_factory() as owned_session:
async with owned_session.begin():
yield owned_session
```
[`async_sessionmaker.begin()`](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.async_sessionmaker.begin) is the right primitive for the owned path because it creates the session and root transaction together, commits on successful exit, rolls back on exceptional exit, and closes the session. It is equivalent in ownership terms to nesting `session_factory()` and `owned_session.begin()` context managers.
This helper makes transaction ownership follow the same explicit borrow-or-own mechanics as session ownership:
Do not call `session.begin()` when a session is supplied. A supplied session means the caller has already chosen the transaction boundary. Silently beginning a transaction would make commit ownership depend on hidden branch behavior and would fail when the session was already active.
- A supplied session and its active transaction remain caller-owned. The helper does not commit, roll back, or close them.
- Without a supplied session, the helper owns the session and transaction. Successful exit commits; exceptional exit rolls back; either path closes the session.
- Use this helper only at a complete operation, service, or use-case boundary. A public CRUD function or repository method may be such a boundary when its optional-session contract explicitly states that omitting the session owns and commits one transaction. Never use it inside a lower-level session-required helper.
- Do not silently begin a transaction on a supplied session. That would make commit ownership depend on hidden helper behavior.
### Autobegin and the defensive check
Callers that supply a session make their ownership visible with an outer transaction:
The `session.in_transaction()` check catches the obvious contract violation of passing an unused session without an outer transaction. It does not prove that the caller intentionally opened a transaction.
```python
async with session_factory() as session:
async with session.begin():
await run_use_case(..., session=session)
```
SQLAlchemy's [autobegin](https://docs.sqlalchemy.org/en/21/orm/session_basics.html#auto-begin) behavior starts transactional state after operations such as `execute()`, `add()`, or modifying a persistent object. A preceding read can therefore make `in_transaction()` return `True`. The real ownership signal is the API call itself: passing `session=` declares that the caller owns the active transaction.
Standalone callers omit the session and let the use case own the complete unit of work:
```python
await run_use_case(...)
```
Applications that require mechanical enforcement can construct sessions with `autobegin=False`, but then every database conversation, including reads and every post-commit reuse, must begin explicitly. That stricter policy is valid but is not the minimalist default.
---
## Repository and Function Boundaries
## Function and Service Boundaries
Pass the database URL to repository constructors. The repository stores repeatable database configuration, not mutable session state, and `session_scope()` resolves the cached factory when a standalone operation needs a session:
Lower-level functions should require a session and contain only data-access behavior:
```python
from sqlalchemy import select
from sqlmodel.ext.asyncio.session import AsyncSession
async def find_item(session: AsyncSession, item_id: int) -> Item | None:
@@ -235,88 +189,109 @@ async def find_item(session: AsyncSession, item_id: int) -> Item | None:
return await session.scalar(statement)
class ItemRepository:
def __init__(self, database_url: str) -> None:
self.database_url = database_url
async def find(
self,
item_id: int,
*,
session: AsyncSession | None = None,
) -> Item | None:
async with session_scope(
database_url=self.database_url,
session=session,
) as active_session:
return await find_item(active_session, item_id)
async def insert_order(session: AsyncSession, payload: OrderCreate) -> Order:
order = Order.model_validate(payload)
session.add(order)
await session.flush()
return order
```
This split gives each layer one job:
These functions do not create, close, commit, roll back, or nest transactions. This keeps them composable and makes transaction behavior a property of the calling use case rather than the query helper.
- The repository object identifies its database configuration and creates a session only for a standalone call.
- Standalone calls reuse the cached factory selected by database URL.
- A caller can pass a session to join an existing unit of work; the repository borrows it.
- The access function owns only the query and requires an existing `AsyncSession`.
- Application wiring supplies the production factory.
- Tests can use a test database URL or call `find_item()` with a transaction-scoped test session.
A complete write operation may accept an optional session and use `transaction_scope()`:
When several repository operations must share one transaction, pass the same session through each call. Put the transaction at the use-case boundary:
```python
async def create_order(
session_factory: SessionFactory,
payload: OrderCreate,
*,
session: AsyncSession | None = None,
) -> Order:
async with transaction_scope(
session_factory,
session=session,
) as active_session:
return await insert_order(active_session, payload)
```
The standalone call owns and commits its work:
```python
order = await create_order(session_factory, payload)
```
A larger use case owns one transaction and passes the same session through every operation:
```python
async with transaction_scope(session_factory) as session:
order = await create_order(
session_factory,
payload,
session=session,
)
await reserve_inventory(session, order)
await create_audit_entry(session, order)
```
The inner `create_order()` scope joins the existing transaction; it does not commit and does not create a SAVEPOINT. If inventory reservation or audit creation fails, the outer scope rolls back all three operations together. This is ordinary service composition, not a nested database transaction.
For standalone reads, use the factory directly rather than routing through a transaction-owning helper:
```python
async with session_factory() as session:
async with session.begin():
item = await repository.find(item_id, session=session)
await update_item(session, item, changes)
item = await find_item(session, item_id)
```
This preserves atomicity without making repository objects hold mutable `AsyncSession` instances across calls.
The session context closes the session and rolls back any unfinished autobegun transaction. It does not commit. If a public read operation supports a caller-supplied session, keep the small borrow-or-create branch in that operation; do not disguise it as transaction ownership.
Application service objects that represent standalone operations may store the immutable session factory, but they must not store a mutable session:
```python
class ItemService:
def __init__(self, session_factory: SessionFactory) -> None:
self.session_factory = session_factory
async def find(self, item_id: int) -> Item | None:
async with self.session_factory() as session:
return await find_item(session, item_id)
```
Code that already owns a transaction should call the session-required function directly. Repositories should normally remain in that session-required layer; the service or use-case boundary owns standalone session creation. This avoids optional-session APIs spreading into every data-access function.
---
## Canonical FastAPI Dependency Pattern
## SAVEPOINTs and Partial Failure
Use [`begin_nested()`](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html#using-savepoint) only when failure inside one portion of an operation should roll back that portion while preserving the outer transaction:
```python
from collections.abc import AsyncGenerator
async with transaction_scope(session_factory) as session:
order = await insert_order(session, payload)
from fastapi import Depends
from fastapi import Request
from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlmodel.ext.asyncio.session import AsyncSession
try:
async with session.begin_nested():
await apply_optional_discount(session, order)
except DiscountError:
pass
type SessionFactory = async_sessionmaker[AsyncSession]
def resolve_session_factory(request: Request) -> SessionFactory:
return get_session_factory(request.app.state.settings.database_url)
async def get_db_session(
session_factory: SessionFactory = Depends(resolve_session_factory),
) -> AsyncGenerator[AsyncSession]:
async with session_factory() as session:
yield session
await reserve_inventory(session, order)
```
Route usage:
Important SAVEPOINT semantics:
```python
from fastapi import APIRouter, Depends
from sqlmodel.ext.asyncio.session import AsyncSession
- `begin_nested()` starts a root transaction if one is not already active, so call it inside a visible outer transaction when that ownership matters.
- Entering `begin_nested()` unconditionally flushes pending session state, regardless of the `autoflush` setting.
- Successful exit releases the SAVEPOINT; it does not commit the outer transaction.
- Exceptional exit rolls back to the SAVEPOINT and leaves the outer transaction active.
- In SQLAlchemy 2.x, `session.commit()` commits the outermost transaction. Never call it to release a SAVEPOINT; let the nested context manager manage its transaction handle.
from .session import get_db_session
Do not create a SAVEPOINT merely because one service calls another. SAVEPOINTs add database work and alter flush and error-recovery behavior. Use them only for explicit partial-failure requirements such as skipping one conflicting row while retaining the rest of a batch.
router = APIRouter()
---
## Framework Integration
@router.post("/items")
async def create_item(session: AsyncSession = Depends(get_db_session)) -> dict:
async with session.begin():
# write operations here
...
return {"status": "ok"}
```
Keep framework adapters outside these session primitives. See [FastAPI database integration](fastapi.md) for lifespan ownership, `Annotated` dependency aliases, and read-versus-write request sessions.
---
@@ -328,9 +303,9 @@ async def create_item(session: AsyncSession = Depends(get_db_session)) -> dict:
## SQLModel Alignment
- Use SQLModel as the default model and statement layer while keeping the same session ownership model: one `async_sessionmaker`, one `AsyncSession` per request/unit-of-work.
- Use SQLModel as the default model and statement layer while keeping the same session ownership model: one `async_sessionmaker`, one `AsyncSession` per task or unit of work.
- SQLModel does not replace SQLAlchemy async lifecycle primitives; it provides model declaration, validation, and typing ergonomics on top of them.
- Do not mix ad hoc session construction with the canonical async dependency.
- Do not mix ad hoc session construction with the canonical session factory.
---
@@ -344,41 +319,44 @@ async def create_item(session: AsyncSession = Depends(get_db_session)) -> dict:
## Anti-Patterns
- A singleton/global AsyncSession reused across requests.
- A singleton/global AsyncSession reused across tasks or operations.
- Sharing one AsyncSession across parallel tasks.
- Passing an application-global AsyncSession to a repository constructor.
- Caching an `AsyncSession` instead of caching `async_sessionmaker`.
- Leaving a cached maker pointing at a disposed or replaced engine.
- Creating a new `async_sessionmaker` in each operation.
- Retaining a session factory after its bound engine scope exits.
- Calling the session factory inside low-level access functions such as `find_item()`.
- Hidden session creation in lower access functions with no caller control.
- Closing or committing a session supplied by the caller.
- Starting a new transaction inside a helper that may receive a session already in a transaction.
- Silently starting or committing a transaction on a supplied session.
- Treating `in_transaction()` as proof that a caller intentionally owns the transaction.
- Creating a SAVEPOINT for ordinary nested service calls.
- Hiding root transaction, joined transaction, and SAVEPOINT behavior behind one mode-driven `atomic_scope()` helper.
- Calling `session.commit()` inside a SAVEPOINT scope.
- Mixing commit/rollback ownership across layers without a declared boundary.
---
## Operational Checks
- Exactly one cached `async_sessionmaker` exists per application engine.
- Session factory caches are cleared before their engines are disposed or replaced.
- Request handlers receive sessions from one canonical dependency.
- Exactly one `async_sessionmaker` is configured inside each application engine scope.
- The session factory does not outlive its bound engine.
- Application operations receive sessions from one canonical session factory.
- No code path creates AsyncSession in module import side effects.
- Background jobs and API handlers each create task-local sessions.
- Concurrent jobs and operations each create task-local sessions.
---
## Testing Checks
- Repository constructors accept a test database URL without FastAPI startup.
- Service constructors accept a test session factory without framework startup.
- Session-taking access functions accept a transaction-scoped test session directly.
- Optional-session tests verify that borrowed sessions remain open and created sessions close.
- Optional-session tests verify that neither path commits implicitly.
- Optional-transaction tests verify supplied sessions require an active transaction and remain caller-owned.
- Optional-transaction tests verify owned transactions commit on success, roll back on failure, and close their sessions.
- Cache tests clear `get_session_factory` before and after replacing engines.
- Dependency override exists for the FastAPI session factory.
- Transaction-scope tests verify supplied sessions require an active transaction and remain caller-owned.
- Transaction-scope tests verify owned transactions commit on success, roll back on failure, and close their sessions.
- Composition tests verify nested service calls join one outer transaction without committing it.
- SAVEPOINT tests verify local rollback preserves the outer transaction and successful exit does not commit it.
- Tests that depend on SAVEPOINT timing account for `begin_nested()` flushing pending state on entry.
- Rollback behavior is verified for failed write units.
- Parallel-task tests verify no shared AsyncSession instances.
- Lifespan tests confirm session factory is initialized and teardown-safe.
- Lifecycle tests confirm the session factory is initialized and teardown-safe.