Pages
Stores Sections
Overview Store Protocol Concurrency Audit Trail SQLite Redis PostgreSQL In-Memory

Reference

Stores

BeliefState persists extracted beliefs in a configurable store backend. Choose based on your deployment model.

Stores Overview

The store acts as the memory bank of the tracking layer. Extracted belief structures are persisted to the backend to support session consistency across multiple turns.

Default for testing

In-Memory

  • Best for: unit tests, local scripts, prototyping
  • Persistence: none — lost on restart
  • Concurrency: single process only
  • Setup: zero configuration
Recommended for single server

SQLite

  • Best for: single-server apps, low-traffic
  • Persistence: file-based, survives restarts
  • Concurrency: single process (WAL mode reads)
  • Setup: single file path config
Recommended for multi-worker

Redis

  • Best for: multi-worker, load-balanced
  • Persistence: configurable (RDB/AOF)
  • Concurrency: multi-process safe
  • Setup: Redis server connection
Note

The tracking store backend is configured by passing the store_type and store_kwargs properties within your TrackerConfig.

Store Protocol

All store backends implement the Store protocol, which defines the public interface for belief persistence. Any custom backend must implement these methods:

Method Signature Description
upsert async upsert(belief, session_id, conversation_id, turn) Insert or update a belief with turn-based optimistic concurrency.
get_beliefs async get_beliefs(session_id, conversation_id=None, category=None, belief_type=None) Query beliefs with optional filters.
get_by_key async get_by_key(subject, predicate, session_id) O(1) exact lookup by belief triple key.
search_beliefs async search_beliefs(session_id, embedding, threshold, limit, conversation_id=None) Semantic search via cosine similarity on stored embeddings.
get_audit_history async get_audit_history(subject, predicate, session_id, limit=10) Retrieve version history for a belief from the audit trail.
clear_session async clear_session(session_id) GDPR deletion — removes all beliefs for a session.

Turn-Based Optimistic Concurrency

All store backends enforce turn-based optimistic concurrency on upsert(). Each write includes a turn parameter — the store compares it against the stored belief's turn and only overwrites if the incoming turn is newer or equal. This prevents stale background writes from overwriting fresh data when concurrent tasks race.

Tip

Turn-based concurrency is a lightweight guard. For coordinated writes within a single session, the tracker also applies per-session async write locks to serialize concurrent background tasks.

Audit Trail

SQLite and PostgreSQL backends maintain a beliefs_audit table that logs every belief mutation. Each row records the full belief state at the time of change, enabling version history queries via get_audit_history().

Column Type Description
id INTEGER Auto-incrementing audit record ID.
session_id TEXT Session that owns this belief.
subject TEXT Belief subject.
predicate TEXT Belief predicate.
value TEXT Belief value at mutation time.
confidence REAL Confidence score at mutation time.
turn INTEGER Turn number when the mutation occurred.
changed_at TEXT ISO 8601 timestamp of the mutation.

SQLite Store

The SQLite backend persists belief triples directly to a local database file. It operates asynchronously using aiosqlite connection pools.

sqlite_config.py
from beliefstate import TrackerConfig # File-based database configuration config = TrackerConfig( store_type="sqlite", store_kwargs={"db_path": "beliefs.db"} ) # Ephemeral memory configuration for testing test_config = TrackerConfig( store_type="sqlite", store_kwargs={"db_path": ":memory:"} )
Parameter Type Default Description
db_path str "beliefs.db" Path pointing to the SQLite database file. Provide ":memory:" to use in-memory SQLite tables.

WAL Mode

Write-Ahead Logging (WAL) is activated automatically on connection. WAL mode allows concurrent read queries to progress without lockouts from background write tasks.

The store applies these PRAGMA operations during initialization:

sqlite_pragmas.sql
PRAGMA journal_mode=WAL PRAGMA synchronous=NORMAL PRAGMA foreign_keys=ON

Connection Management

The store opens one persistent sqlite database connection via open() and closes it via close(). The client supports async context managers.

sqlite_context.py
from beliefstate.store.sqlite import SQLiteStore async with SQLiteStore(db_path="beliefs.db") as store: # Storage queries

Schema

The tracking database creates a structured table layout to record conversational context:

Column Type Description
session_id TEXT Active conversation session identifier. Part of primary key.
conversation_id TEXT Active thread identifier. Used to isolate thread pools.
subject TEXT Subject part of the belief triple. Part of primary key.
predicate TEXT Predicate part of the belief triple. Part of primary key.
value TEXT Normalized factual claim value.
confidence REAL Confidence score (0.0 to 1.0) generated by the extraction model.
turn INTEGER The turn number index when this claim was established.
source TEXT Source actor that asserted the claim: "user" or "assistant".
belief_type TEXT Fact type: "assertion" or "update".
is_hypothetical INTEGER Boolean flag indicating speculative statements (0 or 1).
embedding BLOB Binary representation of the vector embedding array.
embedding_model TEXT Name of the embedding model used to calculate vectors.
embedding_dim INTEGER Dimensionality of the vector embedding array.
created_at TEXT ISO 8601 timestamp logging row creation.
last_referenced_at TEXT ISO 8601 timestamp logging last access.
⚠ Important

Do not use :memory: with multiple connections. Each connection sees a separate empty database. Use a file path for persistence.

Redis Store

The Redis backend offloads tracking state to an external server. It supports concurrent connections across multiple worker nodes.

redis_config.py
from beliefstate import TrackerConfig config = TrackerConfig( store_type="redis", store_kwargs={"redis_url": "redis://localhost:6379/0"} )
Parameter Type Default Description
redis_url str "redis://localhost:6379/0" Connection connection URI indicating server IP, port, and logical database index.

Key Structure

The store maps belief triples to redis keys using a hash-based structure. All beliefs for a session are stored in a single Redis hash keyed by session ID:

beliefstate:session:{session_id}

Each hash field is {conversation_id}:{subject}:{predicate} and the value is a JSON-serialized Belief object.

Example keys generated by the store backend:

redis_keys.txt
beliefstate:session:user_123 field: :USER:likes → {"subject":"USER","predicate":"likes",...} field: :USER:budget → {"subject":"USER","predicate":"budget",...}

Embedding Storage

To save memory and preserve float precision, the store packages embedding vectors into binary byte BLOBs using the Python struct library. This prevents precision loss and shrinks payload size by 4x.

binary_pack.py
import struct def pack_embedding(vector: list[float]) -> bytes: return struct.pack(f"{len(vector)}f", *vector) def unpack_embedding(data: bytes) -> list[float]: count = len(data) // 4 return list(struct.unpack(f"{count}f", data))

TTL Support

Allows setting expiration intervals on session keys. Keys expire automatically after the TTL duration ends, cleaning up inactive records.

redis_ttl.py
# Set session data TTL limit to 24 hours await store.set_session_ttl("user_123", ttl_seconds=86400)

Multi-Worker Safety

Redis operates atomically within a single-threaded server execution loop. This isolates commands and allows multi-worker instances to modify user session facts concurrently without race conditions.

💡 Tip

For Celery or RQ deployments, use RedisBeliefStore — the same Redis instance can serve both your task queue and belief store.

PostgreSQL Store

The PostgreSQL backend provides full production-grade persistence with concurrent connection support, audit trails, and turn-based optimistic concurrency.

postgres_config.py
from beliefstate import TrackerConfig config = TrackerConfig( store_type="postgres", store_kwargs={ "dsn": "postgresql://user:pass@localhost:5432/beliefs" } )
Parameter Type Default Description
dsn str required PostgreSQL connection string (DSN).
min_pool_size int 2 Minimum number of connections in the pool.
max_pool_size int 10 Maximum number of connections in the pool.

Features

  • Turn-based upsert — uses ON CONFLICT ... DO UPDATE with turn comparison for optimistic concurrency.
  • Audit trail — every mutation is logged to a beliefs_audit table for version history.
  • Binary embeddings — embeddings stored as binary BLOBs using struct.pack for precision and space efficiency.
  • Connection pooling — async connection pool with configurable min/max sizes.
  • Semantic search — computes cosine similarity in Python over fetched embeddings (consider pgvector extension for large datasets).
Note

Requires the asyncpg package: pip install asyncpg.

In-Memory Store

The default in-memory store persists records inside local dictionary variables. Data resets when the application process terminates.

memory_default.py
from beliefstate import BeliefTracker, TrackerConfig # Defaults to InMemoryBeliefStore when store_type is omitted tracker = BeliefTracker(config=TrackerConfig())

When to Use

  • Executing unit tests and local integration flows.
  • Prototyping logic and running one-off scripts.
  • Testing tracking pipelines before configuring database instances.
  • Never in production deployments.

LRU Eviction

To prevent memory leaks during long-running processes, the store maintains an OrderedDict and evicts older, least-recently-used beliefs once memory usage exceeds max_bytes (default 100 MB).

lru_setup.py
from beliefstate.store.memory import InMemoryBeliefStore # Cap memory usage to 50 MB store = InMemoryBeliefStore(max_bytes=50 * 1024 * 1024)

Thread Safety

The store uses asyncio-safe OrderedDict operations. It is safe for concurrent access within a single Python process loop, but cannot synchronize across multiple server worker processes.

⚠ Important

Do not use InMemoryBeliefStore in production. All beliefs are lost when the process restarts.