Snapshots Module

The storage and state-machine core behind Snapshot Testing. The pytest toolscore_snapshot fixture and the toolscore record/approve/snapshots CLI commands are built on this API.

toolscore.snapshots.snapshot_check(name, actual, *, store=None, update=False, min_score=1.0, weights=None, strict=False)

Record, approve and replay tool-call snapshots.

Extracts tool calls from actual (via toolscore.integrations.auto_extract(), so raw provider responses are accepted) and runs the snapshot state machine:

1. Snapshot missing. Locally: write an unapproved snapshot, emit a UserWarning telling the user to review and approve it, and return None. In CI (CI env var set): raise ToolScoreAssertionError without writing the file — snapshots must be created and reviewed locally, never minted in CI.

2. update=True. Overwrite the snapshot’s calls, mark it approved, emit a UserWarning, and return None.

3. Exists but unapproved. Locally: emit a UserWarning and return None (an unapproved baseline is never evaluated against). In CI: raise ToolScoreAssertionError.

4. Exists and approved. Evaluate actual against the approved baseline with toolscore.evaluate(), enforce min_score, and return the EvaluationResult.

Parameters:

• name (str) – Snapshot identifier (typically the pytest node id).

• actual (Any) – A raw provider response or a list of tool-call dicts. Always run through auto_extract before storage / comparison.

• store (SnapshotStore | None) – Snapshot store to use; None uses a default-rooted store.

• update (bool) – When True, overwrite and approve the snapshot (state 2).

• min_score (float) – Minimum composite score required when replaying (state 4).

• weights (dict[str, float] | None) – Optional custom weights for the composite score.

• strict (bool) – When True, argument comparison uses pure equality.

Return type:

EvaluationResult | None

Returns:

An EvaluationResult when an approved baseline was replayed (state 4); otherwise None.

Raises:

ToolScoreAssertionError – In CI when the snapshot is missing or unapproved, or when an approved replay scores below min_score.

class toolscore.snapshots.Snapshot(name, calls=<factory>, approved=False, source='pytest', created_at='', updated_at='', schema_version=1, toolscore_version='')

A recorded set of tool calls for a single test.

Variables:

• name – Logical identifier (typically a pytest node id).

• calls – List of tool-call dicts ({"tool": ..., "args": {...}}).

• approved – Whether a human has approved this baseline.

• source – How the snapshot was produced — "pytest", "record" or "trace".

• created_at – ISO-8601 UTC timestamp; filled on creation if empty.

• updated_at – ISO-8601 UTC timestamp; filled on creation if empty.

• schema_version – On-disk schema version.

• toolscore_version – Version of toolscore that wrote the snapshot; filled from the package version if empty.

Parameters:

• name (str)

• calls (list[dict[str, Any]])

• approved (bool)

• source (str)

• created_at (str)

• updated_at (str)

• schema_version (int)

• toolscore_version (str)

name: str

calls: list[dict[str, Any]]

approved: bool = False

source: str = 'pytest'

created_at: str = ''

updated_at: str = ''

schema_version: int = 1

toolscore_version: str = ''

__post_init__()

Fill in timestamps and version defaults when not supplied.

Return type:

None

to_dict()

Serialize the snapshot to a JSON-friendly dict.

Return type:

dict[str, Any]

classmethod from_dict(data)

Construct a Snapshot from a dict produced by to_dict().

Return type:

Snapshot

Parameters:

data (dict[str, Any])

__init__(name, calls=<factory>, approved=False, source='pytest', created_at='', updated_at='', schema_version=1, toolscore_version='')

Parameters:

• name (str)

• calls (list[dict[str, Any]])

• approved (bool)

• source (str)

• created_at (str)

• updated_at (str)

• schema_version (int)

• toolscore_version (str)

Return type:

None

class toolscore.snapshots.SnapshotStore(root='.toolscore/snapshots')

File-backed store for snapshots — one JSON file per snapshot.

The root directory is created lazily on the first save(), so merely constructing a store (or listing an empty/absent root) never touches the filesystem in a way that creates directories.

Parameters:

root (str | Path)

__init__(root='.toolscore/snapshots')

Initialize the store rooted at root (no directory is created).

Parameters:

root (str | Path)

Return type:

None

path_for(name)

Return the on-disk path for a snapshot named name.

The filename is <sanitized-name>-<sha1(name)[:8]>.json. The sha1 of the full name guarantees uniqueness even when sanitization collapses distinct names, and keeps every snapshot inside root regardless of path-traversal sequences in the name.

Return type:

Path

Parameters:

name (str)

exists(name)

Return True if a snapshot named name is stored on disk.

Return type:

bool

Parameters:

name (str)

load(name)

Load the snapshot named name, or None if it does not exist.

Raises:

ValueError – If the file exists but cannot be parsed as a valid snapshot (the path is included in the message).

Return type:

Snapshot | None

Parameters:

name (str)

save(snapshot)

Persist snapshot to disk, creating the root directory if needed.

The snapshot’s updated_at timestamp is refreshed before writing.

Return type:

Path

Returns:

The path the snapshot was written to.

Parameters:

snapshot (Snapshot)

approve(name)

Mark the snapshot named name as approved and persist the change.

Raises:

KeyError – If no snapshot named name exists.

Return type:

Snapshot

Parameters:

name (str)

delete(name)

Delete the snapshot named name if it exists (no error if absent).

Return type:

None

Parameters:

name (str)

list()

Return all snapshots in the store, sorted by name.

Unparseable *.json files in the root are skipped rather than raising, so a single corrupt file does not break listing the rest.

Return type:

list[Snapshot]

pending()

Return all unapproved snapshots, sorted by name.

Return type:

list[Snapshot]