60-second start
Up and running in 60 seconds
Install it, pick one of three entry points, and read the verdict. No API key, no cloud, no per-test bill.
Install
pip install tool-scorer Three ways in
Pick an entry point
See it in 10 seconds
Grade a bundled sample MCP server — no install, no API key, no setup. The fastest way to see the scorecard.
Grade your MCP server
Point it at your own server. Add --fail-under B to gate CI, or --report md to export a Markdown scorecard.
Evaluate a trace
Score a captured agent trace against a gold standard. Get an A–F grade and the same "Top issues to fix" verdict.
The agent loop
Scaffold, record, approve, replay
toolscore init
detects your framework and writes a passing pytest suite plus a CI workflow. From there
it's record once, approve, and replay forever.
toolscore init # scaffold a passing pytest suite
pytest # records snapshots
toolscore approve --all # bless them as the baseline
pytest # replays — and fails on driftWire it into CI
A GitHub Actions gate, copy-paste
Drop this workflow in to grade your MCP server on every push and PR. It fails the build below a B and writes the verdict to the run summary.
name: toolscore
on: [push, pull_request]
jobs:
scorecard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install tool-scorer
# Grade the MCP server on every PR; fail the build below a B
- run: toolscore mcp test "python my_server.py" --fail-under B --ci
Prefer the prebuilt action? Use yotambraun/toolscore@v1 with mcp-command and
mcp-fail-under inputs — or
gold-file / trace-file
for gold-standard evaluation.
Go deeper
Learn more
Grade your tool-calling in seconds
Install it, point it at a server or an agent, and read the verdict. Free, deterministic, offline.
pip install tool-scorer uvx tool-scorer demo