Matchers Module
Argument matchers are placed as values inside expected args dicts to assert
on the shape of an argument rather than its exact value. See Argument Matchers
for a guide with runnable examples.
Argument matcher objects for use inside expected-args dicts.
Matchers are placed as values inside the expected argument dicts passed
to toolscore.evaluate(). They work via operator overloading:
Matcher.__eq__(other) performs the match, so they compose transparently
with plain dict equality and with the argument-comparison logic in
toolscore.metrics.arguments.
Example:
from toolscore import evaluate, ANY, Regex, Approx
result = evaluate(
expected=[{"tool": "get_weather", "args": {"city": Regex(r"NYC|JFK")}}],
actual=[{"tool": "get_weather", "args": {"city": "NYC"}}],
)
assert result.argument_f1 == 1.0
- toolscore.matchers.ANY: _AnyMatcher = ANY
Module-level singleton; matches any value.
- class toolscore.matchers.Approx(value, rel=1e-06, abs=0.0)[source]
Bases:
MatcherNumeric closeness matcher (similar to
pytest.approxsemantics).Matches
intandfloatvalues but explicitly excludesbool(even thoughboolis a subclass ofintin Python).The comparison uses the larger of the relative and absolute tolerances:
|actual - expected| <= max(rel * |expected|, abs_tol)
- Parameters:
value (
float) – The expected numeric value.rel (
float) – Relative tolerance (default 1e-6).abs (
float) – Absolute tolerance (default 0.0). Note this differs frompytest.approx, whose default absolute tolerance is1e-12. Withabs=0.0the match is purely relative, so a comparison against an expected value of0requires exact equality.
- class toolscore.matchers.Contains(item)[source]
Bases:
MatcherMembership matcher: checks
item in value.Works for
str,list,tuple,set, anddict(key membership for dicts). Non-container types never match.- Parameters:
item (
object) – The item to look for inside the value.
- class toolscore.matchers.IsType(*types)[source]
Bases:
MatcherType-check matcher using
isinstance().Note
IsType(int)does not matchTrueorFalseeven thoughboolis a subclass ofintin Python. This avoids a common footgun when you want to match plain integers but not accidental booleans. UseIsType(bool)explicitly to match booleans.- Parameters:
*types (
type) – One or more types to check against.
- class toolscore.matchers.Matcher[source]
Bases:
ABCAbstract base class for all argument matchers.
Subclasses implement
matches()which is called by__eq__. Because__eq__is overridden,__hash__must be explicitly preserved — we delegate toobject.__hash__()so matchers remain usable as dict keys / set members.
- class toolscore.matchers.OneOf(*values)[source]
Bases:
MatcherValue-is-one-of matcher.
Checks whether value equals any of the provided values. The comparison uses
==so the provided values may themselves beMatcherinstances (their__eq__will be invoked).- Parameters:
*values (
object) – Candidate values (or Matchers) to test against.