Contributing

Thank you for your interest in contributing to Toolscore!

This document provides guidelines and instructions for contributing.

Development Setup

Fork and clone the repository:

git clone https://github.com/yotambraun/toolscore.git
cd toolscore

Install development dependencies:

pip install -e ".[dev]"

Verify installation:

pytest
mypy toolscore
ruff check toolscore

Code Style

We use modern Python tooling:

Formatting: ruff format
Linting: ruff check
Type Checking: mypy with strict mode
Testing: pytest with coverage

Run all checks:

# Format code
ruff format toolscore tests

# Lint code
ruff check toolscore tests

# Type check
mypy toolscore

# Run tests
pytest --cov=toolscore

Type Hints

All code must include type hints:

def calculate_metric(
    gold_calls: list[ToolCall],
    trace_calls: list[ToolCall],
) -> float:
    """Calculate some metric.

    Args:
        gold_calls: Expected tool calls.
        trace_calls: Actual tool calls.

    Returns:
        Metric score between 0.0 and 1.0.
    """
    pass

Testing

Write tests for all new functionality:

# tests/unit/test_myfeature.py
def test_my_feature() -> None:
    """Test my new feature."""
    # Arrange
    calls = [ToolCall(tool="test")]

    # Act
    result = my_function(calls)

    # Assert
    assert result is not None

Commit Messages

Use conventional commits format:

feat: add support for custom validators
fix: correct argument matching for nested dicts
docs: update README with new examples
test: add tests for OpenAI adapter
refactor: simplify metric calculation logic

Pull Request Process

Create a feature branch:

git checkout -b feature/your-feature-name

Make your changes with tests and documentation
Ensure all checks pass:

pytest
mypy toolscore
ruff check toolscore

Commit your changes:

git add .
git commit -m "feat: add your feature"

Push and create a pull request:

git push origin feature/your-feature-name

Adding New Features

Adding a New Trace Adapter

Create new adapter in toolscore/adapters/
Inherit from BaseAdapter
Implement the parse method
Add tests in tests/unit/test_adapters.py
Export in toolscore/adapters/__init__.py
Update documentation

Adding a New Metric

Create new metric in toolscore/metrics/
Add tests in tests/unit/test_metrics.py
Export in toolscore/metrics/__init__.py
Integrate in toolscore/core.py
Update documentation

Adding a New Validator

Create validator in toolscore/validators/
Add tests
Register in toolscore/core.py validators dict
Update documentation

Documentation

Update README.md for user-facing changes
Add docstrings to all public APIs (Google style)
Update examples/ if adding new formats
Add inline comments for complex logic

Questions?

Open an issue for bugs or feature requests
Start a discussion for questions
Check existing issues and PRs first

Code of Conduct

Be respectful and inclusive
Focus on constructive feedback
Help others learn and grow

License

By contributing to Toolscore, you agree that your contributions will be licensed under the Apache License 2.0.