# What does a quiet MCP tool diff not tell you?

Agent Tool Contract Drift: Research software. Published 20 Oct 2026. Question: Can we trust the score?
Scope: 5 packages · 25 versions · 20 transitions
Page: https://jasonlovell.ai/work/tool-contract-drift
Code: https://github.com/jlov7/tool-contract-drift

A structural diff of saved MCP tool objects, set beside the findings of a policy classifier, across released versions of five packages.

## Finding

Across 20 transitions between released versions of five packages, 14 changed the tool contract. 7 of those 14 had no historical policy finding.

## In practice

No policy finding is not the same as no change. Re-review a tool when its contract moves, whatever the classifier says.

## What I built

A path-by-path diff of saved MCP tool objects, with the policy classifier's historical findings shown beside each change, and a GitHub Action for review on pull requests.

## Where the evidence stops

Five packages chosen on purpose from three publisher families, with no human review study and no independent approval labels. The counts are not false-negative rates.

---
By Jason Lovell. Independent work, separate from my role at PwC. I build everything here myself, end to end, with Claude Code and Codex. I write the question and the test first, read the runs, and keep the results that go against me.