What does a quiet MCP tool diff not tell you?
A structural diff of saved MCP tool objects, set beside the findings of a policy classifier, across released versions of five packages.
Finding
Across 20 transitions between released versions of five packages, 14 changed the tool contract. 7 of those 14 had no historical policy finding.
In practice
No policy finding is not the same as no change. Re-review a tool when its contract moves, whatever the classifier says.
Why this question matters
An agent trusts a tool through its description and schema, and those change between releases. A policy check that stays quiet can look like proof that nothing important moved.
What I built
A path-by-path diff of saved MCP tool objects, with the policy classifier's historical findings shown beside each change, and a GitHub Action for review on pull requests.
A design choice
I diff the whole saved tool object, path by path, and show the policy classifier's findings beside each change rather than in place of it. Reporting only the verdict would be shorter, but a change with no finding would disappear. Showing both is what makes the 7 unlabelled changes visible. It also means the counts describe what was missed by labels, not how often a classifier is wrong.
Where the evidence stops
Five packages chosen on purpose from three publisher families, with no human review study and no independent approval labels. The counts are not false-negative rates.