Both sides said success.
The state disagreed.
I built a harness that compares both the response and the resulting state against a real system: a pinned local Gitea service (an open-source Git server).
A mock can agree on the reply and still leave a different state behind.
Recorded pilot run.
Now check what each side stored.
- responses differed: E04, E09
- state differed: E06
E06 response: two create_label calls for p2-eval-dupe, colours 111111 then 222222. The simulator and Gitea both returned success with the same name and colour each time.
16 new evaluation cases. 13 agreed; 3 diverged. Read the recorded result
E06 responses agreed: both returned success
| Case | Calls sent to both | Response | Resulting state |
|---|---|---|---|
| E01 | create_issue, create_issue, get_issue | agreed | agreed |
| E02 | edit_issue, edit_issue, get_issue | agreed | agreed |
| E03 | add_label, add_label, get_issue | agreed | agreed |
| E04 | remove_label, remove_label, get_issue | differed: Gitea succeeded, the simulator returned 404 | agreed |
| E05 | create_label | agreed | agreed |
| E06 | create_label, create_label | agreed | differed: Gitea kept 2 label rows, the simulator kept 1 |
| E07 | create_issue, get_issue | agreed | agreed |
| E08 | edit_issue, edit_issue, get_issue | agreed | agreed |
| E09 | edit_issue, add_label, get_issue | differed: Gitea succeeded, the simulator returned 404 | agreed |
| E10 | remove_label | agreed | agreed |
| E11 | create_label, add_label, add_label, get_issue | agreed | agreed |
| E12 | edit_issue, remove_label, edit_issue, get_issue | agreed | agreed |
| E13 | create_issue, add_label, get_issue | agreed | agreed |
| E14 | create_label, create_label, add_label, get_issue | agreed | agreed |
| E15 | get_issue, get_issue | agreed | agreed |
| E16 | create_label, create_label | agreed | agreed |
- Design choice
- A strict observation contract decides what counts as the same: duplicates are kept, and raw database IDs are not compared literally.