We measured Sonnet 5.5 and three Haiku 5.5 effort levels on the CI rule check. The smaller model at its default effort got 54 of 54 known-answer runs right against 51 of 54, at about 1/18 of the cost per CI run, and raising the effort made it flag more violations. Small sample, one repository.
For: Engineers who run an LLM as a judge in CI. Published: 2026-11-10. Evidence: benchmark source at a tag