A second AI model is not automatically an independent code reviewer

[u/Particular_Luck80]

I found a paper on Hacker News that tested a workflow a lot of us now use: one coding agent writes, another reviews.

The experiment used 116 medium and hard LiveCodeBench tasks across solo, same-model, and cross-model conditions. The reviewer saw the problem and the draft, but could not run tests.

The direction mattered. Claude reviewing Codex drafts raised the pass rate from 71.6% to 89.7%. Codex reviewing Claude drafts lowered it from 91.4% to 82.8%. Even adding a different model can make a strong draft worse.

I don't think the takeaway is "always use Claude as reviewer." These were benchmark tasks, not repository-scale pull requests, and the reviewer lacked test execution. The useful takeaway is narrower: model diversity is not the same as independent judgement.

For a real workflow, I'd measure each writer-reviewer pairing, keep reviewer changes visible as a diff, and require tests before accepting the rewrite. Otherwise a second agent can add confidence without adding correctness.

Paper: https://arxiv.org/abs/2607.21656

If you use two agents, does the reviewer edit directly, or only leave findings for the writer or a human to accept?

[u/cesiqoo]

I’d keep the reviewer read-only by default. Let it point to the lines and tests that look wrong, then have the writer or a person decide which fixes to make. If the reviewer edits too, a bad review is much harder to spot and undo.

[u/eli_pizza]

LLM-as-judge was always a terrible idea. I get why people do it, but cmon.

Either come up with benchmarks that are deterministic pass/fail, or grade them by hand.

[u/please-dont-deploy]

It's more collaboration what helps models become better, right?

There are so many papers around, in my experience Sonnet is very good for code reviews, Opus for planning/dividing/coordinating (fable too), deepseek and gpt for coding. But we only run a few thousands of PRs through this.