Malhar Ujawane

Blog

Explainer·(updated October 31, 2026)

What is LLM-as-a-judge? Using models to grade models

LLM-as-a-judge uses a strong model to grade another model's output against a rubric. How it works, its known biases and how to calibrate it, with a Python demo.

What is LLM-as-a-judge? Using models to grade models

LLM-as-a-judge is an evaluation method where a strong language model grades another model’s output, either by scoring one answer against a rubric or by picking the better of two answers. It stands in for human reviewers on qualities code can’t check, such as helpfulness, tone or whether a summary is grounded in its sources. It’s cheap and scales well, and it carries measurable biases you have to correct for.

Where did LLM-as-a-judge come from?

The method was around before the name, but the paper that defined it is Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena by Zheng et al. (June 2023, NeurIPS 2023 Datasets and Benchmarks Track). The authors needed to compare chat assistants on open-ended questions, where existing benchmarks with fixed answers didn’t capture what people preferred, and human ratings were expensive.

Their headline result is why everyone adopted it: strong judges like GPT-4 matched both controlled and crowdsourced human preferences with “over 80% agreement, the same level of agreement between humans.” On MT-bench, with ties excluded, GPT-4’s agreement with human experts reached 85%, higher than the 81% agreement between humans.

The same paper is also the best catalogue of what goes wrong, which is the half people skip.

How does LLM-as-a-judge work?

Zheng et al. describe three variations, and nearly every judge setup today is one of them:

  • Pairwise comparison. The judge sees a question and two answers and picks the better one, or a tie. Good for A/B testing a prompt change or a model swap.
  • Single answer grading. The judge scores one answer directly, for example from 1 to 10 against a rubric. Good for monitoring and for absolute quality bars.
  • Reference-guided grading. The judge gets a reference solution and grades against it. Useful for math and reasoning, where judges are otherwise easily fooled.

Their default pairwise prompt opens with “Please act as an impartial judge,” asks for a short explanation before the verdict, and tells the model to avoid position bias and not to let length or assistant names sway it. Asking for the explanation first matters; it makes the verdict auditable.

In agent evals, a judge is one grader among several. Anthropic’s guide to agent evals puts it last in line: deterministic graders where possible, LLM graders where necessary, humans for calibration. I cover the rest of that setup in how to evaluate AI agents.

What biases do LLM judges have?

The 2023 paper measured three, and later work kept finding them.

Position bias. The judge favors an answer because of where it appears. Zheng et al. built pairs of similar answers and asked judges twice with the order swapped. GPT-4 gave a consistent verdict 65% of the time, and when inconsistent it favored the first answer 30% of the time against 5% for the second. Claude-v1 was consistent only 23.8% of the time, favoring the first position in 75% of cases.

Verbosity bias. The judge prefers longer answers even when they add nothing. The paper’s “repetitive list” attack padded answers by rephrasing list items without new information, and judges fell for it.

Self-enhancement bias. Judges may favor answers from their own model. In their data, GPT-4 favored itself with a 10% higher win rate and Claude-v1 with 25%, compared with human judges. The authors were careful to say the data wasn’t enough to fully confirm this.

Limited reasoning. On math questions, judges marked wrong answers as correct, even when they could solve the problem themselves if asked separately. The paper’s fix is to have the judge solve the problem first, or to give it a reference answer.

How do you correct for position bias?

The paper’s conservative fix: call the judge twice with the answers swapped, and declare a winner only if both orders agree. Otherwise it’s a tie. Few-shot examples also helped; they raised GPT-4’s consistency from 65.0% to 77.5%.

Here is that fix on a simulated judge. judge() stands in for a model call: it prefers the better answer, but on close calls it leans toward whichever answer it reads first. The human labels are the hidden truth.

# judge_swap.py: measure a pairwise judge's position bias, then fix it by swapping.
# Python 3 standard library only. judge() is a stand-in for an LLM call that
# prefers the better answer but leans toward whichever answer it reads first.
import random

random.seed(3)
N = 400
quality = [(random.random(), random.random()) for _ in range(N)]   # hidden truth per pair
human = ["A" if a > b else "B" for a, b in quality]                # what experts would say

def judge(first, second):
    """Returns 'first' or 'second'. Close calls go to whichever came first."""
    gap = first - second + random.gauss(0, 0.15)
    return "first" if gap > -0.12 else "second"                    # the bias lives here

def single_order(a, b):
    return "A" if judge(a, b) == "first" else "B"

def swapped(a, b):                                                 # Zheng et al.'s conservative fix
    v1 = "A" if judge(a, b) == "first" else "B"
    v2 = "B" if judge(b, a) == "first" else "A"
    return v1 if v1 == v2 else "tie"

def kappa(x, y):                                                   # Cohen's kappa, two raters
    labels = set(x) | set(y)
    po = sum(i == j for i, j in zip(x, y)) / len(x)
    pe = sum((x.count(l) / len(x)) * (y.count(l) / len(y)) for l in labels)
    return (po - pe) / (1 - pe)

one = [single_order(a, b) for a, b in quality]
both = [swapped(a, b) for a, b in quality]
rev = ["A" if judge(b, a) == "second" else "B" for a, b in quality]

print(f"verdict unchanged after swapping order: {sum(x == y for x, y in zip(one, rev)) / N:.0%}")
print(f"single order: picks A {one.count('A') / N:.0%}, agrees with humans {sum(x == h for x, h in zip(one, human)) / N:.0%}, kappa {kappa(one, human):.2f}")
decided = [(v, h) for v, h in zip(both, human) if v != "tie"]
print(f"swap-and-agree: {both.count('tie')} ties, agreement on the rest {sum(v == h for v, h in decided) / len(decided):.0%}")

Output (Python 3.13.11):

verdict unchanged after swapping order: 77%
single order: picks A 61%, agrees with humans 87%, kappa 0.73
swap-and-agree: 89 ties, agreement on the rest 97%

The simulated judge looks good on raw agreement (87%), but it picks the first answer 61% of the time when the truth is close to 50/50, and its verdict flips on 23% of pairs when you swap the order. The swap protocol turns those flips into ties: 89 of 400 pairs become undecided, and on the rest the judge agrees with humans 97% of the time. You trade coverage for trust, and you pay for two calls per comparison.

Cohen’s kappa is there because raw agreement flatters a judge when one label dominates. It corrects for the agreement you’d get by chance, so it’s the number I’d report when calibrating a judge against human labels.

When should you use an LLM judge, and when not?

Use one when the quality you care about is real but not checkable by code: is the answer helpful, does a summary stay faithful to its sources, did the support agent’s tone match policy. Anthropic’s guide uses model-based graders for exactly these cases, for conversational and research agents, and recommends calibrating them against human experts.

Don’t use one where code can decide. If the question is “did the tests pass”, “does the JSON validate” or “is the booking in the database”, a judge adds cost and noise. Benchmarks like SWE-bench are trusted precisely because their graders are tests. The same caution applies to RAG: faithfulness and relevance scores in RAG evaluation frameworks are often LLM judges underneath, so they inherit these biases.

Judges also show up in training. Reward models in RLHF and AI feedback setups are, in effect, judges whose verdicts become gradient updates, which is a strong reason to understand their biases before trusting them in a loop.

What I would do

  • Write the rubric as a checklist, one criterion per call. “Is every claim supported by the retrieved passages? yes/no/unknown” beats “rate quality 1 to 10.” Anthropic’s guide recommends grading each dimension with an isolated judge.
  • Give the judge an out. An “unknown” or “cannot tell” option reduces confident guessing.
  • Swap the order for pairwise comparisons and count disagreements as ties. Report the tie rate; a high one means the two systems are closer than any single verdict suggests.
  • Calibrate before you trust. Label 50 to 100 examples by hand, run the judge, and report agreement and kappa. Recalibrate when you change the judge model or the rubric.
  • Use a different model family as judge when comparing your own model’s outputs, to reduce self-preference.
  • Control for length. Check whether the judge’s preferences correlate with answer length; if they do, add a rubric line about concision or compare length-matched pairs.
  • Keep reading samples. A judge is a measurement instrument, and instruments drift.

FAQ

How accurate is LLM-as-a-judge?

In the original MT-Bench and Chatbot Arena study, GPT-4 as a judge agreed with human preferences over 80% of the time, about the same as humans agree with each other. Accuracy on your task depends on the rubric and the judge model, so measure it against your own human labels.

What is position bias in LLM evaluation?

The tendency of a judge to prefer an answer because of its position, usually the first one shown. The standard fix is to judge each pair twice with the order swapped and only count consistent verdicts.

What is the difference between pairwise and pointwise LLM evaluation?

Pairwise asks the judge which of two answers is better; pointwise (single answer grading) asks it to score one answer on its own. Pairwise is more sensitive to small differences; pointwise gives absolute scores you can track over time.

Can an LLM judge its own outputs?

It can, but Zheng et al. saw signs of self-enhancement bias: GPT-4 and Claude-v1 rated their own answers higher than humans did, though the authors say their data was too limited to confirm it. Use a different model family as judge where you can, and calibrate against humans either way.

Sources

Get new posts in your inbox

© 2026 Malhar Ujawane. Built with Astro.