Evaluating LLM outputs for code: what reviewers look for

Code evaluation is among the most valuable AI training work, and among the easiest to get wrong. This is the rubric we use internally.

The rubric

Criterion	Question to ask
Correctness	Does it run and handle the edge cases?
Instruction following	Did it do what was asked, and only that?
Safety	Any injection, secrets, or unsafe defaults?
Clarity	Would a teammate understand it quickly?
Honesty	Does the explanation match what the code does?

Tips

• Run the code whenever the task allows it, instead of judging by reading alone.
• Penalise confident explanations of code that doesn't work.
• Prefer the simpler correct answer over the clever one.

This rubric is part of the onboarding material in our AI Training Assistance program.