Expert: deep learning and AI · Building with generative AI, and MLOps · topic 55 of 62 · 35 min
Evaluating LLM apps
Explanation
Why "it looks good" is not enough
Trying three questions in a chat window and feeling happy is not evaluation. A change that fixes one answer can break five others. LLM outputs also vary between runs. Teams that ship reliable AI features treat evaluation like testing in software: a fixed set of cases, scored automatically, run on every change.
Build a golden test set
- Collect real user questions (with permission), not only ones you invented.
- Write the expected answer or the key facts for each, and where relevant the source document that should be retrieved.
- Include hard cases: questions with no answer in the documents, ambiguous ones, Hindi or mixed-language ones, and attempts to misuse the system.
- Start with 50 to 200 cases; grow the set whenever a bug is found in production.
Automatic metrics
| Metric | What it checks | Good for |
|---|---|---|
| Exact match (after normalising) | Is the answer exactly right? | Short answers: a date, an amount, a category |
| Token F1 | Overlap of words between answer and reference | Short phrases where wording may differ |
| Recall@k (retrieval) | Was a correct source among the top k retrieved chunks? | Debugging RAG retrieval |
| Format validity | Does the output parse as the required JSON? | Any app that reads model output with code |
| Groundedness / faithfulness | Is every claim supported by the retrieved sources? | RAG answers; checked by humans or a judge model |
Normalising before comparing (lowercase, remove punctuation, collapse spaces) avoids failing "7 days." against "7 days". Token F1, used by the SQuAD reading benchmark, counts the words the answer and the reference share: precision is shared ÷ answer length, recall is shared ÷ reference length, and F1 combines them.
LLM-as-a-judge
For long answers, teams often ask a strong LLM to grade answers against a rubric ("Is it correct? Is it supported by the sources? Is it polite?"). This scales well, but judges have biases: they may prefer longer answers, answers in a certain position, or their own style. Use clear rubrics, ask for a short reason, check a sample of judgements by hand, and measure how often the judge agrees with humans.
Beyond accuracy
Track latency (how long users wait), cost per request, refusal rate (how often it wrongly refuses), and safety failures. Run the same test set for every prompt change, model change and retrieval change, and keep the results so you can compare versions. Monitor real usage after launch with user feedback and sampled reviews.
Common mistakes
- A test set made only of easy, invented questions.
- Comparing versions on different questions or on a single run.
- Trusting an LLM judge without ever checking it against people.
- Measuring only answer quality, not whether retrieval found the right document.
Need help?
Stuck on a word or an idea? Search this course, or every course, and jump straight to the explanation or a worked question.
Try:
Watch the lesson
Evaluating LLM apps · 1/6
Trying a few questions in a chat is not evaluation. Build a test set.
Prefer a video by a teacher? Find free videos on “how to evaluate LLM applications RAG evaluation exact match F1 LLM as a judge” on YouTube ↗
In real life: Evaluating an insurance FAQ bot
Before launch, an insurer builds 150 test questions from real customer emails, each tagged with the correct policy clause. For every new prompt or retrieval setting, a script reports recall@5 for the clause, exact match on numbers (waiting periods, limits), token F1 on short answers, and a judge model's groundedness score, with 20 answers checked by a person each week. A change that raised answer fluency but dropped recall@5 from 0.92 to 0.81 was rejected.
Practice (0/3 done)
Write normalize(text): lowercase it, remove every character in string.punctuation, and collapse runs of whitespace into single spaces with no spaces at the ends. Then write exact_match(preds, golds) that returns the share of positions where normalize(pred) == normalize(gold), rounded to 4 decimal places (0.0 for empty lists).
Write token_f1(pred, gold) like the SQuAD metric: split both strings into words (lowercase, .split()), count shared words with collections.Counter (pred_counts & gold_counts), and if none are shared return 0.0; otherwise precision = shared / len(pred_words), recall = shared / len(gold_words), F1 = 2PR / (P + R), rounded to 4 decimal places.
Write recall_at_k(retrieved, relevant, k) where retrieved is a list (one per question) of ranked document ids and relevant is a list of the correct id for each question. Return the share of questions whose correct id is within the first k retrieved ids, rounded to 4 decimal places (0.0 for no questions).
Quick quiz
Read more (free)
Related help
Before this, learn
This topic builds on these. If something here feels new, open them first.
Related topics
- AI safety and responsible deploymentBuilding with generative AI, and MLOps · same module
- Prompting wellNLP, embeddings and large language models
- How large language models workNLP, embeddings and large language models
- Monitoring and observability: metrics, logs and tracesDevOps: Git, Docker, Kubernetes and CI/CD · Monitoring and logging
- Career paths: analyst, scientist, ML engineer, AI engineerEthics, privacy and careers
- Logging and monitoringBackend Development · Performance and reliability
Questions people ask about this
Open a question to see the answer and why. The first ones are this topic's own quiz, so try the quiz above before you peek.
Why normalise answers before exact match?
Answer: So harmless differences like case and punctuation do not count as errors
"7 days." and "7 days" should match.
A RAG bot gives wrong answers. recall@5 is 0.4. What should you fix first?
Answer: Retrieval: the right source is often not even retrieved
Without the right source, the answer cannot be grounded.
What is a known weakness of LLM-as-a-judge?
Answer: It can prefer longer answers or certain styles, so it needs checking against humans
Judges have biases; validate them.
Which belongs in a good test set?
Answer: Real questions, including ones with no answer and misuse attempts
Hard and realistic cases expose real failures.
Besides answer quality, what should an LLM app track?
Answer: Latency, cost per request and safety failures
A correct answer that is too slow or too costly can still fail the product.
Precision is 1.0 and recall is 0.1. What is F1, roughly?Related
Answer: 0.18
2 × 1 × 0.1 ÷ 1.1 ≈ 0.18. F1 stays low unless both are high.
Where should access control be applied in a RAG system?Related
Answer: Before retrieved text reaches the model
Text the user may not see must never enter their prompt.
What is the best way to improve a prompt?Related
Answer: Test on a set of real examples and change one thing at a time
A small test set turns prompting into engineering.
From Prompting well
Why can an LLM state a wrong fact confidently?Related
Answer: It generates likely-sounding text and does not verify facts
That is hallucination; check important outputs.
Pass the quiz (60% or more) or mark this topic as done when you are ready.
Your progress is saved on this device only.
Next steps
- 2 more topics in “Building with generative AI, and MLOps” before its exam: AI safety and responsible deployment, MLOps: from notebook to production.
- Module 11 exam: Building with generative AI, and MLOps · 12 questions · 25 min, without help. 5 topics of this module still to finish first.
- Then the next module, Ethics, privacy and careers. 62 topics left in the course before the final exam.