Prompt Evaluation (Promptfoo, DeepEval, RAGAS)

The Restaurant Menu Analogy

Imagine you run a restaurant and change the recipe for butter chicken. You taste it once and think "yeah, it is fine." But would you serve it to 10,000 customers based on one taste test? Of course not! You would get multiple people to taste it, compare it to the old recipe, check consistency across batches. Prompt evaluation is the same idea - you systematically test your prompts across many inputs to make sure they consistently deliver quality results.

Key Concepts

• Prompt Testing - Running your prompt against a dataset of inputs and checking if outputs meet quality criteria
• Evaluation Metrics - Measurable scores like correctness, relevance, faithfulness, toxicity
• Regression Testing - Making sure a prompt change that fixes one thing does not break ten other things
• Automated Evaluation - Using LLMs or heuristics to score outputs at scale instead of manual review

Why Manual Testing Fails

• Scale - You cannot manually review 500 outputs every time you tweak a prompt
• Bias - You tend to test cases you already know work well
• Consistency - Different reviewers give different scores for the same output
• Regression - You fix one edge case and accidentally break three others without noticing

1. Promptfoo - The Prompt Testing Swiss Army Knife

Think of Promptfoo as Jest/Mocha but for prompts. You define test cases in YAML, run them against multiple models or prompt versions, and get a comparison table.

• Best For: Comparing prompt versions side by side, A/B testing across models (GPT-4 vs Claude vs Gemini)
• Strength: Simple YAML config, beautiful web UI for comparing results, supports 50+ providers
• Eval Types: Contains, regex match, JSON schema validation, LLM-as-judge, JavaScript assertions
• Sweet Spot: Teams iterating on prompts who want fast feedback on changes

2. DeepEval - The Comprehensive LLM Testing Framework

DeepEval is like pytest for LLMs. It gives you 14+ built-in metrics and integrates into your CI/CD pipeline.

• Best For: Production-grade LLM testing with rich metrics, CI/CD integration
• Strength: 14+ metrics (G-Eval, faithfulness, bias, toxicity), Python-native, Confident AI dashboard
• Eval Types: Hallucination, answer relevancy, contextual precision/recall, summarization, bias detection
• Sweet Spot: Python teams building production AI apps who want comprehensive test suites

3. RAGAS - The RAG Evaluation Specialist

RAGAS (Retrieval Augmented Generation Assessment) is purpose-built for evaluating RAG pipelines. If you are building anything with retrieval + generation, RAGAS is your go-to.

• Best For: Evaluating RAG systems end-to-end (retrieval quality + generation quality)
• Strength: RAG-specific metrics that no other framework offers, works with LangChain and LlamaIndex
• Key Metrics: Faithfulness, Answer Relevancy, Context Precision, Context Recall, Context Utilization
• Sweet Spot: Teams building knowledge bases, document QA, or any retrieval-based AI system

Faithfulness (Is the answer grounded in context?)

Checks whether the LLM output is actually supported by the retrieved context. If your RAG retrieves documents about "Indian tax law" but the LLM invents facts about "US tax law", faithfulness score drops.

Score: 0.0 to 1.0 | Higher is better | Critical for RAG systems

Answer Relevancy (Does the answer address the question?)

Measures how relevant the generated answer is to the original question. If someone asks "What is the GST rate for laptops?" and your system responds with a history of Indian taxation, relevancy is low even if the information is factually correct.

Score: 0.0 to 1.0 | Higher is better | Important for user satisfaction

Context Precision (Did retrieval fetch the right documents?)

Among all retrieved documents, how many were actually relevant? If your retriever pulls 10 documents but only 2 are useful, context precision is 0.2. This tells you your retrieval step needs improvement.

Score: 0.0 to 1.0 | Higher is better | Measures retrieval quality

Context Recall (Did retrieval find ALL relevant docs?)

Were all the documents needed to answer the question actually retrieved? If the answer requires info from 5 documents but only 3 were retrieved, context recall is 0.6. High recall means your retriever is thorough.

Score: 0.0 to 1.0 | Higher is better | Critical for completeness

Hallucination Score

Measures the percentage of generated content that is NOT supported by the provided context. A hallucination score of 0.3 means 30% of the answer was fabricated. This is the single most important metric for trust-critical applications (legal, medical, finance).

Score: 0.0 to 1.0 | LOWER is better | Most critical safety metric

Step 1: Build Your Golden Dataset

Start with 50-100 test cases that represent real user queries. For each case, include:

• The user question/input
• The expected answer (or acceptable answer criteria)
• For RAG: the expected source documents
• Edge cases: ambiguous questions, out-of-scope queries, adversarial inputs

Swiggy example: If your chatbot handles food orders, include test cases for "I want butter chicken", "Cancel my order", "Is the restaurant halal?", and "Tell me a joke" (out of scope).

Step 2: Choose Your Evaluators

• Exact Match: For factual questions with definitive answers (GST rates, dates, names)
• Contains/Regex: Check if output includes required keywords or patterns
• LLM-as-Judge: For subjective quality (tone, helpfulness, completeness)
• RAGAS Metrics: For RAG pipelines (faithfulness, relevancy, context quality)
• Custom Assertions: Business-specific rules (never recommend competitor, always include disclaimer)

Step 3: Integrate into CI/CD

Evaluation Pipeline Flow:

[Developer changes prompt] 
    --> [Git Push] 
    --> [CI triggers eval run]
    --> [Run 100 test cases]
    --> [Calculate metrics]
    --> [Compare with baseline]
    --> [Pass: metrics >= thresholds] --> Deploy
    --> [Fail: regression detected] --> Block + Alert

Step 4: Monitor in Production

• Sample 5-10% of production queries for automated evaluation
• Track metric trends over time (is faithfulness slowly degrading?)
• Set up alerts for sudden drops in any metric
• Collect user feedback (thumbs up/down) as ground truth labels

Pitfall 1: Testing Only Happy Paths

If all your test cases are straightforward questions with clear answers, you will miss the real failures. Real users ask ambiguous questions, make typos, ask in Hinglish, and try to break your system.

Fix: Include 30% edge cases - adversarial inputs, multilingual queries, out-of-scope questions, extremely long inputs.

Pitfall 2: Using Only One Metric

A high answer relevancy score does not mean your system is good. It could be highly relevant but completely hallucinated. Always use multiple metrics together.

Pitfall 3: Stale Test Data

Your golden dataset from 6 months ago may not represent what users actually ask today. Regularly refresh your test cases with real production queries.

Pitfall 4: Trusting LLM Judges Blindly

LLM-as-Judge has known biases: it prefers longer responses, favors its own style, and can miss subtle factual errors. Always cross-validate with human reviewers on a sample.

Pitfall 5: Not Versioning Prompts with Eval Results

If you cannot answer "which prompt version scored 0.92 on faithfulness last Thursday?" then you cannot reliably iterate. Always tie eval results to specific prompt versions.

Q: What is the difference between Promptfoo, DeepEval, and RAGAS?

Q: What is faithfulness in RAG evaluation and why does it matter?

Q: How would you set up a prompt evaluation pipeline for a production application?

Q: What is the difference between Context Precision and Context Recall in RAGAS?