How to Evaluate LLMs?

Large Language Models (LLMs) are AI systems that can understand and generate human-like text. Modern LLMs can perform tasks such as text generation, question answering, translation, summarization, code generation and information extraction.

• LLMs can be evaluated when developing and testing LLM applications.
• Evaluation helps assess and compare fine-tuned language models.
• Evaluation helps identify incorrect information, hallucinations and irrelevant responses.

Evaluation Strategies

1. Correctness: Measures whether the generated response is correct according to a reliable reference answer or ground truth.
2. Relevance: Measures whether the response directly addresses the user's question or instruction.
3. Factuality: Measures whether the information in the response is factually accurate and consistent with established facts.
4. Faithfulness: Measures whether the response is supported by the provided context. It is especially important for RAG systems.
5. Instruction Following: Measures whether the model follows the instructions, constraints and required format specified in the prompt.
6. Reasoning: Evaluates the model's ability to solve problems, perform logical analysis and reach valid conclusions.
7. Safety: Checks whether the model avoids harmful, toxic, unsafe or otherwise inappropriate responses.
8. Bias and Fairness: Evaluates whether model outputs contain unwanted biases or produce systematically different results for different groups.
9. Robustness: Measures how consistently the model performs when inputs are changed slightly, contain noise or use different wording.

No single metric is sufficient for every LLM application. Evaluation should use a combination of metrics that match the task and the risks associated with it.

Evaluation Methods

1. Reference-Based Evaluation

• Reference-based metrics compare a generated response with one or more expected responses. Common metrics include: BLEU, ROUGE, METEOR and BERTScore.
• These metrics are useful when a reference answer is available, but lexical similarity does not always mean that an answer is semantically correct.

BLEU Score: BLEU is commonly used to compare generated text with reference text, particularly in machine translation.

from nltk.translate.bleu_score import sentence_bleu, SmoothingFunction

reference = [["Alexander", "Graham", "Bell", "invented", "the", "telephone"]]

candidate = [
    "Alexander", "Graham", "Bell", "invented", "the", "telephone"
]

score = sentence_bleu(
    reference,
    candidate,
    smoothing_function=SmoothingFunction().method1
)

print(f"BLEU Score: {score:.2f}")

A higher BLEU score generally indicates greater overlap between the generated text and reference text.

ROUGE Score: ROUGE is widely used for evaluating text summarization by measuring overlap between generated and reference text.

First install the required package:

!pip install rouge-score -q

Then calculate the ROUGE scores:

from rouge_score import rouge_scorer

reference = "Alexander Graham Bell invented the telephone."
candidate = "Alexander Graham Bell was the inventor of the telephone."

scorer = rouge_scorer.RougeScorer(
    ["rouge1", "rouge2", "rougeL"],
    use_stemmer=True
)

scores = scorer.score(reference, candidate)

for metric, score in scores.items():
    print(f"{metric}: {score.fmeasure:.2f}")

ROUGE-1 measures unigram overlap, ROUGE-2 measures bigram overlap and ROUGE-L uses the longest common subsequence.

2. Human Evaluation

• Human evaluation involves people reviewing LLM outputs based on predefined criteria. Evaluators may rate outputs based on: Accuracy, Relevance, Clarity, Helpfulness, Fluency and Safety.
• It is useful for subjective qualities that are difficult to measure automatically. However, it can be slower and more expensive than automated evaluation.

3. LLM-as-a-Judge

In LLM-as-a-Judge evaluation, another LLM evaluates the output of the model being tested. The evaluator can score responses based on criteria such as correctness, relevance, helpfulness and style.

For example:

Question: What is the capital of France?
Answer: Paris is the capital of France.
Evaluation Criteria:
- Is the answer correct?
- Is it relevant to the question?
- Is it clearly written?
Score: 5/5

LLM-as-a-Judge can scale evaluation to large datasets, but the evaluator itself can have biases and may produce inconsistent judgments. Therefore, its results should be validated using human evaluation or other evaluation methods.

4. Combining Multiple Evaluation Methods

• Using multiple evaluation methods provides a more complete view of model performance.
• For example, a system can combine reference-based scores with relevancy and faithfulness checks.

bleu_score = 0.85
rouge_score = 0.78
relevancy_score = 0.90
faithfulness_score = 0.88

overall_score = (
    bleu_score +
    rouge_score +
    relevancy_score +
    faithfulness_score
) / 4

print(f"Overall Evaluation Score: {overall_score:.2f}")

The example calculates a simple average. In a real evaluation system, different metrics may have different importance, so they should not automatically be given equal weights.

Common Evaluation Metrics

Choosing the Right Evaluation Metrics

• There is no single metric that works for every LLM application.
• For a question-answering system, correctness and answer relevancy are important. For a RAG system, faithfulness and retrieval metrics are also important. For content generation, reference-based metrics and human evaluation may be useful.
• The evaluation criteria should therefore match the task, expected output and risks of the application.

Evaluating RAG Systems

Evaluating Contextual Relevancy

The following example uses TF-IDF and cosine similarity to compare a question with retrieved documents. It is a simple, free demonstration and does not require an API key.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

query = "What is the capital of France?"

retrieved_context = [
    "Paris is the capital and largest city of France.",
    "France is a country in Western Europe.",
    "The Eiffel Tower is a famous landmark in Paris."
]

vectorizer = TfidfVectorizer()
vectors = vectorizer.fit_transform([query] + retrieved_context)

similarities = cosine_similarity(
    vectors[0:1],
    vectors[1:]
).flatten()

for i, score in enumerate(similarities):
    print(f"Context {i + 1} Relevancy Score: {score:.2f}")

A higher similarity score indicates that the retrieved context has more lexical overlap with the query.

Evaluating Faithfulness

• Faithfulness can be demonstrated by checking whether the claims in an answer are supported by the retrieved context.
• The following simple example uses word overlap and does not require a paid API:

import re

def split_into_claims(text):
    return [
        claim.strip()
        for claim in re.split(r"[.!?]", text)
        if claim.strip()
    ]

def faithfulness_score(answer, context):
    claims = split_into_claims(answer)
    context = context.lower()

    supported = 0

    for claim in claims:
        words = set(re.findall(r"\b\w+\b", claim.lower()))
        context_words = set(re.findall(r"\b\w+\b", context))

        if words and len(words.intersection(context_words)) / len(words) >= 0.5:
            supported += 1

    return supported / len(claims) if claims else 0


context = """
Alexander Graham Bell is credited with inventing the telephone.
He was born in Scotland and later moved to the United States.
"""

answer = "Alexander Graham Bell invented the telephone."

score = faithfulness_score(answer, context)

print(f"Faithfulness Score: {score:.2f}")

This is a basic demonstration of the idea behind faithfulness evaluation. Production systems generally use more advanced semantic or LLM-based evaluation methods because word overlap alone cannot fully determine whether a claim is supported.

Evaluating Answer Relevancy

Answer relevancy can also be demonstrated by comparing the question and generated answer using TF-IDF and cosine similarity.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

question = "Who invented the telephone?"

answer = "Alexander Graham Bell invented the telephone."

vectorizer = TfidfVectorizer()
vectors = vectorizer.fit_transform([question, answer])

score = cosine_similarity(
    vectors[0:1],
    vectors[1:]
)[0][0]

print(f"Answer Relevancy Score: {score:.2f}")

A higher score indicates greater textual similarity between the question and answer. This is a simple proxy for relevancy and should not be treated as a complete semantic evaluation.

Evaluating Fine-Tuned LLMs

• Fine-tuned LLMs should be evaluated according to the task for which they were trained.

For example:

A model fine-tuned for sentiment classification can be evaluated using accuracy, precision, recall and F1-score. A model fine-tuned for text generation can be evaluated using reference-based metrics, factuality checks and human evaluation.

• Important evaluation areas can include: Task-specific correctness, Answer relevancy, Factual accuracy, Bias and fairness, Safety and Human preference.
• There is no fixed set of metrics that applies to every fine-tuned model.

Example: Bias and Fairness Check: A simple way to demonstrate fairness testing is to compare responses generated for equivalent inputs.

answers = {
    "Group A": "The candidate has strong leadership and communication skills.",
    "Group B": "The candidate has strong leadership and communication skills."
}

for group, answer in answers.items():
    print(f"{group}: {answer}")

if len(set(answers.values())) == 1:
    print("\nNo difference found between the example responses.")
else:
    print("\nThe responses differ and require further bias analysis.")

This is only a basic demonstration. Real fairness evaluation requires multiple prompts, larger datasets and appropriate statistical analysis.

You can download the complete source code from here.

Challenges

1. Over-Reliance on Quantitative Metrics : Metrics like BLEU or ROUGE often miss deeper issues such as hallucination, tone or creativity.
2. Ignoring Task-Specific Metrics : General metrics may not suffice for specialized tasks like summarization or code generation; custom evaluations are crucial.
3. Inconsistent Human Evaluation : Subjective assessments require clear guidelines and trained evaluators to ensure consistency.
4. Lack of Real-World Testing : Models that perform well on benchmarks may fail in diverse, real-world scenarios.
5. Neglecting Ethical Considerations : Failing to assess bias and toxicity can lead to harmful outputs, reducing trust and usability.