AI systems are increasingly being designed to do more than generate responses to user requests. Applications can retrieve information from external sources, execute tools, interact with databases, call APIs, and complete multi-step tasks with limited human intervention. However, allowing an AI system to perform these operations does not automatically make its output reliable. An application may produce an incorrect answer, misunderstand a request, select an inappropriate tool, or complete a task without satisfying the user’s actual requirements.
One way to address these challenges is to build a self-evaluating AI system. Rather than accepting every generated response as final, the system evaluates its own output against predefined criteria before deciding whether to return the result, revise it, or request additional information. This evaluation can involve checking factual consistency, validating structured data, verifying tool execution, measuring relevance, and assessing whether the original task has been completed.
Building such a system requires more than asking a large language model (LLM) to review its previous response. A reliable implementation combines deterministic validation, model-based evaluation, explicit quality criteria, controlled retries, and mechanisms for preventing repeated failures. The evaluation process should also be observable so that we can determine why a response was accepted or rejected.
This article explains how to design and implement a self-evaluating AI system, using Java and an LLM-based application as a practical example. It explores the architecture, evaluation strategies, implementation, feedback loop, and testing considerations needed to build an application that can assess and improve its own outputs.
1. What is a Self-Evaluating AI System?
A self-evaluating AI system is an application that checks whether an AI-generated result meets specified requirements before accepting it as complete. Rather than treating generation as the final step, the application evaluates the result and uses the outcome to determine what should happen next.
For a customer support application, evaluation criteria might include whether the response addresses the user’s question, whether it contains all required information, whether it follows the expected response format, and whether its claims are supported by the information provided to the model.
These criteria do not all require an LLM. Java, Python or Typescript code can verify whether mandatory fields are present, whether a response is empty, and whether a value falls within an allowed range. LangChain4j can be used to invoke an LLM when the application needs to assess more complex properties, such as relevance or completeness.
The distinction between these responsibilities is fundamental: the component that generates a response should not be the only mechanism responsible for deciding whether that response is acceptable.
1.1 Understanding the Architecture
A typical implementation begins when an application receives a user’s request and prepares it for processing. LangChain4j then invokes the configured large language model (LLM) to generate an initial response. Once the response is available, the application evaluates it against the defined requirements using validation rules and an LLM-based evaluator. Based on the evaluation results, the application decides whether to accept the response, request a revision, or stop processing if the task cannot be completed successfully.
After the evaluation process, the application returns the accepted response together with its status or indicates that further attention is required. This design separates response generation, evaluation, and decision-making into distinct responsibilities, allowing developers to modify the underlying LLM, adjust evaluation criteria, or improve validation rules without having to redesign the entire application.
2. Define What Successful Output Means
Before implementing the evaluation loop, developers must establish what constitutes an acceptable result. An AI system cannot reliably evaluate its performance when its success criteria are vague or contradictory. Suppose an application generates technical explanations. A reasonable evaluation policy might require the response to answer the user’s question, remain consistent with the available evidence, include all requested details, and avoid unsupported claims. A different application that generates customer support responses may prioritize policy compliance, accurate account information, and appropriate handling of sensitive requests.
Evaluation criteria should therefore reflect the intended use of the application rather than relying on a generic measure of response quality. A useful starting point is to divide evaluation into two categories: deterministic checks and semantic checks. Deterministic checks evaluate properties that can be established through explicit rules. These include required fields, valid JSON, numeric ranges, allowed values, successful tool execution, and authorization requirements. They should generally be implemented in code because their outcomes are easier to reproduce and verify.
Semantic checks assess qualities that are more difficult to express through simple rules. These include whether the answer addresses the user’s intent, whether an explanation is logically consistent, and whether a summary preserves the meaning of the source material. An LLM can help evaluate these properties, although its judgment is not guaranteed to be correct.
For instance, a response could receive separate results for completeness, relevance, and factual support. However, developers should not assume that averaging these scores makes an answer acceptable. A response that violates a mandatory safety rule should not pass simply because it performs well on the other dimensions.
A practical policy separates mandatory requirements from quality preferences. Mandatory requirements must pass before the system can accept the result, while quality preferences help the application determine whether another attempt is worthwhile.
3. Implementing an Evaluation Data Model
A self-evaluating application needs a consistent way to represent the outcome of an evaluation. Returning free-form text from the evaluator makes it difficult for the controller to distinguish a genuine failure from an ambiguous explanation. A structured evaluation result provides a more reliable interface between components.
The following example uses Java, Quarkus, and LangChain4j to define a structured evaluation result. A Java record represents the evaluator’s output, while LangChain4j integrates the LLM that assesses the generated response. This approach allows the application to validate evaluation results and use them to determine whether to accept the response, request a revision, or stop processing.
3.1 Define the Evaluation Result
The evaluator needs to return a predictable result that Java code can inspect. Instead of asking the model to provide an unrestricted explanation of its judgment, the application will use a Java record to represent the evaluation outcome.
public enum EvaluationDecision {
ACCEPT,
RETRY,
REJECT,
HUMAN_REVIEW
}
Next, create src/main/java/org/acme/EvaluationResult.java:
public record EvaluationResult(
boolean passed,
double score,
List<String> issues,
String feedback,
EvaluationDecision decision
) {
public EvaluationResult {
if (score < 0.0 || score > 1.0
|| !Double.isFinite(score)) {
throw new IllegalArgumentException(
"Score must be between 0 and 1"
);
}
issues = issues == null ? List.of() : List.copyOf(issues);
feedback = feedback == null ? "" : feedback;
if (passed && !issues.isEmpty()) {
throw new IllegalArgumentException(
"A passing evaluation cannot contain unresolved issues"
);
}
if (decision == null) {
throw new IllegalArgumentException(
"Evaluation decision is required"
);
}
}
}
The evaluation result contains five fields. The passed field indicates whether the evaluator considers the response acceptable. The score represents its normalized quality assessment, while issues identifies specific problems. The feedback field gives the generator instructions for a possible revision, and decision communicates the recommended next step.
The compact constructor validates important invariants before an evaluation result can be used. For example, it prevents invalid scores and rejects a passing result that still contains unresolved issues.
This validation does not establish that the model’s judgment is correct. It ensures that the returned evaluation follows basic application rules. The controller will apply additional checks before accepting the response.
5. Implement the Response Generator
The response generator is responsible for producing an answer to the customer’s request. It should use the available information and any feedback from previous evaluation attempts.
@RegisterAiService
@ApplicationScoped
public interface SupportResponseGenerator {
@SystemMessage("""
You are a customer support assistant.
Answer the customer's request using only the
information supplied in the task.
Do not invent account details, transactions,
company policies, or actions that have not occurred.
Address every part of the request. If the supplied
information is insufficient, explain what is missing.
Follow the revision feedback when it is provided.
Return only the customer-facing response.
""")
@UserMessage("""
Customer support task:
{task}
""")
String generate(String task);
}
The @RegisterAiService annotation allows Quarkus to create and manage the implementation of the interface. The application can inject the service without manually constructing a chat model or writing the underlying request-handling code.
The generate() method accepts a single string containing the task, relevant information, and any evaluation feedback. Keeping the interface simple allows the controller to determine how each attempt is constructed.
The system message instructs the model not to invent facts or claim that actions have occurred when they have not. These instructions guide generation, but they do not enforce the rules by themselves. The application must still validate important requirements and verify external actions independently.
6. Implement the LLM-Based Evaluator
The evaluator is a separate AI service with a different objective. Rather than answering the customer, it assesses whether the generated response satisfies the application’s criteria.
@RegisterAiService
@ApplicationScoped
public interface SupportResponseEvaluator {
@SystemMessage("""
You are an independent evaluator of customer
support responses.
Evaluate the response against the task and the
mandatory criteria supplied by the application.
Check whether the response addresses the request,
covers every required part, and avoids unsupported
factual claims.
A lack of evidence does not automatically prove
that a claim is false. Identify unsupported claims
separately from claims contradicted by the evidence.
Return a structured EvaluationResult.
Set passed to true only when all mandatory criteria
are satisfied and issues is empty.
Use ACCEPT when the response passes.
Use RETRY when a revision could reasonably fix
the identified problems.
Use REJECT when the task must not proceed.
Use HUMAN_REVIEW when reliable evaluation requires
human judgment.
Give actionable feedback. Do not invent facts or
treat your own assessment as external verification.
""")
@UserMessage("""
Evaluate the following support response.
{input}
""")
EvaluationResult evaluate(String input);
}
The evaluator returns an EvaluationResult rather than free-form text. LangChain4j AI services support mapping model output to Java types, including POJOs, records, and enums. The evaluator considers the original task and the generated answer. Later, the controller will supply explicit criteria and any relevant evidence in the input string.
A separate evaluator can provide a different perspective on the generated response, but it is not automatically independent in the statistical or factual sense. If both services use the same model and the same incomplete information, they may share the same blind spots. That is why the implementation will also use deterministic Java validation.
6.1 Add Deterministic Validation
Before building the orchestration layer, implement a Java validator for requirements that do not need an LLM. In this example, the validator checks that the generated answer is not blank and does not exceed a configured length.
@ApplicationScoped
public class SupportResponseValidator {
private static final int MAX_LENGTH = 5000;
public List<String> validate(String response) {
List<String> issues = new ArrayList<>();
if (response == null || response.isBlank()) {
issues.add("The response must not be empty.");
return issues;
}
if (response.length() > MAX_LENGTH) {
issues.add(
"The response exceeds the maximum allowed length."
);
}
return List.copyOf(issues);
}
}
This validator establishes two conditions that can be checked without a model: the response must contain text, and its length must remain within the configured limit. A production support application would normally add more validation, including checks against approved support policies, required response fields, and authorization rules. Where the response contains account-specific claims, the application should verify them against an authoritative data source rather than asking the evaluator to guess whether they are true.
Deterministic validation should run regardless of the LLM evaluator’s score. This ensures that a favorable model judgment cannot override a mandatory application rule.
6.2 Implement the Self-Evaluation Controller
The controller coordinates the generator, evaluator, and deterministic validator. It constructs the evaluation input, records the result of each attempt, and decides whether to accept the response or continue processing.
The example allows up to three generation attempts. If the response fails evaluation and can reasonably be improved, the controller supplies the evaluator’s feedback to the generator. If the response passes, the controller returns it. If the system reaches its attempt limit without satisfying the requirements, it reports failure instead of silently accepting the last response.
@ApplicationScoped
public class SelfEvaluatingSupportService {
private static final int MAX_ATTEMPTS = 3;
@Inject
SupportResponseGenerator generator;
@Inject
SupportResponseEvaluator evaluator;
@Inject
SupportResponseValidator validator;
public SupportResult process(
String question,
String availableInformation) {
String feedback = "";
List<AttemptResult> history = new ArrayList<>();
for (int attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
String task = """
Customer question:
%s
Available information:
%s
Previous evaluation feedback:
%s
""".formatted(
question,
availableInformation,
feedback.isBlank()
? "No previous feedback."
: feedback
);
String response = generator.generate(task);
List<String> ruleIssues =
validator.validate(response);
EvaluationResult evaluation = evaluator.evaluate("""
Original customer question:
%s
Available information:
%s
Mandatory criteria:
1. Address the customer's actual request.
2. Cover every part of the request.
3. Do not present unsupported claims as verified facts.
4. Do not claim that an external action occurred
unless its result has been verified.
Generated response:
%s
Deterministic validation issues:
%s
""".formatted(
question,
availableInformation,
response == null ? "" : response,
ruleIssues
));
history.add(new AttemptResult(
attempt,
evaluation.score(),
evaluation.issues(),
ruleIssues
));
boolean accepted =
evaluation.passed()
&& evaluation.decision() == EvaluationDecision.ACCEPT
&& evaluation.issues().isEmpty()
&& ruleIssues.isEmpty();
if (accepted) {
return new SupportResult(
"ACCEPTED",
response,
attempt,
history
);
}
if (evaluation.decision() == EvaluationDecision.REJECT) {
return new SupportResult(
"REJECTED",
null,
attempt,
history
);
}
if (evaluation.decision() == EvaluationDecision.HUMAN_REVIEW) {
return new SupportResult(
"HUMAN_REVIEW",
null,
attempt,
history
);
}
feedback = """
Revise the previous response to address these issues.
Evaluation feedback:
%s
Evaluation issues:
%s
Deterministic validation issues:
%s
""".formatted(
evaluation.feedback(),
evaluation.issues(),
ruleIssues
);
}
return new SupportResult(
"FAILED",
null,
MAX_ATTEMPTS,
history
);
}
public record AttemptResult(
int attempt,
double score,
List<String> evaluationIssues,
List<String> validationIssues
) {}
public record SupportResult(
String status,
String response,
int attempts,
List<AttemptResult> history
) {}
}
The service combines three sources of information: the original request, the available information, and the evaluation results. It also supplies deterministic validation failures to the evaluator so that the model can identify them as part of its assessment.
The acceptance condition is intentionally stricter than a score threshold. The response must pass the evaluator, receive an ACCEPT decision, contain no unresolved evaluation issues, and satisfy the Java validator. A high score alone cannot establish success.
The controller records a summary of each attempt in history. This allows the application to expose the number of attempts and the problems identified during processing. In a production environment, the history should also be persisted to an observability or audit system where appropriate, with sensitive customer information handled according to the application’s data-protection requirements.
There is an important limitation to understand. The evaluator still receives its instructions and evidence through a prompt, so its judgment can be wrong or inconsistent. The controller enforces the checks it can verify directly, but the semantic assessment remains probabilistic. For business-critical operations, additional authoritative checks and human approval may be required.
6.3 Expose the Application Through a REST Endpoint
The application now has its generator, evaluator, validator, and controller. The final step is to expose the functionality through a REST endpoint.
@Path("/support")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public class SupportResource {
@Inject
SelfEvaluatingSupportService supportService;
@POST
@Path("/evaluate")
public Response evaluate(SupportRequest request) {
if (request == null
|| request.question() == null
|| request.question().isBlank()) {
return Response.status(Response.Status.BAD_REQUEST)
.entity(new ErrorResponse(
"A customer question is required."
))
.build();
}
String information =
request.availableInformation() == null
? ""
: request.availableInformation();
var result = supportService.process(
request.question(),
information
);
return Response.ok(result).build();
}
public record SupportRequest(
String question,
String availableInformation
) {}
public record ErrorResponse(String message) {}
}
The endpoint accepts a JSON request containing the customer’s question and any information the application is permitted to use when generating an answer. It delegates the task to SelfEvaluatingSupportService, which runs the evaluation loop and returns a structured result.
The endpoint returns HTTP 200 when the evaluation workflow completes, even if the result status is FAILED, REJECTED, or HUMAN_REVIEW. This distinguishes a completed evaluation workflow from a transport-level error. A production API could map these statuses to different HTTP codes according to its API contract.
The example checks for a missing question before invoking the LLM. Additional protections, such as request-size limits, authentication, rate limiting, and input validation, should be added when exposing the service to external users.
Run and Test the Application
Start the Quarkus application in development mode if it is not already running:
./mvnw quarkus:dev
Submit a customer support request using curl:
curl -X POST http://localhost:8080/support/evaluate \
-H "Content-Type: application/json" \
-d '{
"question": "How can I request a refund for a duplicate charge?",
"availableInformation": "Customers can request a billing review by contacting support with the transaction date and payment reference. Refund eligibility is determined after the billing team investigates the charge."
}'
The application passes the question and available information to the response generator. The evaluator then checks whether the generated answer addresses the request, explains the appropriate next step, and avoids claiming that a refund has already been approved.
A successful response might look like this:
{
"status": "ACCEPTED",
"response": "To request a review of a duplicate charge, contact the support team with the transaction date and payment reference. The billing team will investigate the charge and determine whether you are eligible for a refund.",
"attempts": 1,
"history": [
{
"attempt": 1,
"score": 0.94,
"evaluationIssues": [],
"validationIssues": []
}
]
}
Results will depend on the configured model, its response, and the evaluation instructions. If the evaluator identifies a missing requirement, it can return a RETRY decision and provide feedback. The controller will use that feedback during the next generation attempt. If the response still fails after three attempts, the service returns FAILED without presenting the last unaccepted response as a verified answer.
This behavior is the defining feature of the implementation: the application does not assume that the first response is correct, and it does not accept the final response merely because the retry budget has been exhausted.
7. Test the Evaluation Logic
Testing should cover more than whether the endpoint responds successfully. The most important question is whether the application makes the correct decision when the generated response passes or fails evaluation. For example, the following scenarios should be included in the test suite.
| Scenario | Expected Behaviour |
|---|---|
| Complete and acceptable response | Return ACCEPTED |
| Response needs additional information | Retry with specific feedback |
| Response contains unresolved mandatory issues | Do not accept |
| Evaluator rejects the task | Return REJECTED |
| Evaluator requests human judgment | Return HUMAN_REVIEW |
| Every attempt fails | Return FAILED |
| Customer question is empty | Return HTTP 400 |
The orchestration service should be tested with mocked generator and evaluator implementations. This makes it possible to simulate different evaluation outcomes without sending requests to a real LLM. For example, a test can configure the first generated response to fail evaluation and the second response to pass. The test should verify that the service makes two generation attempts, forwards the first evaluation’s feedback, and returns the second response.
Another test should simulate three consecutive failures and verify that the service terminates after the configured attempt limit. This protects against accidental infinite loops and unnecessary model calls.
The deterministic validator can be tested separately using ordinary JUnit tests. These tests should cover empty strings, responses exceeding the maximum length, and valid responses. Because the validator uses explicit rules, its expected results can be checked reliably without model-based judgments.
Semantic evaluation requires a different testing strategy. Developers should maintain a collection of representative support requests and human-reviewed acceptable and unacceptable responses. They can then compare the evaluator’s decisions with the expected judgments and measure false acceptance and false rejection rates.
A model that returns valid EvaluationResult objects is not necessarily a reliable evaluator. Structured output validates the response format; evaluation tests establish how well the model performs its intended role.
8. Improving the Design for Production
The implementation provides the foundation for a self-evaluating AI system, but additional controls are needed before using it for consequential business operations.
Verify claims against authoritative sources. The LLM evaluator can identify potentially unsupported statements, but it cannot independently establish the truth of a claim without reliable evidence. If the assistant discusses account balances, payment status, refunds, or order delivery, the application should retrieve the relevant information from the authoritative system and validate claims against that information.
Separate mandatory rules from quality scores. A response should not pass a mandatory business or security rule simply because it receives a high semantic score. Rules such as access permissions, prohibited actions, and required approvals must be enforced by Java code and the relevant business services.
Control retries and model failures. The example limits generation to three attempts, but production applications should also apply timeouts, bounded retries for transient provider errors, and appropriate failure handling. A model service outage should not cause an unlimited sequence of requests or result in an unverified response being accepted.
Improve evaluation observability. Record evaluation decisions, issue categories, attempt counts, latency, and model versions. These records help identify recurring failures and establish whether changes to prompts or evaluation criteria actually improve performance. Avoid logging sensitive customer information unnecessarily.
Use human review when required. Some tasks cannot be evaluated reliably by an LLM alone. Refund approval, account suspension, sensitive customer disputes, and other consequential decisions may require explicit business authorization or human review. An evaluator’s ACCEPT decision should never override those requirements.
Measure the value of each additional model call. Evaluation introduces additional latency and cost. Measure how frequently the first response passes, how often retries resolve issues, and how frequently the evaluator incorrectly accepts a response. These results can help determine which tasks need semantic evaluation and which can be handled through simpler validation.
9. Conclusion
Building a self-evaluating AI system involves more than adding a second LLM call after generating a response. The application must define clear acceptance criteria, represent evaluation results in a structured format, combine semantic judgment with deterministic validation, and control how the system responds when an output fails.
The example demonstrates how Quarkus AI services can separate response generation from evaluation, while a Java controller coordinates retries and determines whether a response can be accepted. It also illustrates why evaluation history, bounded retries, and explicit failure states are necessary for dependable operation.
The most important design principle is that an AI-generated response is a candidate result, not proof that a task has been completed correctly. A reliable system must verify the conditions that matter, use authoritative evidence where necessary, and refuse to treat unresolved failures as success.
