Large language models are good at understanding natural language, but applications usually need information in a predictable form. A customer support application, for example, may receive a message describing a duplicate payment and a refund request, but the application needs specific fields such as the issue type, requested action, email address, and urgency.
Simply telling a model to return JSON does not guarantee that the result will always be correct or usable. The model can omit fields, use unexpected values, misunderstand the input, or return valid JSON that does not satisfy the application’s requirements. Reliable structured output therefore requires more than a formatting instruction.
A better approach is to define a clear data contract, provide meaningful descriptions for the fields, use the model’s structured-output capabilities where available, and validate the result before using it in application logic. When necessary, guardrails can also give the model another opportunity to correct an invalid response.
In this article, we will look at how to get reliable structured data out of an LLM using Quarkus, LangChain4j, and Ollama.
1. From Natural Language to Structured Data
Natural-language output works well when another person is expected to read the response, but software needs predictable data. A model might describe a duplicate payment as “the customer was charged twice” or “the customer was billed more than once.” Both express the same meaning, but string matching or regular expressions can fail when the wording changes.
Structured output avoids this problem by defining named fields with clear meanings. Instead of asking, “What did the model say?” the application can ask, “What value did the model assign to issueType?” This creates a more predictable interface between the LLM and application code.
Consider the following customer request: “I have been charged twice for my subscription. The first charge was yesterday and the second one appeared this morning. Please refund the duplicate payment. My account email is thomas@jcg.com.”
The application could represent the extracted information as:
{
"customerEmail": "thomas@jcg.com",
"issueType": "DUPLICATE_CHARGE",
"requestedAction": "REFUND",
"urgency": "HIGH"
}
The goal is not simply to make the response look like JSON. The goal is to turn an unstructured message into data that the application can safely process.
Why Asking the Model for JSON Is Not Enough
A common first attempt at structured output is to add an instruction such as “Return the answer as JSON.” Although this can improve the response format, it does not establish a reliable contract by itself. A model could return malformed JSON, add explanatory text before or after the JSON, omit a field, use an unexpected value, or change the name of a property.
Even if the response is valid JSON, the values may not be acceptable to the application. For example, an application may expect HIGH, MEDIUM, or LOW for urgency while the model returns "urgent". The application therefore needs a schema that defines what the response is supposed to contain. It also needs a mechanism for mapping the response into a typed representation and validating that representation before the data reaches business logic.
In a Java application, the Java object can serve as the central representation of that contract. Rather than designing an unrelated JSON format and then writing additional mapping code, the application can define the expected fields in a Java type and allow LangChain4j to use that type as the target structured response.
2. Define the Java Data Structure First
A Java class is a good starting point because it gives the application a clear representation of the information it expects. However, the class alone does not completely explain what each field means or how the model should interpret the customer’s message.
Creating the Project
The application can be created with Quarkus using the REST Jackson and LangChain4j Ollama extensions.
mvn io.quarkus.platform:quarkus-maven-plugin:create \ -DprojectGroupId=org.jcg \ -DprojectArtifactId=structured-llm \ -Dextensions="rest-jackson,langchain4j-ollama"
The REST Jackson extension provides JSON serialization for the REST API, while the LangChain4j Ollama extension provides the integration with a local Ollama model. Ollama can run models locally, which makes it possible to build and test the application without sending the request to a cloud-hosted model.
Configure Ollama
The model can be configured in application.properties:
quarkus.langchain4j.ollama.chat-model.model-id=qwen3:8b quarkus.langchain4j.ollama.timeout=60s
The model-id identifies the model that Ollama should use, while the timeout gives the local model enough time to respond. The current Quarkus LangChain4j Ollama configuration also supports a format property that can request JSON responses from Ollama. If the application only needs JSON text, the simplest option is to configure the Ollama model to request JSON output:
quarkus.langchain4j.ollama.chat-model.format=json
This causes the Ollama request to specify JSON as the desired response format. It is useful when an AI service returns a String and the application wants that string to contain JSON. However, JSON mode should not be confused with complete schema validation. Asking the model for JSON does not by itself define which fields must exist, what their values should mean, or whether the returned information is semantically correct.
For example, the model could return valid JSON with an unexpected property or a value that does not match the application’s expected vocabulary. The response may therefore be syntactically valid while still being unsuitable for the application. For a typed Java method, the Java return type and its descriptions provide a more meaningful contract. JSON mode can be viewed as an additional formatting constraint rather than a replacement for the schema.
Start With the Java Object
A good way to design structured LLM output in a Java application is to start with the object that the application actually needs. For this example, we can define a SupportRequest record:
@Description("Structured information extracted from a customer support request.")
public record SupportRequest(
@Description("The customer's email address. Use null when the request does not contain an email address.")
String customerEmail,
@Description("The type of customer issue. Use DUPLICATE_CHARGE when the customer reports being charged more than once for the same purchase or subscription.")
String issueType,
@Description("The action requested by the customer. Use REFUND when the customer explicitly asks for money to be returned.")
String requestedAction,
@Description("The urgency of the request. Use HIGH when the issue requires prompt attention, MEDIUM for a normal request, and LOW for a minor request.")
String urgency)
{ }
This class defines the basic structure that the application expects from the model. The fields are intentionally simple. The model needs to extract the customer’s email address, identify the type of issue, determine what action the customer is requesting, and estimate the urgency based on the content of the message.
The Java object is more useful than an informal description because it gives the application an explicit representation of the expected data. It also creates a natural place to introduce stronger types later. For example, issueType could eventually become an enum instead of a String.
The @Description annotations add meaning directly to the Java structure. A field such as issueType does not tell a model what values it should contain or how those values should be interpreted. The description provides that additional context. LangChain4j can use these descriptions when generating the structured schema presented to the model.
This means that the meaning of the fields becomes part of the structured contract rather than existing only in a long prompt. For a stronger application contract, these string fields can also be replaced with Java enums. That limits the possible values in the application and makes validation clearer. For example:
public enum IssueType {
DUPLICATE_CHARGE,
PAYMENT_FAILED,
ACCOUNT_ACCESS,
OTHER
}
The same approach can be applied to RequestedAction and Urgency. Using enums is preferable when the application has a fixed set of accepted values because the model’s output can then be mapped into a constrained Java type instead of an arbitrary string.
public enum RequestedAction {
REFUND,
INVESTIGATE,
RESTORE_ACCESS,
OTHER
}
public enum Urgency {
LOW,
MEDIUM,
HIGH
}
The main principle is to make the Java structure meaningful before trying to make the prompt more detailed. A well-defined schema gives the model a clearer target and gives the application a stronger contract.
Define the LangChain4j AI Service
Once the Java object and Ollama configuration are in place, the next step is to define the AI service. Quarkus LangChain4j allows the LLM interaction to be declared through an interface. The application does not need to implement this interface manually because Quarkus generates the required AI service implementation.
@RegisterAiService
public interface SupportRequestExtractor {
@SystemMessage("""
You are a customer support request extraction system.
Extract structured information from the user's support request
and return the result as JSON only.
Do not include explanations, comments, Markdown, or any text
outside the JSON object.
""")
@UserMessage("""
Extract structured information from the following customer
support request.
Identify the customer's email address, the type of issue,
the action requested by the customer, and the urgency.
Use DUPLICATE_CHARGE when the customer reports being charged
more
than once for the same purchase or subscription.
Use REFUND when the customer explicitly requests money to
be returned.
Do not invent information that is not present in the request.
Customer request: {{ text }}
""")
SupportRequest extract(String text);
}
The @SystemMessage provides general instructions that apply to the extraction operation, including the requirement to return JSON only. The @UserMessage describes the information that should be extracted from the particular customer request.
The Java return type is also significant. Instead of returning arbitrary text and asking another part of the application to interpret it, the AI service declares that the result should become a SupportRequest.
The prompt and schema have different responsibilities. The schema defines the expected structure and field meanings, while the prompt provides instructions for interpreting the customer’s message. Keeping those responsibilities separate makes the design easier to understand and maintain.
Expose the Extractor Through REST
The AI service can be exposed through a REST endpoint:
@Path("/support")
public class SupportResource {
@Inject
SupportRequestExtractor extractor;
@POST
@Path("/extract")
@Consumes("text/plain")
@Produces("application/json")
public SupportRequest extract(String text) {
return extractor.extract(text);
}
}
The endpoint accepts the customer’s message as plain text and passes it to the AI service. The returned SupportRequest is then serialized by Quarkus as JSON.
Test the Application
Start the application in development mode:
mvn quarkus:dev
Then send the customer message:
curl -X POST \ -H "Content-Type: text/plain" \ --data "I have been charged twice for my subscription. The first charge was yesterday and the second one appeared this morning. Please refund the duplicate payment. My account email is thomas@jcg.com." \ http://localhost:8080/support/extract
A possible response is:
{
"customerEmail": "thomas@jcg.com",
"issueType": "DUPLICATE_CHARGE",
"requestedAction": "REFUND",
"urgency": "HIGH"
}
The exact result can depend on the local model and prompt, but the application now has a structured Java object rather than a paragraph that it needs to interpret.
3. Add Tools When the Model Needs Application Data
Structured extraction may require information that is not available in the user’s message. For example, an application may need to retrieve a customer’s account details before determining how a support request should be processed.
LangChain4j supports this through tools, which allow the model to invoke application methods when additional information or an application operation is required. Tools can be registered with the tools attribute of @RegisterAiService, while @ToolBox can be used when tools are specific to an individual AI service method.
For example, a support application could expose a customer lookup service:
@ApplicationScoped
public class CustomerTools {
@Tool("Find customer account information using the customer's email address.")
public String findCustomer(String email) {
// In a real application, this would query a database or service.
return "Customer account found";
}
}
The AI service can then make the tool available:
@RegisterAiService(tools = CustomerTools.class)
public interface SupportRequestExtractor {
// AI service methods
}
The model can use the tool when the task requires information from the application. The tool itself remains ordinary Java code, so database access, authorization, and other deterministic operations remain under application control. Tools should not be added simply because they are available. If all required information is already present in the user’s message, an extraction operation may not need tool calling.
4. Parsing and Validation Are Different
Parsing and validation are separate steps: parsing converts the model’s response into structured data such as a Java object, while validation checks whether that data has the expected types, follows the schema, and satisfies the application’s business, security, and safety rules. A response can therefore be successfully parsed into a SupportRequest and still contain an invalid or incorrect value.
For example, if the model produces a SupportRequest where issueType is PAYMENT_FAILED even though the customer clearly described a duplicate charge, the object may be structurally valid but semantically wrong. This is why structured output should not be treated as proof that the model’s answer is correct. The application should validate the result before allowing it to influence business operations.
In simple terms, parsing asks whether the response can be converted into the expected Java object, while validation asks whether that object is acceptable for the application. Suppose the model produces an object equivalent to:
{
"customerEmail": "thomas@jcg.com",
"issueType": "DUPLICATE_CHARGE",
"requestedAction": "REFUND",
"urgency": "EXTREMELY_HIGH"
}
The JSON may be structurally valid, but EXTREMELY_HIGH is not one of the values the application understands. Likewise, the model could return:
{
"customerEmail": "not-an-email",
"issueType": "DUPLICATE_CHARGE",
"requestedAction": "REFUND",
"urgency": "HIGH"
}
The structure is present, but the email address is invalid. This is why applications should validate structured LLM output before allowing it to influence business logic.
The application can perform validation after LangChain4j has produced the object. For example, the application can verify that the email address exists and has a reasonable format, that the issue type belongs to the supported set, and that the requested action is recognized. If enums are used in the Java model, invalid categorical values become much harder to accept accidentally.
5. Handle Missing Information Explicitly
Models should not be encouraged to invent values simply because the schema contains a field for them. If the customer does not provide an email address, the model should return null or another explicitly supported representation rather than creating an address.
For example, the description on customerEmail can state that null should be used when the email is not present. The application can then decide whether the request can continue or needs additional information. This distinction is important because missing information is different from a known negative value. A missing email does not mean that the customer’s email is invalid, and the absence of a refund request does not necessarily mean that the customer does not want a refund.
The schema should therefore make the difference clear, and the application should define how each case is handled. For example, the Java class could use a nullable String:
private String customerEmail;
The application can then detect the missing value:
if (request.getCustomerEmail() == null) {
// Request additional customer information.
}
This is much safer than forcing the model to produce a value for every field. A reliable structured-output contract should always provide a legitimate representation for information that is unavailable.
6. Add an Output Guardrail
Even with a schema and clear instructions, it is useful to have a final validation layer around the model response. LangChain4j provides output guardrails for this purpose. An OutputGuardrail can inspect the model’s response and either accept it, reject it, retry the request, or send an additional correction prompt to the model.
A guardrail can check that the raw model response is valid JSON and contains the fields required by the extraction contract. A simple guardrail can be implemented as follows:
@ApplicationScoped
public class SupportRequestOutputGuardrail implements OutputGuardrail {
private final ObjectMapper objectMapper;
public SupportRequestOutputGuardrail(ObjectMapper objectMapper) {
this.objectMapper = objectMapper;
}
@Override public OutputGuardrailResult validate(AiMessage responseFromLLM) {
try { JsonNode json = objectMapper.readTree(responseFromLLM.text());
if (!json.has("customerEmail")
|| !json.has("issueType")
|| !json.has("requestedAction")
|| !json.has("urgency")) {
return reprompt( "The response is missing one or more required fields.",
"""
Return the support request again as JSON only.
Include customerEmail, issueType, requestedAction, and urgency.
Do not include any additional text.
""" );
} return success();
} catch (Exception e) {
return reprompt( "The response is not valid JSON.", e,
"""
Return the support request again as a valid JSON object only.
Do not include Markdown, explanations, or text outside the JSON.
""" );
}
}
}
The guardrail first attempts to parse the model’s response as JSON. If parsing fails, it asks the model to correct the response. It also checks that the expected fields are present and requests another response when they are missing. The guardrail can then be attached to the AI service:
@RegisterAiService
public interface SupportRequestExtractor {
@SystemMessage("""
You are a customer support request extraction system.
Extract structured information from the user's support request
and return the result as JSON only.
Do not include explanations, comments, Markdown, or any text
outside the JSON object.
""")
@UserMessage("""
Extract structured information from the following customer
support request.
Identify the customer's email address, the type of issue,
the action requested by the customer, and the urgency.
Use DUPLICATE_CHARGE when the customer reports being charged
more
than once for the same purchase or subscription.
Use REFUND when the customer explicitly requests money to
be returned.
Do not invent information that is not present in the text.
Customer request: {{ text }}
""")
@OutputGuardrails( value = SupportRequestOutputGuardrail.class, maxRetries = 2 )
SupportRequest extract(String text);
}
The @OutputGuardrails annotation associates the guardrail with the extraction method and allows the application to define how many times the model should retry when validation fails. The guardrail can either retry the original request or reprompt the model with additional instructions that explain how the response should be corrected.
A custom guardrail is useful when the application requires checks that go beyond basic JSON parsing and validation, such as enforcing fields, allowed values, or other rules specific to the data contract.
7. Conclusion
Getting reliable structured data from an LLM requires more than asking for JSON. A strong implementation starts with a clear Java data structure, adds semantic descriptions and constraints, uses appropriate model features, and validates the response before it reaches business logic.
Quarkus, LangChain4j, and Ollama provide the components needed to build this pattern with a local model. JSON mode can help with response formatting, structured Java types provide a stronger contract, tools allow the model to access application capabilities when needed, and output guardrails provide a way to detect and recover from invalid responses. The key principle is to let the LLM handle the interpretation of natural language while keeping structure, validation, and business decisions under the control of the application.
8. Download the Source Code
You can download the full source code of this example here: how to get reliable structured data out of an llm
