Large Language Models (LLMs) are commonly accessed through cloud APIs provided by services such as OpenAI, Anthropic, or Google Gemini. However, there are many situations where developers may want to execute an LLM locally instead. Running an LLM locally can be useful for development, experimentation, privacy-sensitive applications, offline environments, and situations where calling an external AI service for every request is unnecessary. LM Studio makes running local language models relatively simple by allowing developers to download and execute compatible open-source models and expose them through an HTTP API. More importantly for Java developers, LM Studio provides an OpenAI-compatible API. Spring AI already supports the OpenAI API format, allowing a Spring Boot application to communicate with LM Studio without creating a custom HTTP client.
1. Understanding Local LLMs and LM Studio
In this article, we will build a simple Spring Boot application that integrates with a local Large Language Model (LLM) running in LM Studio using Spring AI. The overall flow is straightforward: a client sends a question to the Spring Boot REST API, the controller passes the prompt to Spring AI, and Spring AI sends an OpenAI-compatible request to LM Studio. LM Studio then forwards the prompt to the locally loaded LLM, which generates a response and returns it through LM Studio and Spring AI back to the Spring Boot application, where the final answer is sent to the client.
1.1 What is a Local LLM?
A local LLM is a language model that runs directly on your computer or on infrastructure you control, rather than relying on an external cloud-based AI service. This means prompts, such as “Explain dependency injection in Java,” can be processed locally without being sent to a third-party API. Popular model families that can be run locally include Llama, Mistral, Qwen, Gemma, Phi, and DeepSeek. The right model depends on factors such as available system memory, CPU or GPU capabilities, desired response quality, model size, and expected inference speed.
1.1.1 Why Use Local LLMs?
Running LLMs locally provides several advantages, particularly for privacy, cost, and development flexibility. Since prompts can be processed directly on the local machine, sensitive information does not need to be sent to an external AI provider. Once a model has been downloaded, it can also perform inference without an active Internet connection, making it useful for offline environments. Local models do not have per-request API charges because they use your own computing resources, although hardware and electricity costs still apply. They are also convenient for developing and testing AI-powered applications without repeatedly calling external services. In addition, developers have greater control over which model is used, how it is configured, and where the inference process takes place.
1.2 What is LM Studio?
LM Studio is a desktop application that allows developers to download, load, and run LLMs locally. One of its most useful features is the built-in local API server, which is commonly available at http://localhost:1234. LM Studio exposes OpenAI-compatible endpoints such as GET /v1/models, POST /v1/chat/completions, and POST /v1/embeddings. For this example, the most important endpoint is POST http://localhost:1234/v1/chat/completions, which is used to send prompts to the locally running model. Since Spring AI already supports the OpenAI API format, we do not need to create a custom LM Studio client. Instead, we can simply configure Spring AI’s OpenAI integration to use LM Studio’s local server URL.
2. Integrating Spring AI with LM Studio
2.1 Maven Dependencies
To get started, we need to configure the required dependencies in the project’s pom.xml file. The following Maven configuration includes Spring Web for creating the REST API and Spring AI’s OpenAI-compatible integration for communicating with the local LLM running in LM Studio.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://maven.apache.org/POM/4.0.0
https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>stable__jar__version</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>4.1.0</version>
<relativePath/>
</parent>
<groupId>com.example</groupId>
<artifactId>local-llm-demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>local-llm-demo</name>
<description>Spring AI integration with LM Studio</description>
<properties>
<java.version>21</java.version>
<spring-ai.version>2.0.0</spring-ai.version>
</properties>
<dependencies>
<!-- Spring MVC / REST -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- Spring AI OpenAI-compatible client -->
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-bom</artifactId>
<version>${spring-ai.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
The Maven configuration uses spring-boot-starter-parent as the project parent and Java 21 as the Java version. The spring-boot-starter-web dependency provides Spring MVC and the components required to expose REST endpoints, while spring-ai-starter-model-openai provides Spring AI’s OpenAI-compatible chat model integration, which can communicate with LM Studio because LM Studio exposes an OpenAI-compatible API. The spring-ai-bom is imported through dependencyManagement to keep Spring AI dependency versions consistent, and the spring-boot-maven-plugin allows the application to be packaged and run as a Spring Boot application.
2.2 Configure application.properties
Next, we need to configure the Spring Boot application so that Spring AI communicates with the local LLM through LM Studio instead of an external OpenAI server. Add the following properties to the src/main/resources/application.properties file, and make sure to replace the model name with the actual model ID loaded in your LM Studio instance.
spring.application.name=local-llm-demo # LM Studio local server spring.ai.openai.base-url=http://localhost:1234 # LM Studio does not require an API key by default. # Spring AI expects a value, so we provide a placeholder. spring.ai.openai.api-key=lm-studio # Replace this with the model ID shown by LM Studio spring.ai.openai.chat.options.model=qwen2.5-7b-instruct spring.ai.openai.chat.options.temperature=0.7 server.port=8080
This configuration sets the application name to local-llm-demo and points Spring AI’s OpenAI client to the LM Studio server running at http://localhost:1234. The spring.ai.openai.api-key property uses lm-studio as a placeholder because LM Studio does not require an API key by default, while Spring AI expects this property to have a value. The spring.ai.openai.chat.options.model property specifies the local model that will process the prompts, so qwen2.5-7b-instruct should be replaced with the model ID available in your LM Studio setup. The temperature value of 0.7 controls the variability of generated responses, while server.port=8080 configures the Spring Boot application to run on port 8080.
2.3 Create the Spring Boot Application
Now, create the main Spring Boot application class that serves as the entry point for the project. The LocalLlmApplication class starts the Spring application, initializes the application context, and enables Spring Boot’s auto-configuration so that the required Spring Web and Spring AI components can be configured automatically.
// LocalLlmApplication.java
package com.example.localllm;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class LocalLlmApplication {
public static void main(String[] args) {
SpringApplication.run(LocalLlmApplication.class, args);
}
}
The @SpringBootApplication annotation marks this class as the main configuration class and combines component scanning, auto-configuration, and Spring Boot configuration. The main() method calls SpringApplication.run() to start the application and initialize the Spring context. During startup, Spring Boot discovers components such as controllers and services and configures the Spring AI integration based on the settings defined in application.properties.
2.4 Create the Response Model
Next, create a simple response model that represents the data returned by our REST API. Instead of returning the LLM-generated answer as plain text, we will wrap it in a ChatResponse record so that Spring Boot automatically returns the response in a clean JSON format.
// ChatResponse.java
package com.example.localllm;
public record ChatResponse(String answer) {
}
The ChatResponse record contains a single answer field that stores the text generated by the local LLM. Java records provide a concise way to create immutable data carrier classes without manually writing constructors, getters, or other boilerplate code. When a ChatResponse object is returned from the REST controller, Spring Boot automatically serializes it into JSON, producing a response such as {"answer":"Spring Boot simplifies the development of Spring applications."}.
2.5 Create the LLM Service
Next, create a service class that handles the interaction between the Spring Boot application and the local LLM running in LM Studio. The LocalLlmService uses Spring AI’s ChatClient to build a prompt, send the user’s message to the configured model, and return the generated response.
// LocalLlmService.java
package com.example.localllm;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.stereotype.Service;
@Service
public class LocalLlmService {
private final ChatClient chatClient;
public LocalLlmService(ChatClient.Builder chatClientBuilder) {
this.chatClient = chatClientBuilder.build();
}
public String ask(String message) {
return chatClient
.prompt()
.system("""
You are a helpful Java programming assistant.
Explain concepts clearly and concisely.
Use simple examples when appropriate.
""")
.user(message)
.call()
.content();
}
}
The @Service annotation registers LocalLlmService as a Spring-managed component, while ChatClient.Builder is injected through the constructor and used to create a ChatClient. Inside the ask() method, prompt() starts building the request, system() provides instructions that define how the model should respond, and user(message) adds the actual question received from the user. The call() method sends the request through Spring AI to the OpenAI-compatible endpoint provided by LM Studio, where the local LLM generates the response, and content() extracts and returns the generated text.
2.6 Create the REST Controller
Finally, create a REST controller that exposes an HTTP endpoint for interacting with the local LLM. The ChatController receives the user’s message as a request parameter, passes it to LocalLlmService for processing, and returns the model-generated answer as a ChatResponse object.
// ChatController.java
package com.example.localllm;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
@RestController
@RequestMapping("/api")
public class ChatController {
private final LocalLlmService localLlmService;
public ChatController(LocalLlmService localLlmService) {
this.localLlmService = localLlmService;
}
@GetMapping("/chat")
public ChatResponse chat(@RequestParam String message) {
String answer = localLlmService.ask(message);
return new ChatResponse(answer);
}
}
The @RestController annotation marks ChatController as a REST API controller, while @RequestMapping("/api") defines the base path for its endpoints. The @GetMapping("/chat") annotation exposes the GET /api/chat endpoint, and @RequestParam String message reads the user’s prompt from the message query parameter. The controller passes this message to the LocalLlmService.ask() method, which sends it to the local LLM through Spring AI and LM Studio. The generated answer is then wrapped in a ChatResponse object and automatically serialized by Spring Boot into a JSON response.
2.7 Run and Test the Application
Before running the Spring Boot application, make sure LM Studio is running, the required LLM is loaded, and the local API server is started on http://localhost:1234. Once LM Studio is ready, start the Spring Boot application using the following Maven command.
mvn spring-boot:run
After the application starts successfully, it will be available on port 8080. We can now test the REST endpoint by sending a message through the message query parameter using the following command.
curl "http://localhost:8080/api/chat?message=Explain%20dependency%20injection%20in%20Java"
The request is received by ChatController and passed to LocalLlmService, which uses Spring AI to send the prompt to the local model running in LM Studio. The generated answer is then returned by the application as a JSON response. Since the response is generated by an LLM, the exact content may vary depending on the model and its configuration.
{
"answer": "Dependency injection is a design pattern in which a class receives its dependencies from an external source instead of creating them itself. In Spring, dependencies are commonly provided through constructor injection."
}
This output confirms that the complete integration is working successfully: the client sends a request to the Spring Boot REST API, Spring AI communicates with LM Studio through its OpenAI-compatible API, the local LLM processes the prompt, and the generated response is returned to the client without requiring a cloud-based LLM API.
3. Conclusion
Integrating a local LLM with Spring Boot using Spring AI and LM Studio provides a simple way to build AI-powered Java applications without depending on a cloud-based LLM service. LM Studio makes it easy to download and run models locally while exposing an OpenAI-compatible API, and Spring AI provides the ChatClient abstraction needed to communicate with that API using familiar Spring programming patterns. With only a few dependencies, configuration properties, a service, and a REST controller, we can send prompts from a Spring Boot application to a locally running model and return the generated responses to clients. This approach is particularly useful for local development, experimentation, offline environments, and applications where greater control over model execution and data privacy is important.
