Large language models can perform many tasks without any additional training. They can answer questions, summarize text, generate code, classify information, and interact with tools. However, when building an AI agent for a specific application, a general-purpose model may not always behave in the way the application requires.
An AI agent may need to follow a particular workflow, select the correct tool, generate arguments in a specific format, use domain-specific terminology, or consistently respond according to a set of rules. Developers can use prompting to guide this behavior, and they can use Retrieval-Augmented Generation (RAG) when the agent needs access to external information. In some cases, however, these approaches are not enough, and fine-tuning becomes an option.
Fine-tuning allows a pretrained language model to learn from additional examples. Instead of training a model from the beginning, we start with an existing model and adapt it for a particular task or style. For AI agents, this can mean teaching the model how to use tools, how to produce a particular output format, or how to follow a consistent workflow.
There are several ways to fine-tune a language model. This article focuses on three concepts that are especially relevant when working with modern LLMs: Supervised Fine-Tuning (SFT), LoRA, and QLoRA.
1. What is LLM Fine-Tuning?
An LLM is normally trained on a very large amount of text before developers use it in an application. This initial process is called pretraining. During pretraining, the model learns general patterns about language and how different pieces of text relate to one another.
For example, a pretrained model can learn that a question usually has an answer, that programming languages have particular syntax, and that words have different meanings depending on their context. However, it does not automatically know how a particular company’s customer-support agent should behave.
Fine-tuning is an additional training process performed after pretraining. We provide the model with examples that represent the behavior we want it to learn. The model then adjusts some or all of its parameters based on those examples.
Imagine that we are building an order-management agent. The agent has access to tools such as get_order, check_inventory, and cancel_order. A general-purpose model may understand these tool names, but it may not consistently know when each tool should be used or what arguments should be supplied. A training example could show the desired behavior:
{
"instruction": "Cancel order 7281.",
"tool": "cancel_order",
"arguments": {
"order_id": "7281"
}
}
After seeing many examples of this type, the model can learn patterns associated with the task.
The goal is not to teach the model everything it needs to know about the world. Instead, fine-tuning is normally used to adapt an existing model to a particular task, behavior, format, or domain.
Why Fine-Tuning Matters for AI Agents
AI agents normally do more than generate a final text response. They may need to decide what action to take, call one or more tools, process the results, and then produce a final response.
For example, a customer-support agent might receive a request such as: I was charged twice for my subscription. The agent may need to identify the customer, retrieve the payment history, determine whether two charges exist, and then create a refund request if the duplicate charge is confirmed.
A general model may understand the overall task but still occasionally select the wrong tool or produce arguments in an inconsistent format. A carefully designed fine-tuning dataset can provide clear examples of the expected behavior and help the model handle these tasks more consistently.
A good dataset should include successful interactions, edge cases, invalid requests, tool failures, and situations where the agent should refuse or escalate a request. It should also contain a variety of examples, including negative examples, rather than focusing only on successful interactions.
This is one reason dataset quality is important. If the training data contains inconsistent or incorrect examples, the model can learn and reproduce those same inconsistencies.
Before Fine-Tuning, Try Prompting
Fine-tuning should not normally be the first solution to every LLM problem. Before changing the model, it is worth checking whether the desired behavior can be achieved with a better prompt.
Prompting means providing instructions to the model at inference time. The model itself does not change. Instead, the application changes the instructions and context sent with each request.
A simple prompt might say:
You are a customer-support agent.
Always ask for the order number before checking an order. Use the order lookup tool when the customer asks about order status. Do not create a refund until the payment has been verified.
The application can also include examples in the prompt. For example:
User: Where is order 1234?
Assistant: I will check the current status of order 1234.
The model can use this example to understand the expected behavior for similar requests. Prompting is attractive because it is quick to change and does not require a training pipeline. If the behavior you want can be reliably achieved through instructions and examples, there may be little reason to fine-tune the model.
The distinction between prompting, RAG, and fine-tuning is important. Prompting changes the instructions given to the model, RAG provides additional information at runtime, and fine-tuning changes the model’s learned behavior.
2. What is RAG?
Retrieval-Augmented Generation, or RAG, is another common technique for adapting an LLM application. Instead of changing the model, RAG retrieves relevant information from an external source and places that information into the model’s context before generating a response.
Suppose a company has thousands of internal documents containing product information. Instead of fine-tuning the model on those documents, the application can store the documents in a searchable system. When the user asks a question, the application retrieves the relevant documents and sends them to the LLM.
This is useful because the information can be updated without retraining the model. For example, if a company changes its refund policy, the updated policy can be added to the knowledge base. A RAG system can retrieve the new policy during the next request.
Fine-tuning works differently. It changes the model’s behavior rather than acting as a live document store. A simple way to remember the distinction is:
| Approach | Main purpose |
|---|---|
| Prompting | Tell the model what to do |
| RAG | Give the model information it needs |
| Fine-tuning | Teach the model a repeatable behavior |
This distinction is important when building AI agents because many problems that initially look like fine-tuning problems are actually prompting or knowledge-retrieval problems.
3. What is Supervised Fine-Tuning?
Supervised Fine-Tuning, usually shortened to SFT, is a training method where the model learns from examples containing an input and an expected output. Suppose we want an agent to classify support requests. Our training dataset could contain examples such as:
{
"instruction": "The customer was charged twice.",
"response": "DUPLICATE_CHARGE"
}
Another example might be:
{
"instruction": "The customer wants their money returned.",
"response": "REFUND"
}
During SFT, the model learns from these examples and adjusts its parameters so that it becomes more likely to produce the desired responses for similar inputs. SFT can be used for many different tasks. For an AI agent, examples could teach the model how to classify requests, format responses, select tools, generate tool arguments, follow a workflow, or respond to specific types of errors.
It is important to understand that SFT and LoRA are not competing techniques. SFT describes the training approach, while LoRA describes a parameter-efficient way of modifying the model during that training. You can perform SFT using full fine-tuning, LoRA, or QLoRA.
Full Fine-Tuning
The most direct way to fine-tune an LLM is to update the model’s parameters during training. Large language models can contain billions of parameters. Updating all of these parameters requires significant memory because the training process needs to handle the model weights, gradients, optimizer states, and intermediate activations.
As models become larger, full fine-tuning becomes increasingly demanding.
For this reason, developers often use Parameter-Efficient Fine-Tuning, or PEFT. Instead of updating the entire model, PEFT methods train a much smaller number of additional parameters while keeping most of the original model unchanged. The Hugging Face PEFT library provides implementations of several such techniques, including LoRA.
LoRA is one of the most widely used PEFT approaches.
4. What is LoRA?
LoRA, short for Low-Rank Adaptation, was introduced in the paper LoRA: Low-Rank Adaptation of Large Language Models by Edward Hu and other researchers. The original paper proposed freezing the pretrained model weights and adding trainable low-rank matrices to Transformer layers. Instead of updating a large weight matrix directly, the training process learns a much smaller representation of the required update.
Suppose the original model contains a weight matrix called W. With full fine-tuning, we could update that matrix directly:
W → W + ΔW
LoRA represents the update using two smaller matrices:
ΔW = B × A
The resulting weight can therefore be thought of as:
W' = W + B × A
The matrices A and B have a much smaller rank than the original matrix. This means that considerably fewer parameters need to be trained. For beginners, the most important idea is that the original model is kept mostly unchanged, while a small trainable adapter learns how the model should behave differently.
Why LoRA is Useful
Imagine that you have a 7-billion-parameter model and want to adapt it for customer support. With full fine-tuning, the training process would update the model’s parameters directly, which can require significant memory and computational resources.
With LoRA, the original model remains frozen while the training process focuses on a much smaller set of adapter parameters. This reduces the number of parameters that need to be trained and makes the fine-tuning process more resource-efficient.
Another advantage is that the same base model can be adapted for different tasks using separate LoRA adapters. For instance, one adapter could be trained for customer support, another for legal assistance, and another for code generation. The base model remains unchanged, while each adapter contains the additional behavior learned for its specific task.
Because the adapters are much smaller than complete copies of the model, they are also easier to save, share, and manage. This makes it simpler to experiment with different specialized versions of a model without maintaining a separate fully fine-tuned model for every task.
5. What is QLoRA?
QLoRA extends the LoRA approach by combining LoRA adapters with quantization. The technique was introduced in the paper QLoRA: Efficient Finetuning of Quantized LLMs by Tim Dettmers and other researchers. The paper describes an approach that backpropagates through a frozen, 4-bit quantized language model into LoRA adapters.
The basic idea is:
Large pretrained model
↓
4-bit quantization
↓
Frozen base model
+
LoRA adapters
↓
Train the adapters
The base model is loaded using a lower-precision representation, reducing the memory required to store the frozen model. The LoRA parameters remain trainable. QLoRA introduced several techniques for reducing memory consumption, including 4-bit NormalFloat (NF4), double quantization, and paged optimizers.
A simple way to understand the relationship is that LoRA reduces the number of parameters that need to be trained. QLoRA combines LoRA with quantization to reduce the memory required to work with the base model.
LoRA vs QLoRA
| Technique | Base model | What is trained? | Main purpose |
|---|---|---|---|
| Full fine-tuning | Normal precision | Most or all parameters | Maximum flexibility at high resource cost |
| LoRA | Normal precision | Small LoRA adapters | Efficient parameter adaptation |
| QLoRA | Quantized, commonly 4-bit | Small LoRA adapters | Reduce memory requirements during fine-tuning |
QLoRA is therefore not a replacement for LoRA. It is a way of using LoRA while keeping the frozen base model in a quantized representation.
Why Quantization Helps
Quantization is the process of representing numerical values using fewer bits. For large language models, using a lower-precision representation can significantly reduce the amount of memory required to load the model.
In QLoRA, the base model remains frozen during fine-tuning. Loading this frozen model using a quantized representation, such as 4-bit precision, reduces its memory footprint while still allowing the smaller LoRA adapter parameters to be trained.
This makes it possible to fine-tune larger models with more limited hardware resources. The actual memory requirements still depend on factors such as the model size, sequence length, batch size, optimizer configuration, gradient accumulation, and other training settings.
6. Fine-Tuning vs RAG vs Prompting
Fine-tuning, RAG, and prompting are sometimes presented as competing approaches, but they solve different problems. Prompting changes the instructions sent to the model. RAG provides external information at runtime. Fine-tuning changes the model’s learned behavior. The following table provides a comparison:
| Approach | What changes? | Good for | Example | Main limitation |
|---|---|---|---|---|
| Prompting | The instructions sent to the model | Roles, instructions, formatting, simple behavior | “Always respond using JSON” | Instructions must be supplied at runtime |
| RAG | The information provided to the model | Company documents, policies, current information | Retrieve the latest refund policy | Depends on retrieval quality |
| SFT | The model learns from examples | Task-specific behavior | Teaching a support workflow | Requires training data |
| LoRA | Small adapter parameters are trained | Efficient model adaptation | Teaching consistent tool usage | Requires training and evaluation |
| QLoRA | Quantized model + LoRA adapters | Memory-efficient fine-tuning | Fine-tuning a larger model on limited hardware | Training configuration is more involved |
A useful approach is to start with prompting. If the problem is that the model does not have access to current or private information, consider RAG. If the problem is that the model repeatedly fails to follow a stable behavior even after good prompting and evaluation, fine-tuning can be considered.
When Should You Use Fine-Tuning?
Fine-tuning makes sense when you have identified a specific behavior that needs to become more consistent. For example, suppose an AI agent must always produce a particular structured response and you have many examples showing the exact desired format. Fine-tuning may help the model learn that behavior.
Tool use is another possible application. If a model repeatedly selects the wrong tool despite clear tool descriptions and examples in the prompt, a dataset containing correct tool-use examples can be used for supervised fine-tuning.
Fine-tuning can also help with a specialized style or terminology when the desired behavior is stable and repeated across many requests. The key is to identify a behavioral problem rather than simply deciding that the model needs more information.
When Should You Use RAG Instead?
RAG is usually more appropriate when the model needs access to information that changes regularly. Consider a company with a large collection of product manuals. The manuals may change every few weeks. Training the model every time a document changes would be inconvenient.
A RAG system can retrieve the latest document when the user asks a question. The same applies to policies, documentation, product catalogs, internal knowledge bases, and other information that needs to remain current.
Fine-tuning should generally not be treated as a replacement for a searchable knowledge base. A useful rule is: Use RAG when the model needs information. Consider fine-tuning when the model needs a more consistent behavior.
7. Conclusion
Fine-tuning allows developers to adapt an existing language model for a specific task, workflow, or behavior. For AI agents, this can include more consistent tool use, structured responses, specialized workflows, and domain-specific interaction patterns.
SFT, or Supervised Fine-Tuning, provides a way to train a model using examples that demonstrate the desired input and output behavior. LoRA makes fine-tuning more efficient by keeping the original model parameters frozen and training a smaller set of low-rank adapter parameters. QLoRA extends this approach by combining LoRA with low-bit quantization, reducing the memory required during training.
Fine-tuning is not always the first solution to consider. Prompting can handle many behavioral instructions, while RAG is often a better choice when an agent needs access to current, private, or frequently changing information. Fine-tuning becomes more relevant when the model needs to learn a stable behavior that should be applied consistently across many requests.
When fine-tuning is appropriate, LoRA and QLoRA provide practical ways to adapt models with fewer computational and memory requirements than traditional full fine-tuning. With a carefully prepared dataset, clear evaluation criteria, and an appropriate training configuration, these techniques can help build AI agents that perform specialized tasks more consistently.
