A language model comes trained on a huge amount of general text. Fine-tuning means continuing that training on a set of chosen examples, to change its behaviour permanently: adopting a style, following a format, mastering specialized vocabulary or getting better at a specific task.
How it works
The principle is simple, the execution less so. You prepare hundreds or thousands of examples in the form "here is a request, here is the expected answer". You run a training process that adjusts part of the model's parameters so it reproduces those answers. You get a new version of the model, which you evaluate on examples it has not seen, then put into service.
Several providers offer this service for some of their models, and open models can be fine-tuned on your own servers. Lighter techniques, such as LoRA, published in 2021 by a Microsoft team, adjust only a small part of the parameters and make the operation less expensive.
What fine-tuning does well
- Imposing a style or format: a very particular brand tone, an answer structure that is always the same, a precise data format
- Specializing a repetitive task: classifying requests, extracting fields from a document, summarizing according to a fixed template
- Shortening instructions: learned behaviour no longer needs to be described in the prompt with every question
- Using a smaller model: a light model fine-tuned on a task can match a more powerful model that is not, at a lower usage cost
What fine-tuning does badly
Fine-tuning is often wrongly presented as the way to teach the model a company's information. It is rarely the right tool for that:
- Facts change. A changed rate, opening hour or condition would require retraining. With a knowledge base, you only need to correct the document
- Learned facts cannot be cited. A fine-tuned model cannot say where a piece of information comes from. An assistant that reads your documents can show its source
- Learned facts get mixed up. Training does not prevent hallucinations; it can even make the model more confident about poorly retained details
- Maintenance is expensive. Every new version of the base model, and they come several times a year, requires redoing the fine-tuning and the tests
Fine-tuning or RAG?
The two approaches meet different needs and can be combined:
| Need | Most suitable approach |
|---|---|
| Answering from documents that change | RAG or documents passed in context |
| Citing the source of every answer | RAG or documents passed in context |
| Adopting a very particular tone or format | Detailed prompt, then fine-tuning if it is not enough |
| Performing a repetitive task at high volume | Fine-tuning a smaller model |
| Mastering very specialized jargon | Glossary in the documents, then fine-tuning if needed |
For an assistant that answers the visitors of an SME's website, the sensible order is almost always the same: first a good prompt, then good documents, then testing. Fine-tuning only comes as a last resort, for a specific need the first two steps do not cover.
Questions to ask before considering fine-tuning
- Does the problem come from the model or the documents? Most of an assistant's errors come from a document that is vague, outdated or missing
- Would a more precise prompt be enough? A few examples in the prompt, known as in-context examples, often achieve the same effect without training
- Do you have enough good-quality examples? Inconsistent examples produce an inconsistent model
- Who will maintain the fine-tuned model? And what happens when the provider retires the base model
Fine-tuning and personal data
Training examples end up, in a sense, inside the model. If real conversations are used as examples, the personal data they contain must be removed or anonymized, and this purpose must have its own legal basis under the GDPR. This is one reason the question "are my conversations used to train a model?" deserves a written answer from every supplier.
And in KernChat?
KernChat does not fine-tune any model on its customers' data. Assistants use models available on the market (Google Gemini, Anthropic Claude or OpenAI), steered by a mission written with the organization and fed by its knowledge spaces. Messages are sent to the model provider to generate the answer; KERN-IT does not use them to train a model. Knowledge is corrected by changing its source, and the correction applies from the next question.