# What is fine-tuning? Definition and business implications > Fine-tuning is an adaptation technique for an already-trained AI model, which consists of continuing its training on a dataset specific to your use case. It modifies the model's internal parameters, in contrast to RAG, which simply injects context at query time. ## Definition Three families of fine-tuning coexist in 2026. Full fine-tuning: all the model's parameters are readjusted, which requires high compute power and a substantial data volume. Parameter-efficient fine-tuning (PEFT): only a small fraction of parameters is adjusted, often less than 1% via the LoRA technique (*Low-Rank Adaptation*, Hu et al., 2021). In the original paper, LoRA reduces the number of trainable parameters by 10,000 times and the GPU memory requirement by 3 times, compared with a fully fine-tuned GPT-3 175B, with comparable quality. Instruction tuning: a variant consisting of teaching the model to follow structured instructions rather than adapting a business domain. Fine-tuning is powerful but expensive to maintain: each update of the knowledge base requires a retraining. For cases where data changes frequently, RAG is generally preferable. ## Example Illustrative case. A 200-employee financial audit firm seeks to automate the drafting of analysis notes from accounting datasets. Three quantified options. Full fine-tuning of an open-weight 70-billion-parameter model on 5,000 historical notes: training cost about 15,000 euros, optimal quality but six-week setup time. LoRA fine-tuning on the same model: cost 800 euros, quality 90% to 95% of full fine-tuning, four days. RAG on the 5,000 notes without fine-tuning: indexing cost 50 euros, quality 75% to 85%, one day, but automatic re-indexing for every new note. The choice depends on the priority criterion: quality, cost, freshness. ## For a leader Launching a full fine-tuning project as the first AI-adaptation reflex, thinking you have to train the model on your data for it to work with your business. This route commits a significant budget, a rare technical skill (specialised data scientists), and a heavy maintenance cycle: each documentary update triggers a costly retraining. | Following an escalation logic. Start by testing in zero-shot, then few-shot, then RAG. Fine-tuning comes only fourth, and only for cases where RAG plateaus: a very specific editorial style to reproduce, business terminology inaccessible to instructions, critical latency. Prioritise parameter-efficient fine-tuning (LoRA), which sharply reduces training cost without significantly degrading quality, and keep RAG as a complement for documentary freshness. ## FAQ **What is fine-tuning, and how does it differ from RAG?** Fine-tuning adapts an already-trained model by continuing its training on a dataset specific to your use case, which modifies its internal parameters. RAG, by contrast, simply injects context at query time without touching the model. The difference drives cost and maintenance. **What forms of fine-tuning exist?** Three families coexist. Full fine-tuning readjusts all parameters and demands high compute power. Parameter-efficient fine-tuning (PEFT), via the LoRA technique, often adjusts less than 1% of parameters and sharply reduces the number of trainable parameters and the memory required, with no quality loss measured in the original paper. Instruction tuning teaches the model to follow structured instructions rather than master a business domain. **Should we start with fine-tuning to adapt AI to our business?** No, that is the reflex to avoid. Better to follow an escalation logic: test zero-shot first, then few-shot, then RAG, and keep fine-tuning only in fourth place. A full fine-tuning project commits a significant budget, a rare skill and heavy maintenance. **What do the different options actually cost?** Illustrative case. Take an audit firm that wants to automate analysis notes from 5,000 historical notes, on an open 70-billion-parameter model. Full fine-tuning: about 15,000 euros and six weeks, for optimal quality. LoRA fine-tuning: 800 euros and four days, for 90% to 95% of that quality. RAG without fine-tuning: 50 euros and one day, 75% to 85% quality, with automatic reindexing for every new note. **When is fine-tuning truly justified?** Only where RAG plateaus: a very specific editorial style to reproduce, business terminology inaccessible to instructions, or critical latency. In those cases, favour parameter-efficient fine-tuning (LoRA), which sharply reduces training cost without significantly degrading quality, and keep RAG as a complement for documentary freshness. ## See also - [Training](https://wyp.agency/en/glossary/training/) - [RAG (Retrieval-Augmented Generation)](https://wyp.agency/en/glossary/rag/) - [Parameter](https://wyp.agency/en/glossary/parameter/) - [LLM (Large Language Model)](https://wyp.agency/en/glossary/llm/) - [Prompt engineering](https://wyp.agency/en/glossary/prompt-engineering/) - [RLHF (reinforcement learning from human feedback)](https://wyp.agency/en/glossary/rlhf/) ## Sources - LoRA: Low-Rank Adaptation of Large Language Models, Hu et al., arXiv:2106.09685, 2021 (https://arxiv.org/abs/2106.09685) - Language Models are Few-Shot Learners, Brown et al., NeurIPS 2020 (https://arxiv.org/abs/2005.14165) --- Entry from the WYP.agency AI glossary · https://wyp.agency/en/glossary/fine-tuning/ · Last updated : 2026-10-14