Last reviewed: May 24, 2026

What is fine-tuning? Definition and business implications

Fine-tuning is an adaptation technique for an already-trained AI model, which consists of continuing its training on a dataset specific to your use case. It modifies the model's internal parameters, in contrast to RAG, which simply injects context at query time.

Three families of fine-tuning coexist in 2026. Full fine-tuning: all the model's parameters are readjusted, which requires high compute power and a substantial data volume (typically 1,000 to 100,000 examples). Parameter-efficient fine-tuning (PEFT): only a small fraction of parameters is adjusted, often less than 1% via the LoRA technique (Low-Rank Adaptation, Hu et al., 2021). This approach divides compute cost by 100 to 1,000, with no significant quality degradation in most cases. Instruction tuning: a variant consisting of teaching the model to follow structured instructions rather than adapting a business domain. Fine-tuning is powerful but expensive to maintain: each update of the knowledge base requires a retraining. For cases where data changes frequently, RAG is generally preferable.

Concrete example

A 200-employee financial audit firm seeks to automate the drafting of analysis notes from accounting datasets. Three quantified options. Full fine-tuning of a Llama 3 70B on 5,000 historical notes: training cost about 15,000 euros, optimal quality but six-week setup time. LoRA fine-tuning on the same model: cost 800 euros, quality 90% to 95% of full fine-tuning, four days. RAG on the 5,000 notes without fine-tuning: indexing cost 50 euros, quality 75% to 85%, one day, but automatic re-indexing for every new note. The choice depends on the priority criterion: quality, cost, freshness.

Good vs bad reflex

BAD REFLEX

Launching a full fine-tuning project as the first AI-adaptation reflex, thinking you have to train the model on your data for it to work with your business. This route commits a significant budget (10,000 to 100,000 euros per cycle), a rare technical skill (specialised data scientists), and a heavy maintenance cycle: each documentary update triggers a costly retraining.

GOOD REFLEX

Following an escalation logic. Start by testing in zero-shot, then few-shot, then RAG. Fine-tuning comes only fourth, and only for cases where RAG plateaus: a very specific editorial style to reproduce, business terminology inaccessible to instructions, critical latency. Prioritise parameter-efficient fine-tuning (LoRA), which divides cost by 100 with no quality degradation, and keep RAG as a complement for documentary freshness.

Sources

LoRA: Low-Rank Adaptation of Large Language Models, Hu et al., arXiv:2106.09685, 2021. https://arxiv.org/abs/2106.09685 (accessed 2026-05-24)
Language Models are Few-Shot Learners, Brown et al., NeurIPS 2020. https://arxiv.org/abs/2005.14165 (accessed 2026-05-24)

← Back to glossary

What is fine-tuning? Definition and business implications

Concrete example

See also

Further reading

Sources