# What is RAG? Definition and business implications

> RAG (Retrieval-Augmented Generation) is an AI architecture that pairs a search engine across your documents with a generative model. The model answers by relying on citable business data rather than on its training knowledge alone.

## Definition
RAG operates in three steps invisible to the end user. First, your documents (PDFs, web pages, knowledge base) are split and converted into numerical vectors, that is, mathematical coordinates representing their meaning, then stored in a vector database. Second, when a user asks a question, the system retrieves the passages semantically closest to the query. Third, these passages are injected into the prompt sent to the LLM, which composes an answer that explicitly relies on these sources. The practical consequence is major. The model no longer draws on its training memory; on every query it reads the relevant documents you supply. This is what sets RAG apart from fine-tuning, which modifies the model's internal parameters. RAG leaves the model intact and operates solely on what it has in view at the moment of responding.

## Example
A 2025 study published in *JMIR Cancer* compared chatbots with and without RAG, built on GPT-4 and GPT-3.5, on 62 cancer questions asked in Japanese. Without RAG, about 40% of answers contained a hallucination. Connected to Japan's Cancer Information Service, the RAG chatbots fell to 0% (GPT-4) and 6% (GPT-3.5) on topics covered by that source. Connected to Google search, they stayed between 6% and 10%, and rose to 19% and 35% on questions the reliable source did not cover. RAG sharply reduces hallucination, provided the source is reliable and covers the question. Another effect: the RAG chatbots were able to say they did not know.

## For a leader
Considering fine-tuning as soon as AI needs to be adapted to your company's specifics. That route demands rare technical skills, heavy compute infrastructure, and triggers a full retraining cycle every time documentary knowledge is updated. | Testing a RAG first. RAG only requires reindexing, and offers three decisive assets. Traceability first: every answer cites its documentary source, a condition of legal defensibility. In February 2024, in Moffatt v. Air Canada, a British Columbia tribunal held the airline liable for wrong information its website chatbot gave about a bereavement fare. Then freshness: adding a document to the base triggers its indexing without retraining. Finally control: the documentary base remains your property, regardless of the model provider.

## FAQ

**In practice, how does RAG work?**
In three steps invisible to the user. Your documents are split and turned into vectors stored in a vector database; for each question, the system retrieves the passages closest in meaning to the query; those passages are injected into the prompt, and the model writes its answer relying on these sources. The model no longer draws on its training memory, it reads the documents you supply on every query.

**What is the difference between RAG and fine-tuning?**
Fine-tuning modifies the model's internal parameters, which means retraining each time knowledge is updated. RAG leaves the model untouched and works only on the documents it has in view when answering: adding a document just requires reindexing. For data that changes often, RAG is generally preferable.

**Does RAG eliminate hallucinations?**
No, it reduces them, provided the source is reliable and covers the question. In a study published in 2025 in JMIR Cancer (62 questions asked in Japanese), about 40% of answers from chatbots without RAG contained a hallucination. Connected to Japan's Cancer Information Service, the RAG chatbots fell to 0% (GPT-4) and 6% (GPT-3.5) on topics covered by that source. Connected to Google search, they rose to 19% and 35% on questions the reliable source did not cover.

**Why test a RAG before considering fine-tuning?**
Because a RAG only requires reindexing, whereas fine-tuning demands rare skills, heavy compute infrastructure and a retraining cycle every time your documents change. RAG also offers three assets: traceability (every answer cites its source), freshness (a new document is indexed without retraining) and control (your documentary base stays your property, whatever the model provider).

**Why is RAG traceability a legal issue?**
Because every answer cites the documentary source it used, which makes the answer auditable after the fact and defensible. The Air Canada case in 2024 set a precedent: a chatbot hallucinated a refund policy and the airline lost the lawsuit that followed. Citing the source is the condition for defensibility in a dispute.

## See also

- [Hallucination](https://wyp.agency/en/glossary/hallucination/)
- [Fine-tuning](https://wyp.agency/en/glossary/fine-tuning/)
- [Vector database](https://wyp.agency/en/glossary/vector-database/)
- [Embedding](https://wyp.agency/en/glossary/embedding/)
- [LLM (Large Language Model)](https://wyp.agency/en/glossary/llm/)
- [Chatbot (conversational agent)](https://wyp.agency/en/glossary/chatbot/)

## Sources

- Reducing Hallucinations and Trade-Offs in Responses in Generative AI Chatbots for Cancer Information, Nishisako, Higashi & Wakao, JMIR Cancer, 2025;11:e70176 (https://pmc.ncbi.nlm.nih.gov/articles/PMC12425422/)
- Moffatt v. Air Canada, British Columbia Civil Resolution Tribunal, February 2024 (https://www.canlii.org/en/bc/bccrt/doc/2024/2024bccrt149/2024bccrt149.html)

---
Entry from the WYP.agency AI glossary · https://wyp.agency/en/glossary/rag/ · Last updated : 2026-10-14