Last reviewed:
What is a foundation model? Definition and business implications
A foundation model is a generalist AI model trained on massive, diversified data, which serves as a reusable base for dozens or even hundreds of different business applications, by adaptation (fine-tuning, RAG, prompt) rather than by specific training.
The term foundation model was introduced by the Stanford CRFM in 2021 (Bommasani et al., On the Opportunities and Risks of Foundation Models) to designate a new class of models characterised by two properties. First, large-scale training on generalist corpora: these models are not designed for a specific task, but to capture the statistical regularities of a vast domain (language, image, code). Second, their downstream adaptability: the same model can be specialised afterwards for translation, summarisation, classification, code, image analysis, through adaptation techniques (fine-tuning, RAG, prompt engineering) far less costly than the initial training. GPT, Claude, Gemini, Llama, and Mistral are textual foundation models. Stable Diffusion and DALL-E are visual foundation models. Nearly all current enterprise AI applications rely on a foundation model, without necessarily knowing it.
Concrete example
The 2021 Stanford CRFM report identified foundation models as a paradigm shift: before 2018, each AI task (translation, summarisation, classification, vision) required its own model specifically trained, on dedicated data. After 2018 and the emergence of large transformers, a single generalist model can address dozens of tasks through light adaptation. For a mid-cap, this changes the economic equation: instead of funding five specialised AI developments at 200,000 euros each, you adapt an existing foundation model five times, at 10,000 to 30,000 euros per adaptation. The entry barrier collapses by an order of magnitude.
See also
Further reading
On the Opportunities and Risks of Foundation Models, Stanford CRFM, Bommasani et al., 2021
Sources
- On the Opportunities and Risks of Foundation Models, Bommasani et al., Stanford CRFM, arXiv:2108.07258, 2021. https://arxiv.org/abs/2108.07258
- Center for Research on Foundation Models, Stanford HAI. https://crfm.stanford.edu/