Last reviewed:
What is AI training? Definition and business implications
Training is the construction phase of an AI model, during which the model ingests a massive corpus and adjusts its billions of internal parameters to learn the statistical regularities of that corpus. It is the most expensive operation in a model's life cycle.
Training an AI model breaks down into two distinct phases. Pre-training consists of exposing the model to a very large, generalist corpus (Wikipedia, digitised books, source code, web archives) so it learns the statistical structures of language. This phase lasts weeks to months on clusters of thousands of GPUs. At its end, the model correctly predicts the continuation of a text without knowing your business use case. Post-training covers the alignment and adaptation stages: supervised learning on annotated examples, reinforcement learning with human feedback (RLHF), and fine-tuning on specific data. These steps orient the model's behaviour towards what is expected of a useful assistant (safety, tone, format). Training is a high fixed-cost operation. Once done, the model is frozen: to modify its knowledge, you must either retrain it (expensive) or supply context with each query via a RAG (economical).
Concrete example
According to the Stanford AI Index 2025 report (Epoch AI estimates), training costs for frontier models have exploded: about 670 dollars for the original transformer in 2017, 79 million dollars for GPT-4 in 2023, 192 million dollars for Google Gemini 1.0 Ultra, released at the end of 2023. Meta's Llama 3.1 405B (2024) is estimated to have cost about 170 million dollars. This inflation, estimated at 2.4 times per year since 2016 (Cottier et al.), places frontier-model training out of reach of almost all organisations, apart from a few labs funded with billions of dollars. For a mid-cap, the question is not whether to train, it is to choose the right pre-trained model.
FAQ
What is training an AI model?
It is the construction phase, during which the model ingests a massive corpus and adjusts its billions of parameters to learn that corpus's statistical regularities. It is the most expensive operation in a model's life cycle. Once done, the model is frozen.
What is the difference between pre-training and post-training?
Pre-training exposes the model to a very large generalist corpus (Wikipedia, books, code, web archives) so it learns the structures of language; it runs for weeks to months on thousands of GPUs. Post-training covers alignment and adaptation (supervised learning, RLHF, fine-tuning) and steers the model's behaviour towards what is expected of a useful assistant: safety, tone, format.
Should a company train its own model?
For nearly all companies, no. According to the Stanford AI Index 2025, training a frontier model rose from about 670 dollars for the 2017 transformer to 79 million for GPT-4 in 2023, with inflation estimated at 2.4 times a year. That puts such training out of reach of almost all organisations. For a mid-cap, the question is not whether to train, but which pre-trained model to choose.
If we can't train, where does adaptation to our business happen?
In post-training (fine-tuning) or in the architecture around the model (RAG, agents), not in the initial training. That is where the only accessible ground for differentiation lies. To change a frozen model's knowledge without retraining it, you supply context with each query through a RAG, an economical approach.
Does training carry an environmental stake?
Yes, a large one. According to the Stanford AI Index, the footprint of frontier training ranges from 5,184 tons of CO2 equivalent for GPT-4 in 2023 to about 72,800 tons for Grok 4 in 2025. Choosing a smaller, already-trained model is therefore also a sobriety choice.
See also
Sources
- Artificial Intelligence Index Report 2025, Stanford HAI, chapter 1. https://hai.stanford.edu/ai-index/2025-ai-index-report
- The Rising Costs of Training Frontier AI Models, Cottier et al., arXiv:2405.21015, 2024. https://arxiv.org/abs/2405.21015
- Artificial Intelligence Index Report 2026, Stanford HAI, chapter 1 (Research and Development). https://hai.stanford.edu/ai-index/2026-ai-index-report/research-and-development