Last reviewed:
What is an LLM? Definition and business implications
An LLM (Large Language Model) is a type of artificial intelligence trained on text corpora of several hundred billion words, which produces natural language by predicting, word by word, the most probable continuation of a given text.
An LLM is a very large neural network, typically built on the transformer architecture (Vaswani et al., 2017), trained to predict the next word in a sequence from the preceding words. This simple objective, repeated over trillions of tokens, is enough to produce models capable of answering questions, drafting texts, translating, reasoning, and coding. The LLM family spans very different sizes, from the lightweight 7-billion-parameter model (Mistral 7B) to latest-generation models, whose size most vendors no longer disclose (OpenAI has not published the parameter count of GPT-4). Size is no longer the sole quality criterion: since 2024, well-trained 70-billion-parameter models rival on common benchmarks with models almost six times larger (Llama 3.3 70B against Llama 3.1 405B, according to Meta), at a much lower inference cost.
Concrete example
The original transformer, published by Google in 2017, contained 65 million parameters. GPT-3, unveiled by OpenAI in 2020, had 175 billion, that is 2,700 times more in three years. Since then, inflation has continued: Llama 3.1 (Meta) reaches 405 billion in open-source, while most proprietary model vendors no longer publish the parameter count of their recent models. Yet size is no longer the only lever. Mixture-of-experts architectures activate only a fraction of the parameters for each token, and distillation transfers the capabilities of a large model to a smaller one: two ways to cut inference cost without following this inflation.
FAQ
What exactly is an LLM?
An LLM (Large Language Model) is a very large neural network, typically built on the transformer architecture, trained on gigantic text corpora. Its objective is simple: to predict, word by word, the most probable continuation of a text. Repeated over trillions of tokens, that objective is enough to yield models that can answer, draft, translate, reason, and code.
Is a bigger model necessarily a better one?
No, size is no longer the sole quality criterion. Since 2024, well-trained 70-billion-parameter models have rivalled models almost six times larger (Llama 3.3 70B against Llama 3.1 405B, according to Meta) on common benchmarks, at a much lower inference cost. Two techniques help: mixture-of-experts architectures, which activate only a fraction of the parameters per token, and distillation, which transfers a large model's capabilities to a smaller one.
Why treat the choice as infrastructure rather than software?
Because parameter size primarily determines inference cost, and only secondarily quality. A smaller model is often enough for a specific use case, at a much lower cost: on 14 October 2026, a million input tokens cost $10 with Claude Fable 5.1 and $0.10 with Claude Haiku 5.5 (prompts up to 100,000 tokens), a ratio of 1 to 100. It is therefore wise to test mid-sized models systematically before paying for the flagships.
Which families of LLM exist, and what does the choice commit?
Three families coexist: proprietary (GPT, Claude, Gemini), open-source (Llama, Mistral, DeepSeek), and sovereign (Mistral AI in Europe). The choice simultaneously commits cost, data confidentiality, and the degree of supplier dependence. It is not a purely technical decision.
Is a single benchmark enough to judge a model?
No. A model's performance is also that of its tokeniser, its context window, and its robustness to adversarial instructions. No single benchmark captures these three axes, which nonetheless weigh heavily in production.
See also
Sources
- Attention Is All You Need, Vaswani et al., NeurIPS 2017. https://arxiv.org/abs/1706.03762
- Language Models are Few-Shot Learners, Brown et al., NeurIPS 2020. https://arxiv.org/abs/2005.14165
- Introducing Meta Llama 3, Meta AI (pretrained on over 15T tokens). https://ai.meta.com/blog/meta-llama-3/
- GPT-4 Technical Report, OpenAI, 2023 (no information on model size). https://arxiv.org/abs/2303.08774
- Llama 3.3 70B, model card, Meta. https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md
- Pricing, Claude documentation (Anthropic), accessed 14 October 2026. https://platform.claude.com/docs/en/about-claude/pricing