# What is an AI parameter? Definition and business implications

> A parameter of an AI model is a numerical value learned during training that determines the strength of connections between the artificial neurons of the network. The parameter count measures the model's capacity to store associations between the elements of its training corpus.

## Definition
An artificial neural network is made up of layers of neurons linked by connections, each weighted by a numerical value. These weights are the model's parameters. At the start of training, they are initialised randomly. With each example processed, the backpropagation algorithm slightly adjusts these weights to reduce the error between the model's prediction and the expected answer. After billions of iterations, the parameters converge towards values that encode the statistical regularities of the corpus. The higher the parameter count, the more fine-grained associations the model can store, but the more its training costs (in GPU, electricity, data) and the slower its inference. A technique called *quantisation* makes it possible to reduce the numerical precision of each parameter (from 32 to 8 or 4 bits), dividing the required memory by 4 to 8, generally without significant degradation in quality.

## Example
In 2017, the original transformer had 65 million parameters (base model). Llama 4 Scout, released by Meta in April 2025, has 109 billion parameters (17 billion of them active for each token): its weights take about 436 GB at full precision (FP32), and about 55 GB at 4-bit quantisation (INT4), which lets it fit on a single H100 GPU. This compression makes it possible to deploy models on far more modest hardware: a 7-billion-parameter model quantised to 4 bits takes only about 3.5 GB (7 billion × 0.5 byte), whereas the weights of GPT-3 (175 billion parameters) weigh about 350 GB in FP16 and require several GPUs.

## For a leader
“What is the parameter count of the model you are proposing, and why this size?” This question, asked of your IT director or your AI provider, opens three more. First, would a smaller model do the job for our use case? For many specific use cases, mid-sized models are enough and cost far less to run; test this on your own cases. Second, is this model quantised, and at what precision (FP16, INT8, INT4)? Quantisation divides the required memory and directly conditions the hosting cost. Third, where are these parameters stored and executed? The answer to this question determines your actual level of sovereignty over the data processed.

## FAQ

**What is a parameter in an AI model?**
It is a numerical value learned during training that sets the strength of a connection between the network's artificial neurons. These weights start out random, then get adjusted over billions of iterations until they encode the statistical regularities of the corpus. The parameter count measures the model's capacity to store associations.

**Should you always pick the model with the most parameters?**
Not necessarily. For many specific use cases, mid-sized models are enough and cost far less to run; test this on your own cases. A higher parameter count means costlier training and slower inference; the right question is whether a smaller model is enough for your use case.

**What is quantisation, and why should an executive care?**
Quantisation lowers the numerical precision of each parameter (from 32 to 8 or 4 bits), which divides the required memory by 4 to 8, generally without significant loss of quality. It therefore conditions hosting cost directly. For illustration, the weights of Llama 4 Scout (109 billion parameters) take about 436 GB at full precision, but only about 55 GB at 4-bit, which lets it fit on a single H100 GPU.

**How does the parameter count bear on data sovereignty?**
Because it conditions where the model can be stored and run. A quantised lightweight model, such as Mistral 7B, runs on standard hardware (about 3.5 GB of weights at 4 bits), whereas a model like GPT-3 (175 billion parameters) demands several GPUs (about 350 GB of weights in FP16). Knowing where the parameters are stored and executed determines your actual level of sovereignty over the data processed.

## See also

- [LLM (Large Language Model)](https://wyp.agency/en/glossary/llm/)
- [Transformer](https://wyp.agency/en/glossary/transformer/)
- [Fine-tuning](https://wyp.agency/en/glossary/fine-tuning/)
- [Inference](https://wyp.agency/en/glossary/inference/)
- [Training](https://wyp.agency/en/glossary/training/)
- [SLM (Small Language Model)](https://wyp.agency/en/glossary/small-language-model/)
- [Neural network (artificial)](https://wyp.agency/en/glossary/neural-network/)

## Sources

- Language Models are Few-Shot Learners, Brown et al., NeurIPS 2020 (https://arxiv.org/abs/2005.14165)
- The Llama 4 herd, Meta AI, 5 April 2025 (https://ai.meta.com/blog/llama-4-multimodal-intelligence/)
- Attention Is All You Need, Vaswani et al., arXiv:1706.03762, 2017 (base model: 65 million parameters) (https://arxiv.org/abs/1706.03762)

---
Entry from the WYP.agency AI glossary · https://wyp.agency/en/glossary/parameter/ · Last updated : 2026-10-14