What is…
Parameters
The billions of internal numbers a model learns during training — its 'knowledge', stored as weights.
Parameters (also called weights) are the adjustable numbers inside a neural network. Training nudges them, trillions of times, until the model gets good at predicting text. A "70B" model has about 70 billion of them.
More parameters generally means more capability — and more cost to run. But size isn't everything: training data quality, techniques like distillation, and smart architectures like mixture-of-experts let smaller models punch far above their weight.
💡 Think of it like
The knobs on a gigantic mixing board. Training is turning billions of knobs until the song sounds right.
📌 Example
"8B" and "405B" in a model's name refer to billions of parameters.
🧠 Test yourself
Which of these describes Parameters?
Related terms
LLM (Large Language Model)
A model trained on huge amounts of text to predict the next word — the engine behind ChatGPT, Claude, Gemini and Grok.
Open weights
A model whose trained parameters are published so anyone can download and run it themselves.
Quantization
Shrinking a model by storing its numbers with less precision, so it runs faster on smaller hardware.
Mixture of Experts (MoE)
A model built from many specialist sub-networks where only a few 'experts' activate for each token.
🎮 Learn AI by playing
40 bite-size missions, boss battles and a certificate. Free.
Start the bootcamp →☀️ One AI term every morning
Plus the day's top stories, in your inbox by 8am.