What is…
Quantization
Shrinking a model by storing its numbers with less precision, so it runs faster on smaller hardware.
Model weights are normally stored as 16-bit numbers. Quantization rounds them to 8, 4, or even fewer bits. The model gets 2–4× smaller and faster with surprisingly little quality loss.
It's the reason you can run a capable open model on a laptop or phone (apps like Ollama and LM Studio rely on it).
💡 Think of it like
Saving a photo as a high-quality JPEG instead of a giant RAW file. Slightly less detail, dramatically smaller.
🧠 Test yourself
Which of these describes Quantization?
Related terms
🎮 Learn AI by playing
40 bite-size missions, boss battles and a certificate. Free.
Start the bootcamp →☀️ One AI term every morning
Plus the day's top stories, in your inbox by 8am.