What is…
Inference
Running a trained model to get an answer — as opposed to training it.
Training builds the model (months, huge cost, done once). Inference uses it (milliseconds to minutes, every time you hit Enter). When companies talk about GPU costs for serving users, or "tokens per second," they're talking about inference.
Reasoning models spend extra inference compute "thinking" before they answer — trading speed for quality.
💡 Think of it like
Training is going to medical school. Inference is seeing a patient.
🧠 Test yourself
Which of these describes Inference?
Related terms
🎮 Learn AI by playing
40 bite-size missions, boss battles and a certificate. Free.
Start the bootcamp →☀️ One AI term every morning
Plus the day's top stories, in your inbox by 8am.