What is…

Attention

The mechanism that lets a model weigh which earlier words matter most for the next one.

In "The trophy didn't fit in the suitcase because it was too big," attention is what lets the model connect "it" to "trophy" rather than "suitcase." Every token scores its relevance to every other token, and the model blends information accordingly.

Attention is also why long contexts are expensive: comparing every token to every other token grows fast as the text gets longer.

💡 Think of it like

A highlighter the model runs over the conversation, over and over, deciding what to focus on for each word it writes.

🧠 Test yourself

Which of these describes Attention?

Related terms

Transformer

The neural-network design (from Google, 2017) that nearly every modern AI model is built on.

Context window

How much text a model can 'see' at once — your conversation, files and instructions, all measured in tokens.

🎮 Learn AI by playing

40 bite-size missions, boss battles and a certificate. Free.

Start the bootcamp →

☀️ One AI term every morning

Plus the day's top stories, in your inbox by 8am.

← Transformer Hallucination →