Architecture
Attention
The mechanism at the core of the transformer: for each word fragment, the model calculates which other places in the text are relevant to its meaning — what “she” refers to, which verb belongs to the subject. From many such weighted relations, understanding of context arises. “Attention Is All You Need” was the name of the paper, and the title has proved to be a program.
An example
In the sentence “The lawyer advised the client because she was uncertain,” a model has to decide what “she” refers to. Attention is the mechanism that does this: when processing “she,” “client” gets a high weight and “lawyer” a low one. The same calculation runs for every word against every other, in dozens of parallel passes, each specialized in a different kind of relationship.
A closer look
For each token, the model calculates how strongly it should look at every other and forms a weighted average from that. That way a pronoun can refer to a noun fifty words earlier without this connection being hard-wired.
Why it matters
Attention weights can be read out and color-coded, and that is exactly what is often sold as explainability: “See, the model paid attention to this word.” That is not a justification. The weights show which words computationally worked together, not why the result came out as it did — the same output can often be produced with entirely different weight distributions. When a regulator or a court demands a justification, an attention heat map is not one. Take it as a diagnostic tool for developers, not as proof.
Commonly misunderstood
The weights can be color-coded and then look like a justification. They are only conditionally suited as an explanation: high attention does not mean that the place was decisive for the answer.