Before 2017, models read text the way you read a fax coming out of a machine: one word at a time, in order, carrying a summary forward. Long sentences degraded that summary. The transformer replaced the process with something closer to a search.
Queries, keys and values
Every word in the input is turned into three vectors. The query asks a question. The key advertises what a word has to offer. The value is the content it hands over.
To process one word, the model compares its query against every key in the sentence, scores how well each pair matches, converts those scores into weights, and returns a blend of the corresponding values. A word that needs a subject will score highly against the noun that is its subject, wherever that noun sits in the sentence.
The distance between two words stops mattering. What matters is whether they are relevant to each other.
Why it scaled
Because every comparison is independent, they can all be computed at once. That maps cleanly onto GPUs, which are built for doing thousands of small multiplications in parallel. The older sequential approach could not be parallelised across time steps — step five needed step four to finish first.
That is the whole trick. Not a better theory of language, but an operation that a graphics card could run flat out.
The bill
Comparing every word to every other word means the work grows with the square of the input length. Double the context, quadruple the computation. Much of the research since has been about paying that bill more cheaply — sparse patterns, sliding windows, cached keys — without losing the property that made attention worth having.
