Transformers

Dissecting the Transformer. Part 1 In this article the author attempts to make sense of the transformer architecture — everyone has heard of it, everyone “knows” it, yet hardly anyone truly understands it. So the goal here is to explain things at a conceptual level, without arcane mathematical formulas and unreadable definitions. Let’s admit that explaining theory in simple, clear words is a true art, accessible to few of the Landau-and-Lifshitz caliber — but in our age of decadence and postmodernism we shall try as best we can. And yes, the author does not guarantee 100% accuracy of what follows — only his own understanding of it. ...

How Transformers Work: A Plain Explanation Let’s revisit transformers and try to figure out how they actually work — as usual, minimal math and no AI-generated filler. The author makes no claims to absolute truth and may occasionally get things slightly wrong. 1. Let’s Start Simple Let’s refresh our memory on how recurrent networks worked — the ones that “loop back to themselves”. Hard not to think of Hegel’s Absolute Spirit, which travels a path of self-alienation only to return to itself — pure Hegelian recurrence. ...