When people say a book is good what does that mean? Good for infotainment? To get an intuitive understanding? To be able to open up Openthropic and compete against the frontier labs? What can you get in the book thats not available via a chatbot? Interesting question
Nicely done. For me the most fascinating thing about attention heads is the place where Attention matrix is already computed and is getting multiplied by Value vector. It behaves exactly like pushing Value vector through Dense layer of ordinary network where Attention matrix forms weights of that layer. So attention head is trained to construct this small single layer network dynamically during inference from Key and Query. And that's the point. That's rarely underlined in explanations of LLMs architecture and for me it's quite amazing that it works so well. This mechanism easy to observe in this particular visualization if you click through it.
It's definitely part of many intro explanations, but it sits in a weird position where you have to know MLPs well but not Transformers for this to be enlightening, which is a shrinking set of people nowadays. Either someone is a novice to all this, or know both, so beginner Transformer explanations can rarely assume deep familiarity with MLPs.
But yes, a Transformer block can be thought of as basically input-dependently deciding the weight vector of a dense layer. In classical MLP-like networks there was never any multiplication between input-dependent values (there are exceptions, like Squeeze-and-Excitation layers). Multiplication is always between parameters (that don't depend on the input) and activations that depend on the input. With Transformers attention provides a lot of multiplicative interactions between input dependent activations.
Also obligatory: Schmidhuber talked about this a long time ago.
Welch Labs has a beautiful visualization in their YouTube video about Vision Language Action Models, where the attention of the prompt can be traced to the exact origin pixels in the image from one of those matrices.
Look at this poster [1] (its low-res, full res is paid). Also see this video for context [2] - it talks about deepseek's innovation, but explains attention well.
Above I was pointing to the moment where multiplication AV happens. In normal Dense layer in MLPs when you go through the layer you compute y=Wx, where x is an input and y is output (before gactivation) and W is a weight matrix. This W is usually what's produced through training process. This multiplication AV looks like Wx. If we take one column from matrix V in the poster and call it v, we can pretend for one moment that W=A and x=v [3]. So this multiplication Av works like linear transformation in ordinary network Wx. But in case of attention heads A is not trained directly but produced during inference, and is not trained directly like W is in ordinary network. In this case network is trained to produce A during inference.
> "Instead of picking the highest-probability token, we can use different selection strategies to balance safety and creativity in the generated text".
Safety is definitely the wrong word here.
Temperature 0 generated text actually has a weird "lack of surprise" character that makes it seem artificial. [1]
> "high-probability texts can be dull or repetitive. Humans use language as a means of communicating information, aiming to do so in a simultaneously efficient and error-minimizing manner; in fact, psycholinguistics research suggests humans choose each word in a string with this subconscious goal in mind."
I'd completely drop the dropout explanation. It's just not part of the modern recipe anymore, AFAICT.
As for the ambitious goal of explaining transformers with a single interactive visualization, I just have a hard time imagining a person is going to newly understand both word embeddings (word2vec blew my mind in 2014) and also gain an understanding of attention.
I am making my own visualizations for a presentation on "Full Bandwidth Transformers"[2] that I am giving tomorrow at the Deep Learning Study Group (SF) (on zoom for the non-locals)[3]. It's not meant to be stand alone/context free, but I'd love some feedback.
I get that this is for explaining GPT-2, but I really hope laymen don't use it as an example of how modern models work (ex. absolute positional encoding is no longer used)
edit: I know that it mentions its not modern, but these kinds of details have major implications in terms of the representations a model can learn, which is in many ways the most important part!
Well the embedding itself is in some latent space.
Wq projects it to the space of queries. I.E What questions is this token asking?
Wk projects it to the space of keys. I.E What questions does this token answer.
Wk projects it to the space of values. I.E What are those answers?
Of course this explanation is prescribed onto the matrixes after the fact.
You can in fact do weird stuff like construct weights so attention calculates least squares, or sorts numbers, or other weird constructions like a transformer that calculates gradient descent steps. It seems to be very flexible in terms of what functions on data it can encode.
For the uninitiated, I can't recommend enough, The Illustrated Transformer:
https://jalammar.github.io/illustrated-transformer/
His book is also very good: https://www.amazon.com/Hands-Large-Language-Models-Understan...
DRM-free EPUB and PDF (for those who'll pay double to legally avoid Kindle DRM):
https://www.ebooks.com/en-us/book/211460386/hands-on-large-l...
seconded, the book is very good.
When people say a book is good what does that mean? Good for infotainment? To get an intuitive understanding? To be able to open up Openthropic and compete against the frontier labs? What can you get in the book thats not available via a chatbot? Interesting question
Excellent resource for beginners
Can it do Jev too?
Nicely done. For me the most fascinating thing about attention heads is the place where Attention matrix is already computed and is getting multiplied by Value vector. It behaves exactly like pushing Value vector through Dense layer of ordinary network where Attention matrix forms weights of that layer. So attention head is trained to construct this small single layer network dynamically during inference from Key and Query. And that's the point. That's rarely underlined in explanations of LLMs architecture and for me it's quite amazing that it works so well. This mechanism easy to observe in this particular visualization if you click through it.
It's definitely part of many intro explanations, but it sits in a weird position where you have to know MLPs well but not Transformers for this to be enlightening, which is a shrinking set of people nowadays. Either someone is a novice to all this, or know both, so beginner Transformer explanations can rarely assume deep familiarity with MLPs.
But yes, a Transformer block can be thought of as basically input-dependently deciding the weight vector of a dense layer. In classical MLP-like networks there was never any multiplication between input-dependent values (there are exceptions, like Squeeze-and-Excitation layers). Multiplication is always between parameters (that don't depend on the input) and activations that depend on the input. With Transformers attention provides a lot of multiplicative interactions between input dependent activations.
Also obligatory: Schmidhuber talked about this a long time ago.
Welch Labs has a beautiful visualization in their YouTube video about Vision Language Action Models, where the attention of the prompt can be traced to the exact origin pixels in the image from one of those matrices.
Can you explain this more, maybe dumb it down a little? Sounds important. I have t quite been able to get the attention section to click for me.
Look at this poster [1] (its low-res, full res is paid). Also see this video for context [2] - it talks about deepseek's innovation, but explains attention well. Above I was pointing to the moment where multiplication AV happens. In normal Dense layer in MLPs when you go through the layer you compute y=Wx, where x is an input and y is output (before gactivation) and W is a weight matrix. This W is usually what's produced through training process. This multiplication AV looks like Wx. If we take one column from matrix V in the poster and call it v, we can pretend for one moment that W=A and x=v [3]. So this multiplication Av works like linear transformation in ordinary network Wx. But in case of attention heads A is not trained directly but produced during inference, and is not trained directly like W is in ordinary network. In this case network is trained to produce A during inference.
[1] https://www.welchlabs.com/store/mladeepseek-attention-poster...
[2] https://www.youtube.com/watch?v=0VLAoVGf_74
[3] When multiplying A by V, we perform the same linear transform Av_i for each i-th column of V.
As someone with an EE degree (though a sysadmin), this use of the term "transformer" is constantly confusing. :)
(Also "cryto" for cryptocurrency rather than cryptography.)
transformer is climbing up in the most-overloaded-terms category, jockeying for a spot somewhere among “service”, “template” and “zone”
Seriously. I was expecting more Optimus Prime.
Regarding the temperature explanation:
> "Instead of picking the highest-probability token, we can use different selection strategies to balance safety and creativity in the generated text".
Safety is definitely the wrong word here.
Temperature 0 generated text actually has a weird "lack of surprise" character that makes it seem artificial. [1]
> "high-probability texts can be dull or repetitive. Humans use language as a means of communicating information, aiming to do so in a simultaneously efficient and error-minimizing manner; in fact, psycholinguistics research suggests humans choose each word in a string with this subconscious goal in mind."
I'd completely drop the dropout explanation. It's just not part of the modern recipe anymore, AFAICT.
As for the ambitious goal of explaining transformers with a single interactive visualization, I just have a hard time imagining a person is going to newly understand both word embeddings (word2vec blew my mind in 2014) and also gain an understanding of attention.
I am making my own visualizations for a presentation on "Full Bandwidth Transformers"[2] that I am giving tomorrow at the Deep Learning Study Group (SF) (on zoom for the non-locals)[3]. It's not meant to be stand alone/context free, but I'd love some feedback.
https://rrenaud.github.io/fullbandwidth_transformer_viz/
[1] https://arxiv.org/abs/2202.00666 [2] https://arxiv.org/abs/2608.08888 [3] https://www.meetup.com/deep-learning-sf/events/316601593/
Hm, I wonder why my laptop is suddenly pulling about 5 fps while I'm just browsing HN?
The nefarious "2.2 GB RAM usage within 10 seconds" tab open in the background:
Great site, intuitive description. I also found [1] very useful in the past.
[1] https://bbycroft.net/llm
this is awesome, thanks for sharing
Damn that page took down my Chromebook, never happened before..
Text under the "Examples" section...
"Try examples while GPT-2 model is being downloaded (600MB)"
That's a hefty chunk of download and likely compute too.
twice...
I get that this is for explaining GPT-2, but I really hope laymen don't use it as an example of how modern models work (ex. absolute positional encoding is no longer used)
edit: I know that it mentions its not modern, but these kinds of details have major implications in terms of the representations a model can learn, which is in many ways the most important part!
Yeah it's a bit odd, especially since RoPe is a lot more conceptually simple imo.
This is how pedagogy works. You must simplify something.
I think a better simplification would be to remove the position-encoding altogether, and just discuss transformers as set models.
I'm talking about bringing up a specific outdated details, not about simplifying it
Why did the inner EE in me think this was going to be a power transformer lol
Would have been very interested. I know there are genius/insane people out there winding their own transformers.
I didn't realize this was a think until I heard of people doing it in a a CRT forum.
I suspect it will be getting more popular as flyback transformers get harder and harder to find.
I first heard of it here: https://ludens.cl/paradise/turbine/turbine.html (CTRL+F transformer), a pretty fascinating DIY medium distance power transmission setup.
Could have been the cars that turn into robots for that matter
I never understood the thinking behind the separate key query value matrixes? What are they doing exactly?
Well the embedding itself is in some latent space.
Wq projects it to the space of queries. I.E What questions is this token asking?
Wk projects it to the space of keys. I.E What questions does this token answer.
Wk projects it to the space of values. I.E What are those answers?
Of course this explanation is prescribed onto the matrixes after the fact.
You can in fact do weird stuff like construct weights so attention calculates least squares, or sorts numbers, or other weird constructions like a transformer that calculates gradient descent steps. It seems to be very flexible in terms of what functions on data it can encode.
Nice work. I really appreciate this tool for enhancing my limited understanding the mechanism(s) behind attention and LLMs.
This is not at all what I was hoping for. Expected a lot more Unicron.
Definitely more than meets the eye
The page diagram looks like Cybertron
Great UI and it was very helpful to me as a newcomer