Transformer, Attention is all you need

Transformer Deep Dive blog post

“If you want to learn something well, explain it.” – Richard Feynman Over the Christmas break I wrote a deep dive about the fundamental building block of so many awesome AI models nowadays (GPT-3, DALL-E, etc.): Transformers! Includes background, PyTorch code and formulas. Hope you learn as much from it as I did writing it!…
Read more