Transformer Deep Dive blog post
“If you want to learn something well, explain it.” – Richard Feynman
Over the Christmas break I wrote a deep dive about the fundamental building block of so many awesome AI models nowadays (GPT-3, DALL-E, etc.): Transformers!
Includes background, PyTorch code and formulas. Hope you learn as much from it as I did writing it! 🙂