Transformer Deep Dive blog post

Carlo Lepelaars

Transformer Deep Dive blog post

Transformer, Attention is all you need

“If you want to learn something well, explain it.” – Richard Feynman

Over the Christmas break I wrote a deep dive about the fundamental building block of so many awesome AI models nowadays (GPT-3, DALL-E, etc.): Transformers!

Includes background, PyTorch code and formulas. Hope you learn as much from it as I did writing it! 🙂

*** Article Link ***

This post was written for Weights and Biases. More blog posts that I wrote for this platform can be found here.

https://wandb.ai/carlolepelaars/transformer_deep_dive/reports/Transformer-Deep-Dive–VmlldzozODQ4NDQ