Attention & Transformer Lecture 3 01 Seq2Seq Bottleneck → Bahdanau Attention 02 Soft Dictionary 03 Scaled Dot-Product Attention 04 FLOPs 05 Memory Bottlenecks & FlashAttention 06 Multi-Head Attention 07 Transformer Block & Residual Stream 08 Post-LN vs Pre-LN 09 QK Norm 10 Feed-Forward Network 11 Mixture of Experts 12 Positional Information 13 ALiBi · NoPE · Hybrid Attention 14 Context Extension 15 Causal Mask & Next-Token Loss 16 KV Cache 17 MHA · GQA · MQA · MLA 18 Attention Head Patterns 19 NoPE Model Cards 20 Context Extension Revisited 21 Shifted Targets & Backward