Beginner workshop shows how LLMs are built
The workshop teaches how work without requiring prior math or knowledge. It uses code and spreadsheet-style examples to move from basic to deep , transformer design, pre-training, and post-training.
It covers tokenizers that split text by character, word, BPE, and SentencePiece methods. It also explains embeddings, attention, normalization, loss functions, , optimizers, and training settings such as learning rate and batch size.
GPU-focused topics include PyTorch, torch.compile(), fused kernels, CUDA, and Triton. The attention section includes MHA, GQA, MQA, and MLA, which are useful for understanding why some model designs can be faster or cheaper to run.
Key points
- The material starts from basic and moves toward full LLM design.
- It covers tokenizers, embeddings, attention, transformers, and model training.
- It includes GPU execution topics such as PyTorch, CUDA, and Triton.
- Attention variants like GQA, MQA, and MLA can help explain speed and cost tradeoffs.