Back to the blog 
Reading series
Build a Large Language Model (from Scratch)
By Sebastian Raschka
Chapter-by-chapter reading notes on Sebastian Raschka's book. A complete path to understanding and building an LLM from the ground up.
LLMDeep LearningNLPTransformers
Chapters 4/7
- 00
Introduction to PyTorch
Chapter 0: A hands-on introduction to PyTorch, tensors, computation graphs and autograd.
65 min read
- 01
Understanding large language models
Reading notes on Chapter 1: an introduction to LLMs, the transformer architecture, and the pretraining and fine-tuning process.
12 min read
- 02
Working with text data
Chapter 2: Tokenization, BPE encoding, and preparing data for language models.
27 min read
- 03
Coding attention mechanisms
A deep dive into the heart of LLM architecture: simplified self-attention, trainable weights, causal attention and multi-head attention.
52 min read
- 04
Implementing a GPT model from scratch
Coming soon
- 05
Pretraining on unlabeled data
Coming soon
- 06
Fine-tuning for classification and instruction following
Coming soon