Reference
Learning
Places to go further than this guide does: explanations drawn by hand, videos, pages that run a model in front of you, courses, books, writing that keeps up with the field, and code small enough to read. Every one is somebody else’s work, chosen because it made something clearer.
The list is picked, not gathered, and it will grow. Some of it asks for no more than curiosity and some of it asks for code, so each card says what it is. Each link was checked on the date above, and each opens in a new tab.
Reference
Writing that explains one idea thoroughly, with the pictures to go with it.
The Illustrated Transformer (opens in a new tab)
Jay Alammar · June 2018
The transformer drawn one step at a time, from a single word’s vector through self-attention and many heads to the decoder. Still the gentlest way in to the diagrams every later explanation borrows.
jalammar.github.io
The Illustrated GPT-2 (opens in a new tab)
Jay Alammar · August 2019
The same hand-drawn approach turned on a model that only predicts the next word, which is the shape of every chat model since.
jalammar.github.io
The Illustrated Stable Diffusion (opens in a new tab)
Jay Alammar · October 2022
How a picture is made by taking noise away, step by step, steered by the words of a prompt. A companion to the generative media module.
jalammar.github.io
What Is ChatGPT Doing … and Why Does It Work? (opens in a new tab)
Stephen Wolfram · February 2023
A long essay in plain language on how choosing one likely word after another can add up to something that reads like thought. Older than today’s models, and still sound on the idea underneath them.
writings.stephenwolfram.com
The Big LLM Architecture Comparison (opens in a new tab)
Sebastian Raschka
What actually differs inside recent open models such as DeepSeek, Qwen, Gemma, Kimi and gpt-oss, drawn side by side. Kept up to date as new models appear.
magazine.sebastianraschka.com
The Annotated Transformer (opens in a new tab)
Harvard NLP
The paper that introduced the transformer, with working code beside each paragraph. For readers who would rather see the idea run than read about it.
nlp.seas.harvard.edu
Tracing the thoughts of a large language model (opens in a new tab)
Anthropic · March 2025
What researchers found when they looked inside a model at work: planning a rhyme before writing the line, and doing sums by routes it cannot describe. The most readable way in to interpretability.
anthropic.com
Videos
Explanations that move, for when an idea is easier watched than read.
How DeepSeek Rewrote the Transformer [MLA] (opens in a new tab)
Welch Labs
How multi-head latent attention shrinks the memory a model keeps for every token it has read, and why that let DeepSeek serve long contexts cheaply.
youtube.com
Transformers, the tech behind LLMs (opens in a new tab)
3Blue1Brown · April 2024
What a GPT does with a sentence, animated from words to vectors to a prediction. Perhaps the clearest visual explanation there is.
3blue1brown.com
Attention in transformers, step-by-step (opens in a new tab)
3Blue1Brown · April 2024
The chapter after it, on attention alone: how each word takes meaning from the words around it, with every multiplication shown.
3blue1brown.com
Deep Dive into LLMs like ChatGPT (opens in a new tab)
Andrej Karpathy · February 2025
Three and a half hours for a general audience on how a chat model is made, from a crawl of the web through training to the assistant, and why it hallucinates.
youtube.com
[1hr Talk] Intro to Large Language Models (opens in a new tab)
Andrej Karpathy · November 2023
The shorter way in: what a model is, how it is trained, and where it is going, in an hour.
youtube.com
Let’s build GPT: from scratch, in code, spelled out. (opens in a new tab)
Andrej Karpathy · January 2023
A small GPT written live, one line at a time, and trained on Shakespeare. For readers who code.
youtube.com
Visualisers
Pages that run or draw a real model, so you can see each step happen.
Transformer Explainer (opens in a new tab)
Polo Club of Data Science, Georgia Tech
GPT-2 small running in the browser. Type a prompt and watch it become tokens, attention and a choice of next word, with temperature and sampling to play with.
poloclub.github.io
LLM Visualization (opens in a new tab)
Brendan Bycroft
A model drawn in three dimensions, every weight and sum in place, walked through one layer at a time from a tiny nano-GPT up to the scale of GPT-3.
bbycroft.net
Tiktokenizer (opens in a new tab)
David Duong
Type anything and see it cut into the tokens a model actually reads, and how many there are. It makes sense of token prices and limits at a glance.
tiktokenizer.vercel.app
Diffusion Explainer (opens in a new tab)
Polo Club of Data Science, Georgia Tech
Stable Diffusion in the browser, from prompt to picture, with the noise taken away one step at a time. By the team behind Transformer Explainer.
poloclub.github.io
AnimatedLLM (opens in a new tab)
Zdeněk Kasner and Ondřej Dušek
Step-by-step animations of open models working on real inputs, from how the next token is chosen down to what each layer does to it. Made for teaching.
animatedllm.github.io
A Neural Network Playground (opens in a new tab)
Daniel Smilkov and Shan Carter
Train a tiny network by hand and watch it learn to separate two colours of dot. The oldest page here, and still the quickest feel for what training is.
playground.tensorflow.org
Neuronpedia (opens in a new tab)
Neuronpedia
Browse the features researchers have found inside real models, and what switches each one on. Advanced, and the nearest thing to looking inside a model yourself.
neuronpedia.org
Courses
Free courses with a beginning, a middle and an end, from no maths at all to building a model yourself.
Elements of AI (opens in a new tab)
University of Helsinki and MinnaLearn
What AI is and is not, for anyone, with no maths and no code. A natural next step after this guide’s primer.
elementsofai.com
Anthropic Academy (opens in a new tab)
Anthropic
Free courses from the maker of Claude, among them AI Fluency, on working with a model well. Its product courses are about Claude alone.
anthropic.skilljar.com
LLM Course (opens in a new tab)
Hugging Face
Hands-on, from tokenisers and transformers to fine-tuning a model and training it to reason. It goes further than running AI locally goes.
huggingface.co
AI Agents Course (opens in a new tab)
Hugging Face
What an agent is and how to build one, with the libraries people use for it. A companion to agentic work in practical AI.
huggingface.co
Diffusion Models Course (opens in a new tab)
Hugging Face
How image models are built and trained, and how to shape one of your own, in code.
huggingface.co
Neural Networks: Zero to Hero (opens in a new tab)
Andrej Karpathy
A series of videos that starts with how a network learns at all and ends with a GPT, everything written and trained on screen.
karpathy.ai
Practical Deep Learning for Coders (opens in a new tab)
fast.ai
Deep learning taught from the top down: build something that works first, then find out why. For readers who can already program.
course.fast.ai
CS336: Language Modeling from Scratch (opens in a new tab)
Stanford University
The most serious course here: building a language model yourself, from the tokeniser through training to alignment, with the lectures online.
cs336.stanford.edu
Books
Whole textbooks, free to read online, for when a page is not enough.
Understanding Deep Learning (opens in a new tab)
Simon J. D. Prince
A textbook free to read as a PDF, with some of the best diagrams in the subject, covering transformers and diffusion models along the way.
udlbook.github.io
Dive into Deep Learning (opens in a new tab)
Aston Zhang, Zachary Lipton, Mu Li and Alexander Smola
A textbook with code on every page that runs as you read it, used in hundreds of university courses.
d2l.ai
Blogs and newsletters
People who write about the field as it moves, which this guide cannot do.
Simon Willison’s Weblog (opens in a new tab)
Simon Willison
The most useful day-to-day record of using models well: what each new release is good for, prompt injection, and tools built in an afternoon.
simonwillison.net
One Useful Thing (opens in a new tab)
Ethan Mollick
A professor at Wharton on what AI changes at work and in teaching, written for people who use it rather than build it.
oneusefulthing.org
Lil’Log (opens in a new tab)
Lilian Weng
Long technical surveys, each the best single page on its subject, among them agents, hallucination and how models are made to reason.
lilianweng.github.io
Chip Huyen’s blog (opens in a new tab)
Chip Huyen
Building real products on models: evaluation, agents, and what goes wrong between a demo and something people rely on.
huyenchip.com
Repositories
Code written to be read, where the clearest explanation of a model is one small enough to run yourself.
nanochat (opens in a new tab)
Andrej Karpathy · October 2025
A complete small chat model, from tokeniser and training to a web page to talk to it, in about eight thousand lines meant to be read. It calls itself “the best ChatGPT that $100 can buy”.
github.com
nanoGPT (opens in a new tab)
Andrej Karpathy
The full version of the model built in the Let’s build GPT video: small enough to read in an evening, and able to reproduce the smallest GPT-2. Its author now calls it old and points to nanochat, but it is the shorter read.
github.com
LLMs-from-scratch (opens in a new tab)
Sebastian Raschka
The code for his book Build a Large Language Model (From Scratch), a chapter at a time in notebooks: tokenising, attention, pretraining, then fine-tuning to follow instructions. The code is free; the book is not.
github.com
Prompt Engineering Interactive Tutorial (opens in a new tab)
Anthropic
Nine chapters of notebooks with exercises, from the shape of a good prompt to avoiding hallucination. It was written around Claude 3 Haiku, an older model, and needs an API key to run; the lessons still hold.
github.com