Learn / How LLMs Actually Work
Tokens & embeddings
Turn text into vectors: tokenizers, embeddings, similarity, and position. A class in How LLMs Actually Work. Free to start, no card.
Curated lessons in this class
A character tokenizer
Turn text into integers a model can read, and back again.
- Play with a tokenizer first
- Build the vocabulary
- String -> integer
- Integer -> string
- Encode and decode
Train a mini BPE tokenizer
Learn a subword vocabulary by merging frequent pairs.
- Words as lists of symbols
- Count adjacent pairs
- Merge the top pair
- Repeat to grow the vocabulary
Embeddings are a lookup table
Map token ids to vectors the model can do math on.
- An embedding table
- Look up one token
- Look up a whole sequence
Cosine similarity
Measure how close two vectors point, the basis of 'meaning'.
- The dot product
- Lengths (norms)
- Put it together
- Find the nearest
Positional encoding
Give an order-blind model a sense of position.
- Why position needs encoding
- sin for even, cos for odd
- The full (seq_len, d) matrix
- Add it to the embeddings
How CodeTrain teaches this
No videos and no multiple-choice quiz at the end. Each lesson is a handful of small steps in a real editor. You type the code yourself, it runs, and the tutor reviews what you actually wrote. It will not hand you the answer, and the step doesn't advance until your code passes. What you get at the end is a record of which steps you wrote unaided, not a completion percentage.
These curated courses are the starting point, not the product
Everything in this catalog is a curated course we wrote by hand. They exist so there is something to start on. What CodeTrain actually does is build a lesson on demand: name any topic you want to understand and it writes one for you on the spot, or paste any public repository and it teaches you that codebase from the code that is really in it. You are not limited to the list below, and most people stop using it within a session or two.