Neural Networks, Built and Explained
A learning notebook by Satish Bajagain, following Andrej Karpathy’s Zero to Hero.
These are the explanations I collected while learning: comments beside the code, handwritten derivations, small experiments, and the steps that made an idea click.
I have completed the seven lessons represented here, including the character-level GPT implementation. The collection follows a path from scalar autograd and character models to a decoder transformer. It accompanies my portfolio and independent sleep-stage research and my completed GPT-from-scratch project. These are learning implementations following Karpathy, with original explanations and derivations; they are not peer-reviewed research results.
Read the notebooks¶
| Notebook | Explore |
|---|---|
| 01 · Micrograd | Derivatives, computational graphs and gradient descent |
| 02 · Bigrams | Counts, probability, likelihood and learning |
| 03 · Embeddings and the MLP | Representations, context and generalization |
| 04 · Activations and normalization | Initialization, saturation and training diagnostics |
| 05 · Backpropagation by hand | Handwritten mathematics and manual gradients |
| 06 · Hierarchical context | Progressively combining characters |
| 07 · Decoder transformers | Generation, attention and transformer components |
A short study session¶
Start with a recall question, think through an answer, then return to the original explanation. For a worked example, follow the matrix-gradient reading guide alongside the handwritten derivation.
The notebooks preserve the original wording, comments, intermediate attempts and saved outputs. Additional reading guides and recall answers are labeled separately. About these notes.