Repository: karpathy/LLM101n Branch: master Commit: c6de374acd4b Files: 1 Total size: 2.3 KB Directory structure: gitextract__b81y52r/ └── README.md ================================================ FILE CONTENTS ================================================ ================================================ FILE: README.md ================================================ # LLM101n: Let's build a Storyteller --- **!!! NOTE: this course does not yet exist. It is current being developed by [Eureka Labs](https://eurekalabs.ai). Until it is ready I am archiving this repo !!!** ---  > What I cannot create, I do not understand. -Richard Feynman In this course we will build a Storyteller AI Large Language Model (LLM). Hand in hand, you'll be able to create, refine and illustrate little [stories](https://huggingface.co/datasets/roneneldan/TinyStories) with the AI. We are going to build everything end-to-end from basics to a functioning web app similar to ChatGPT, from scratch in Python, C and CUDA, and with minimal computer science prerequisites. By the end you should have a relatively deep understanding of AI, LLMs, and deep learning more generally. **Syllabus** - Chapter 01 **Bigram Language Model** (language modeling) - Chapter 02 **Micrograd** (machine learning, backpropagation) - Chapter 03 **N-gram model** (multi-layer perceptron, matmul, gelu) - Chapter 04 **Attention** (attention, softmax, positional encoder) - Chapter 05 **Transformer** (transformer, residual, layernorm, GPT-2) - Chapter 06 **Tokenization** (minBPE, byte pair encoding) - Chapter 07 **Optimization** (initialization, optimization, AdamW) - Chapter 08 **Need for Speed I: Device** (device, CPU, GPU, ...) - Chapter 09 **Need for Speed II: Precision** (mixed precision training, fp16, bf16, fp8, ...) - Chapter 10 **Need for Speed III: Distributed** (distributed optimization, DDP, ZeRO) - Chapter 11 **Datasets** (datasets, data loading, synthetic data generation) - Chapter 12 **Inference I: kv-cache** (kv-cache) - Chapter 13 **Inference II: Quantization** (quantization) - Chapter 14 **Finetuning I: SFT** (supervised finetuning SFT, PEFT, LoRA, chat) - Chapter 15 **Finetuning II: RL** (reinforcement learning, RLHF, PPO, DPO) - Chapter 16 **Deployment** (API, web app) - Chapter 17 **Multimodal** (VQVAE, diffusion transformer) **Appendix** Further topics to work into the progression above: - Programming languages: Assembly, C, Python - Data types: Integer, Float, String (ASCII, Unicode, UTF-8) - Tensor: shapes, views, strides, contiguous, ... - Deep Learning frameworks: PyTorch, JAX - Neural Net Architecture: GPT (1,2,3,4), Llama (RoPE, RMSNorm, GQA), MoE, ... - Multimodal: Images, Audio, Video, VQVAE, VQGAN, diffusion
gitextract__b81y52r/ └── README.md
Condensed preview — 1 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (2K chars).
[
{
"path": "README.md",
"chars": 2398,
"preview": "# LLM101n: Let's build a Storyteller\n\n---\n\n**!!! NOTE: this course does not yet exist. It is current being developed by "
}
]
About this extraction
This page contains the full source code of the karpathy/LLM101n GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 1 files (2.3 KB), approximately 713 tokens. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.