TL;DR
nanochat is Andrej Karpathy's minimal end-to-end ChatGPT clone: tokenizer, pretraining, fine-tuning, inference and a web UI in one small codebase you can read.
nanochat is a complete, minimal implementation of a ChatGPT-style model, written by Andrej Karpathy and released as open source in October 2025. One repository takes you from raw text to a chat interface you can talk to: tokenizer, pretraining, fine-tuning, inference, and a small web UI.
The point is not the model it produces. A model trained this way is not competitive with anything you would pay for. The point is that the entire pipeline fits in one codebase small enough to read end to end, which is a different and rarer thing.
For engineers deciding whether it is worth their weekend. 7 minute read.
What is actually in the repository?
| Stage | What it does |
|---|---|
| Tokenizer | Trains a byte-pair encoding tokenizer from scratch, implemented in Rust for speed |
| Pretraining | Trains a transformer on a large web-text corpus |
| Midtraining | Adapts the base model toward conversation and structured formats |
| Supervised fine-tuning | Trains on instruction and chat data |
| Reinforcement learning | Optional, and the least essential stage for most readers |
| Inference | A serving engine with KV caching |
| Web UI | A ChatGPT-like interface to talk to what you trained |
That list is the reason the project exists. Most educational LLM code stops after pretraining, which is the half that is well explained everywhere and the half that produces nothing you can use.
Who made it, and why does that matter?
Andrej Karpathy: founding member of OpenAI, former director of AI at Tesla, and author of the teaching material a large share of working ML engineers learned from. nanoGPT, his earlier project, is one of the most forked educational repositories in the field.
That matters here for a practical reason rather than a reputational one. Code written to be read is different from code written to work, and very little published LLM training code is written to be read. The pedagogical intent is the feature.
How is nanochat different from nanoGPT?
| nanoGPT | nanochat | |
|---|---|---|
| Scope | Pretraining only | The whole pipeline, tokenizer to chat UI |
| What you get at the end | A base model that continues text | Something you can hold a conversation with |
| Tokenizer | Uses an existing one | Trains its own |
| Serving | Not included | Included, with a web interface |
nanoGPT answers "how is a transformer trained". nanochat answers "how does a trained transformer become a product you can open in a browser", and that second question is where most people's understanding actually stops.
What hardware does nanochat need?
This is the question most people arrive with, and the honest answer has two halves.
The published target is a single 8xH100 node. Karpathy describes a speedrun costing roughly $100 and taking around four hours on that hardware, which is a rented machine for an afternoon rather than a purchase.
It is not a laptop project. You can read every line on a laptop, and you can run the inference stage against a small model, but the pretraining stage assumes serious GPUs. Anyone planning to work through it should budget for rented compute rather than expect it to run locally.
Both figures above are the author's, from the project's own announcement. We have not reproduced them, and you should treat any cost quoted for a training run as a function of the hardware market on the day it was measured.
What is it good for, and what is it not?
Good for: understanding the shape of the whole pipeline; seeing where each stage's data comes from; having a reference implementation small enough to modify and observe. If you have ever been unsure what "midtraining" means in practice, the answer is fifty lines here rather than a paper.
Not good for: producing a model to put in front of users. The result is a demonstration. Judged as a product it is worse than every free API you could call instead, and it is not trying to be otherwise.
The value is the readable pipeline, not the weights that fall out of the end of it.
Why would an engineering team care?
Because the failure modes in production retrieval and agent systems are usually not in the model. They are in the boundaries between stages, and this is the smallest artefact that shows every boundary at once: what the tokenizer decided, what pretraining saw, what fine-tuning changed, what inference does with a KV cache.
A team that has read one full pipeline end to end argues about the right things afterwards. That is worth an afternoon of rented GPUs whether or not you ever train anything again.
Share this article
Evaluation harnesses, judges, retries and circuit breakers — how we keep an agent system honest once real traffic arrives.
