nanochat: Train and Serve a $100 Mini ChatGPT in 4 Hours

Nanochat is a minimalist, full-stack ChatGPT-style LLM you can train and serve on a single 8×H100 node in about four hours for roughly $100. It includes the entire pipeline and produces a report with benchmark metrics, plus a simple web UI for chatting. The repo provides straightforward guidance for scaling to larger, more capable models while keeping the code readable and hackable.

Key Points

• End-to-end, minimal, hackable LLM pipeline that trains and serves a ChatGPT-like model with a single script on one 8×H100 node in ~4 hours (~$100).
• Post-run, a web UI (python -m scripts.chat_web) provides chat, and report.md summarizes benchmarks such as CORE, ARC, GSM8K, HumanEval, MMLU, and ChatCORE.
• Scaling guidance: a ~$300 d26 model (~12 hours) can surpass GPT‑2 CORE; a ~$1000 tier (~41.6 hours) is discussed; adjustments involve more data shards, higher depth, and tuning device_batch_size to avoid OOM.
• Runs on 8×A100 or a single GPU via gradient accumulation (with longer runtime); mostly vanilla PyTorch, with potential tinkering for other backends.
• Project ethos: small, readable, dependency-light code; basic tests (especially tokenizer); recommended tools for Q&A (files-to-prompt, DeepWiki); MIT licensed and intended as LLM101n capstone.

Sentiment

The Hacker News community is strongly supportive of nanochat, viewing it primarily as an important educational contribution. While a small minority dismisses it as Karpathy hype or questions the practical utility of the resulting model, the overwhelming consensus is that making LLM training accessible and understandable for $100 is a significant achievement. The most substantive criticism relates to the irony of Karpathy finding AI coding tools unhelpful for this project, which generated lively but good-natured debate rather than genuine opposition.

In Agreement

• The project is a landmark educational resource that democratizes LLM training knowledge and makes the full pipeline accessible to individuals
• $100 to train an end-to-end conversational LLM is a remarkable achievement in accessibility, and the minimal, readable code philosophy is ideal for learning
• Karpathy's educational content (nanoGPT, LLM101n) is exceptionally valuable, and this project will propel small language model development
• The ChatGPT-style web interface bundled with training and inference code makes this a uniquely complete educational package

Opposed

• The project is 'just hype for Karpathy' since many other tiny LLMs already exist that run on cheap hardware, and some people have trained models for less
• The $100 claim is misleading since it requires renting an 8xH100 node, and buying such hardware costs hundreds of thousands of dollars
• The irony of the 'vibe coding' coiner finding AI coding tools unhelpful raises questions about whether these tools work for anything beyond standard applications
• The model's factually incorrect outputs undermine the 'ChatGPT' branding, though defenders note the errors are intentionally shown to illustrate training-stage limitations