Training GPT Models with PyTorch
A training runbook for GPT models in PyTorch: torch.compile, DDP scaling, checkpoints that actually resume, and LR schedules tied to real steps.
A training runbook for GPT models in PyTorch: torch.compile, DDP scaling, checkpoints that actually resume, and LR schedules tied to real steps.
What this skill does
Four failure modes this prevents: a GPU left underused, a checkpoint that restores weights but not optimizer state, a learning-rate schedule misaligned with the steps actually trained, and a loop with no signal to stop before it overfits.
Extracted by Loreto from github.com/karpathy/nanoGPT on 10 September 2026. One run, eighty six seconds, three skills — this is one of them. Nobody wrote it down first; it was read out of the source.
Free, with the whole package: SKILL.md, a README.md, and a generated test.
Get Training GPT Models with PyTorch
Free to download — drops straight into .claude/skills/.