imdigitalashish / gpt-from-scratch

A single executed notebook building and training a small GPT with RoPE and a phase-aware sparse MoE experiment