🎛️Zachary is a brilliant instructor — laser-focused on helping us learn how AI pros truly work at scale.This course genuinely bridges the gap between academic theory and real-world distributed training.
🎛️What I’ll Apply Next
🔹 Build Expert Parallelism (MoE) from scratch using a small local GPU cluster — and later scale it up with cloud GPUs for training compact models.
🔹 Recreate parts of the OLMo-2 pre-training pipeline at a much smaller scale, at least up to a few checkpoints, to study the training dynamics hands-on.
#ScratchToScale#Maven#DistributedTraining#DeepLearning#AI#LLMs#ZacharyMueller#MachineLearning#ScalingAI
New website is now live! The course, my newsletter, and the lightning lessons all can be accessed at scratchtoscale(.)tech, a one-stop-shop for everything distributed that I'm up to right now.