We did it โ the first decentralized training of a 10B model is complete!
Trained across the US, Europe, and Asia ๐
Post-training with
@arcee_ai is underway, and a full open-source release is coming in ~1 week, including: base model, checkpoints, post-trained model and data.