Introducing AI2 OLMo (Open Language Model)
Last month, the Allen Institute for AI (AI2) announced the development of an open, state-of-the-art generative language model: AI2 OLMo (Open Language Model)...
Cloudflare R2 and MosaicML: Train LLMs on Any Compute with Zero Switching Costs
Together, Cloudflare and MosaicML give users the freedom to train LLMs on any compute, anywhere in the world, for faster, cheaper training runs...
Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs
Introducing MPT-7B, the first entry in our MosaicML Foundation Series. MPT-7B is a transformer trained from scratch on 1T tokens of text and...
How We Trained Stable Diffusion for Less than $50k (Part 3)
In our previous blog post, we showed how we used the MosaicML platform, Streaming datasets, and the Composer library to train a Stable...
Benchmarking Large Language Models on NVIDIA H100 GPUs with CoreWeave (Part 1)
Benchmarking Large Language Models on NVIDIA H100 GPUs with CoreWeave The research and engineering teams here at MosaicML collaborated with CoreWeave, one of...
Training Stable Diffusion from Scratch for <$50k with MosaicML (Part 2)
We've replicated Stable Diffusion 2 for less than $50k, and we've open-sourced the training code so you can too! This is a 3x...
MosaicBERT: Pretraining BERT from Scratch for $20
With the MosaicBERT architecture + training recipe, you can now pretrain a competitive BERT-Base model from scratch on the MosaicML platform for $20...
MosaicML StreamingDataset: Fast, Accurate Streaming of Training Data from Cloud Storage
Loading your training data becomes an escalating challenge as datasets grow bigger in size and the number of nodes scales. We built StreamingDataset...
Blazingly Fast LLM Evaluation for In-Context Learning
With MosaicML you can now evaluate LLMs on in-context learning tasks (LAMBADA, HellaSwag, PIQA, and more) hundreds of times faster than other evaluation...
5x Faster Image Segmentation Training with MosaicML Recipes
Can't stop, won't stop. Earlier this year, we shared a new baseline for semantic segmentation (basically, classifying an image at the pixel level)...