Databricks + MosaicML

Today, we’re excited to share that we’ve completed our acquisition of MosaicML, a leading platform for creating and customizing generative AI models for your enterprise. Since the founding of Databricks, our mission has been to democratize data and AI for every enterprise. Generative AI will power the next wave of enterprise data applications, and together with MosaicML, we plan to provide the best-in-class experience for training, customizing, and deploying generative AI applications.

We will work together with the MosaicML team to accelerate what we view as the three most important developments required to move generative AI into the mainstream for enterprises:

Rapid democratization of model capabilities. We believe in a future where models are widely available to every company. Democratization of any technology requires reduced prices and increased access. The MosaicML team has laser focused on hardware and software efficiency improvements to drastically reduce the price of training and customizing large language models. We will continue to work together to drive training and serving costs down in order to provide high quality modeling capabilities to a larger and larger portion of the market. Democratization also requires access to techniques. We will work with MosaicML’s research team to continue to innovate, in the open, on modeling architectures like those which powered the popular MPT-7B and MPT-30B base LLMs. As of today, MPT-7B has been downloaded over 3 million times, making it the most downloaded open LLM ever.

Making generative AI models work for enterprises. We’ve all become familiar with general purpose chat bots. Like consumer chat bots, enterprise AI applications also benefit from reasoning abilities and language-based interfaces. But unlike general chat bots, enterprise applications have many additional requirements. They must incorporate the large volumes of custom data: information about business processes, customers, accounts, orders, or other aspects of their business. They must ensure the privacy and safety of their users and their data. Enterprise applications also have little tolerance for hallucinations or incorrect responses. Databricks and MosaicML together will make it much easier for enterprises to incorporate their own data to deploy safe, secure, and effective AI applications.

Unifying the AI and data stack. Every part of the model development life cycle requires good data. Enterprises will differentiate from competitors by using proprietary data that allows them to create better models and more intelligent applications. With MosaicML, Databricks will continue to put data at the center of the AI journey. This includes upstream data preparation like cleaning, featurization, and embedding of data for use in models. It also includes using data to make models better, in a positive reinforcement loop, by learning from generated data how models are performing, and teaching models based on prior generated results. In every stage of the machine learning lifecycle, the data and models must be jointly curated in order to build the best applications. This is even more important for generative models, where quality and safety depend so much on good training data.

MosaicML’s product, which enables companies to efficiently build large AI models on their own data and business processes, will remain available for direct purchase. Going forward, we will integrate MosaicML tightly with the Lakehouse AI Platform - bringing the training stack closer to the customer data and other capabilities of Lakehouse AI. 

We are excited about the future of Databricks and MosaicML and how we can help our customers with their generative AI ambitions.

Sign up here to stay updated on Databricks’ latest AI innovations, including updates regarding the MosaicML.