Databricks Announces Lakehouse Apps and Expands the Databricks Marketplace with AI Models and New Data Providers
June 20, 2023
Discover, deploy, and manage data and AI applications directly from Databricks
San Francisco, CA — June 20, 2023 — Databricks, the Data and AI company, today introduced Lakehouse Apps, a new way for developers to build native, secure applications for Databricks. Lakehouse Apps will enable over 10,000 Databricks customers to unlock the value of their data in the Lakehouse. Customers will have easy access to a wide range of powerful applications that run entirely inside their Lakehouse instance, using their data, with the full security and governance capabilities of Databricks.
The company also introduced new data sharing providers and AI model-sharing capabilities to the Databricks Marketplace — the only marketplace for data, AI, and applications — and announced that Databricks Marketplace will be generally available at the Data + AI Summit.
Lakehouse Apps Simplifies Access to Data and AI
Data and AI applications are among the fastest-growing software categories, and the growth in generative AI and large language models (LLMs) has accelerated that trend. For customers, Lakehouse Apps will be the most secure way to run applications that unlock the full value of data in their Lakehouse, leverage Databricks-native services, and extend Databricks with new capabilities. Lakehouse Apps will give users safe and easy access to a wide range of innovative new applications and reduce time and effort to adopt, integrate, and manage data and AI applications.
Lakehouse Apps Offer Security Without Compromise for Developers
To get next generation of innovative applications in the hands of users, software vendors must clear significant hurdles to securely access customer data, integrate with customers’ security and governance solutions, and efficiently run close to customer data. To secure enterprise adoption, many developers have taken one of two approaches: restrict the capabilities of their application and rebuild vital parts of their application in SQL or proprietary code from data platform vendors; or build versions of their products that customers have to install and operate themselves, which are fragile and hard to scale.
Lakehouse Apps helps developers overcome this dilemma with a native, secure, no-compromise solution. By running directly on a customer’s Databricks instance, these apps can easily and securely integrate with the customer’s data, use and extend Databricks services, and enable users to interact with a single sign-on experience — all without data ever leaving the customer’s instance. Lakehouse Apps inherit the same security, privacy, and compliance controls as Databricks. Developers can use any technology and language of their choice to build apps and aren’t limited to a proprietary framework.
Developers also benefit from easier distribution by listing their Lakehouse Apps in the Databricks Marketplace, enabling customers to quickly discover and deploy their software.
Early development partners for Lakehouse Apps include Retool, Posit, Kumo.ai, and Lamini:
- Retool enables customers to quickly build and deploy internal apps, powered by their data. Developers can assemble UIs with drag-and-drop building blocks like tables and forms, and write queries to interact with data using SQL and JavaScript.
- Posit is an open source data science company that empowers data professionals with cutting-edge tools for code-first data science.
- Kumo.ai is an AI-powered platform tackling predictive problems in business. Its platform works directly on relational data by using graph neural networks, a class of AI system for processing data that can be represented as a series of graphs.
- Lamini is an LLM platform for every developer to build customized, private models: easier, faster, and better-performing than any general-purpose LLM.
New AI Model-Sharing Capabilities and Data Providers
Databricks will also offer AI model sharing in the Databricks Marketplace, enabling data consumers and providers to discover and monetize AI models and integrate AI into all their data solutions. With AI model sharing, Databricks customers will have access to best-in-class models, which can be quickly and securely applied on top of their data. Databricks itself will curate and publish open source models across common use cases, such as instruction-following and text summarization, and optimize tuning or deploying of these models on Databricks.
Databricks Marketplace also welcomes new data providers, including financial services leaders such as S&P Global, Experian, London Stock Exchange Group, Nasdaq, Corelogic and YipitData; healthcare innovators like Datavant and IQVIA; geospatial leaders like Divirod, Accuweather and Safegraph; data collaboration companies like LiveRamp; and business information services companies like LexisNexis and ZoomInfo.
Quotes
“With Lakehouse Apps, software providers can offer their rich, secure apps within the lakehouse, which is exciting both for Databricks customers and for software vendors, greatly reducing the friction for applications to reach new customers,” said Matei Zaharia, Co-Founder and CTO at Databricks. “In addition, the expansion of Databricks Marketplace to cover AI models as well as apps satisfies a critical need in today’s business world, as collaboration between enterprises is evolving beyond the mere exchange of datasets, to secure computations and AI modeling on joint data.”
"At Edmunds, we provide car shoppers with a wealth of insights, which is why we rely so much on data ourselves. The Databricks Marketplace simplifies the process of discovering and evaluating external data with pre-built notebooks, without locking us into a single vendor or prolonging procurement cycles. We can access the data within our Databricks workspaces with just a few clicks. We also look forward to leveraging the notebooks, dashboards, and AI models through the Marketplace to enhance our analytics and AI initiatives,” said Greg Rokita, AVP of Technology at Edmunds, a Databricks Marketplace customer.
"Developers use Retool to build and deploy custom business software with an easy-to-use graphical user interface that sits on top of any data stack. Retool’s native integration with Databricks allows builders to create applications that get information into the hands of those that need it most — without compromising on speed, security, or reliability. Data engineers, data scientists, software developers and end users of data applications can now all benefit from our powerful offering listed on the Databricks Marketplace." - Mark Schaaf, COO at Retool, a development partner for Lakehouse Apps.
“In a partnership with Databricks, we are excited to offer R and Python users seamless access to their favorite authoring environments, such as RStudio, Jupyter, and VS Code. By bringing Posit Workbench to the Databricks Marketplace, we significantly lower the barrier for our joint customers, granting them secure data access while harnessing the security, reliability, scalability, and performance of the Databricks Lakehouse Platform. As we move forward, we eagerly anticipate supporting over 10,000 Databricks customers with the advanced publishing capabilities that our loyal Posit Connect users currently enjoy.” - Tareef Kawaf, President of Posit Software, PBC, a development partner for Lakehouse Apps.
“At Kumo.ai, we aim to make AI over your relational data easy using powerful graph learning to get predictions. Relational data has a natural graph structure, and graph approaches bring the benefits of deep representation learning that eliminates the need for feature engineering and training set generation. All that is needed is a simple SQL-like declarative query. We are excited about building on top of Databricks and hosting our app on Databricks, enabling us to connect with our customers’ data securely and would allow users to interact through a single-sign-on experience — all this while benefiting from the underlying scalability, security, and governance of the Databricks Lakehouse Platform.” - Vanja Josifovski, Co-Founder and CEO at Kumo.ai, a development partner for Lakehouse Apps.
“Lamini exists to make it incredibly easy for any developer to train powerful LLMs, specialized to their own data. With Lakehouse Apps, customers will be able to adopt and use Lamini just as easily: to discover and install Lamini with a few clicks, govern and secure usage with Unity Catalog, and train and serve LLMs on Databricks-managed auto-scaling infrastructure. The customer's data never has to leave their instance.” - Sharon Zhou, CEO at Lamini AI, a development partner for Lakehouse Apps.
"As the adoption of AI and advanced analytics grows, we are excited to partner with Databricks to help customers drive faster time to insights. This collaboration aims to empower businesses with real-time, unified data access, fostering collaboration and driving enhanced analytics and quantitative decision-making capabilities in the financial industry and beyond." - Stuart Brown, Group Head of Enterprise Data Solutions at London Stock Exchange, a data provider for Databricks Marketplace.
Availability
Databricks Marketplace will be generally available on June 28, 2023, coming out of public preview. Lakehouse Apps and AI model sharing in Databricks Marketplace are expected in preview in the coming year.
To learn more about Databricks Marketplace, register for the Data + AI Summit.
About Databricks
Databricks is the Data and AI company. More than 10,000 organizations worldwide — including Comcast, Condé Nast, and over 50% of the Fortune 500 — rely on the Databricks Lakehouse Platform to unify their data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe. Founded by the original creators of Delta Lake, Apache Spark™, and MLflow, Databricks is on a mission to help data teams solve the world’s toughest problems. To learn more, follow Databricks on Twitter, LinkedIn, and Facebook.
Contact: [email protected]
Safe Harbor Statement
This information is provided to outline Databricks’ general product direction and is for informational purposes only. Customers who purchase Databricks services should make their purchase decisions relying solely upon services, features, and functions that are currently available. Unreleased features or functionality described in forward-looking statements are subject to change at Databricks discretion and may not be delivered as planned or at all.