Developing next-gen data analytics tools with Databricks Mosaic AI

Recognizing a need for GenAI integration

zeb recently launched their SuperInsight self-service reporting engine, built with Databricks Mosaic AI tools. Part of their SuperDesk suite of AI-powered service desk products that integrate with existing platforms like ServiceNow and Jira, the idea arose from growing demand from customers who were also using the Databricks Data Intelligence Platform. These customers sought a way to interact with their data more efficiently and reduce the workload on data analysts who manually handle numerous requests. zeb initially developed a generative AI–based system for a large enterprise logistics company, where data plays a critical role. Prior to the implementation, a team of several data analysts managed a substantial backlog of data requests.

Rather than manually processing data requests, the system transformed user requests into actionable insights, significantly reducing the workload on data analysts by up to 80–90%. It seamlessly integrated with the customer’s existing workflow, such as Slack channels, where data requests were already managed. The success of this deployment led to further development and the creation of SuperInsight. According to Sid Vivek, Head of AI at zeb, “The goal with SuperInsight is to have a product built 100% on the Databricks Data Intelligence Platform that takes user requests from Slack, Teams or whatever communication platform the customer is using and seamlessly integrates with their existing Databricks data warehouse.” Users can ask questions and make forecasts based on their internal data, and the system can even generate ML models to fulfill forecasting requests.

Developing a solution to support multiple industries

Although the initial inspiration for SuperInsight came from working with the logistics and supply chain industry, zeb wanted to develop a product that could also support retail, fintech, and health and life sciences deployments. Explained Mal Vivek, CEO at zeb, “Our goal with SuperInsight is to make all of this data much more accessible to smaller operational teams, and at a price point that will give them real value and real results very quickly.” They decided to apply a compound AI system that could leverage internal canonical data models aligned with those four different industries.

zeb went all in on the Databricks Platform to develop SuperInsight. They applied Databricks Mosaic AI Agent Framework for retrieval-augmented generation (RAG) and Mosaic AI Training for fine-tuning, with data and Model Serving endpoints federated and secured by Unity Catalog. Sid Vivek explained why they chose this compound solution, “With fine-tuning, we’re trying to change the behavior and scope of the model itself and put it within the context understanding of a specific industry. With RAG, we assume that the model is already trained for an industry, but now we’re trying to understand a particular organization’s data schema and context.”

For SuperInsight, zeb chose the open source DBRX model from Databricks for both fine-tuning and RAG. According to Sid Vivek, “The reason we chose DBRX is because we saw its ability to take instruction-based fine-tuning with reduced latency. The DBRX Mixture-of-Experts (MoE) architecture is really unique and able to deliver accurate results extremely quickly.”

The end result: zeb was able to easily develop an advanced compound AI system architecture running securely within the Databricks Data Intelligence Platform. End users can send a request through email, Slack or other communication channels. The request is fed through a DBRX model that classifies the intent, and then Databricks Vector Search provides relevant context from a knowledge base stored in Unity Catalog. The Model Serving endpoint combines another DBRX model with a fine-tuned adapter based on the customer’s industry. The final output is then either sent to a data warehouse to generate a CSV file, to a deployed AutoML endpoint or to a reporting tool to generate a visual report.

Giving customers faster access to insights

The end-to-end GenAI capabilities of the Databricks Data Intelligence Platform were a huge benefit to the zeb team in building SuperInsight. Stated Sid Vivek, “We can confidently say we reduced the time to develop our solution by 40% using the Databricks Platform. The customer’s data is in Databricks. Our fine-tuning, model serving, security and federation are all consolidated within Databricks. If we didn’t use the Databricks Data Intelligence Platform, we would have 30 different solutions to piece together.”

zeb’s customers also gain from this integration. Claimed Sid Vivek, “By using SuperInsight to augment the work of their data analyst teams, customers are seeing a 40% cost savings while their data analysts are able to focus on more critical tasks.” Another beneficial outcome was that more end users began interacting with their data to gain insights as they realized the effectiveness of the tool. “We actually saw a 72% uptake in the number of reports requested from our generative AI solution vs. the previous manual process,” explained Sid Vivek.

Although zeb developed their initial version of SuperInsight in just a few months, they’re not taking time off. According to Sid Vivek, “We’re constantly trying to improve as we do more implementations. We’re also getting more data understanding and more knowledge that we can feed back into our canonical data models.” The collaboration with Databricks has been hugely beneficial to zeb — and their customers. Concluded Mal Zivek, “The Databricks Platform and DBRX have enabled us to democratize access to data for every organization and every user, regardless of where they are in their data journey.”