Databricks Brings AI to the Enterprise using NVIDIA AI and Accelerated Computing

The world of artificial intelligence (AI) and data analytics is about to get a significant boost, thanks to Databricks’ collaboration with NVIDIA. This work brings together the cutting-edge capabilities of Databricks’ Mosaic AI platform and NVIDIA AI, enabling organizations to transform their large-scale AI and data workflows.

Unlocking the Potential of Mosaic AI

Databricks Mosaic AI empowers organizations to build and deploy high-quality Agent Systems on the data lakehouse. It enables secure customization of models with enterprise data, delivering accurate, domain-specific results. Mosaic AI connects seamlessly with open-source or commercial models, offering tools to evaluate and optimize outputs with a rapid development workflow. With robust governance across data and AI models, customers gain full visibility and control over their AI applications and outputs.

New Possibilities with NVIDIA Blackwell Platform

One of the key advancements in the Blackwell platform is the introduction of FP4 and FP6 precision formats, which can dramatically improve AI model serving performance while maintaining quality. Using FP4 precision can potentially double the performance compared to FP8, without compromising model accuracy. For large language model (LLM) inference, NVIDIA BlackwellGPUs will deliver up to 30x faster performance for language models compared to the previous-generation NVIDIA Hopper GPUs.

Databricks expects to be one of the first AI platforms to leverage the NVIDIA Blackwell platform.

“With the NVIDIA Blackwell platform, we plan to harness the benefits inherent in the next-generation architecture, including the second-generation Transformer Engine for increased inference performance and accuracy and the high-bandwidth memory compression engines. We will work closely with NVIDIA to accelerate new AI features on our platform, leading to exciting capabilities and performance gains for our customers,” said Naveen Rao, VP of AI at Databricks.

When combined with Databricks’ Mosaic AI platform, this powerful hardware unlocks new possibilities for enterprises, allowing them to tackle increasingly sophisticated AI challenges with greater speed and efficiency. With NVIDIA Blackwell platform computing and the Databricks Data Intelligence Platform, data scientists and engineers can push the boundaries of what’s possible in AI, from developing more accurate predictive models to creating more engaging and responsive generative AI applications.

Transforming Industries with AI and Data Analytics

Databricks’ collaboration with NVIDIA helps address key industry challenges. By powering Mosaic AI with NVIDIA accelerated computing, organizations can:

• Accelerate data processing: Databricks will use NVIDIA accelerated computing to speed up data processing, cleaning, and curation. This combination reduces the time it takes to prepare and transform large-scale datasets for model training.
• Train models faster: Mosaic AI’s integration with NVIDIA accelerated computing enables faster model training, allowing data teams to iterate and refine their models more quickly. With Mosaic AI, teams can also enhance AI model accuracy and reliability through techniques like retrieval-augmented generation (RAG).
• Optimize data pipelines: Mosaic AI’s automated data pipeline optimization capabilities help data teams identify and eliminate bottlenecks, ensuring that data flows efficiently and effectively through the pipeline.

This collaboration has the potential to redefine sectors such as healthcare, finance, and retail, where AI and data analytics are crucial for success. By optimizing each stage of the workflow—from data ingestion to AI model training and deployment—customers can leverage cutting-edge technology to uncover new insights, enhance decision-making, and foster innovation.

Databricks Open-Source Collaboration with NVIDIA

Databricks is also collaborating with NVIDIA to support, via Databricks Notebooks, many open-source software tools, such as NVIDIA NeMo for model development and customization, NVIDIA Morpheus for cybersecurity AI workflows, NVIDIA Triton Inference Server, and NVIDIA TensorRT-LLM for AI inference optimization, all part of the NVIDIA AI Enterprise software platform. NVIDIA AI Enterprise delivers enterprise-grade security, stability, and support to help ensure seamless transitions from pilot to production AI.

Key benefits include:

• Seamless integration: Leverage the power of Databricks notebooks.
• Scalable AI model development: Databricks’ collaboration with NVIDIA enables data teams to scale their AI model development, from small-scale experimentation to large-scale production environments.
• Faster time to market: Data teams can develop and deploy AI models more quickly, reducing the time it takes to bring new AI-powered products and services to market.

Key Benefits and Features for Enterprises

Seamless Deployment & Scalability

With Databricks, enterprises can deploy and scale their AI models with confidence, knowing that the platform is designed to handle large volumes of data and traffic. Databricks, with NVIDIA technologies, also enables enterprises to:

• Monitor and debug model performance with Lakehouse Monitoring, which provides auto-generated metrics and dashboards, anomaly detection, and automated root cause analysis
• Leverage the Databricks Feature Store and Vector Search to simplify the integration of features and embeddings into models
• Use MLflow, the leading open-source MLOps framework, to experiment with, build, evaluate, and deploy GenAI applications and systems

Integrated Security and Compliance

At Databricks, we take security and compliance seriously, working closely with customers in industries with high compliance needs. Our platform provides a range of security features, including:

• Enterprise-grade security and governance capabilities, such as access controls, auditing, and encryption
• Support for industry-specific compliance standards, such as HIPAA, PCI-DSS, and GDPR
• Data governance with Unity Catalog, which provides a centralized metadata management system for data and AI assets
• AI Gateway, which provides a standard query interface and centrally governs rate limits, tracks usage, enforces guardrails, and data audits model APIs
• Lakehouse Monitoring, which provides monitoring and debugging capabilities for all data and AI assets, from feature tables to AI foundation models

Real-World Use Cases and Industry Applications

Let’s examine a real-world example of how Databricks’ collaboration with NVIDIA can be applied to a customer service use case.

Example: Customer Service Agent

A company wants to build a customer service agent that can handle a large volume of customer inquiries. The agent needs to understand customer intent, provide personalized responses, and escalate complex issues to human customer support agents. It also needs to summarize prior conversations during escalations and handoff processes and log all interactions through an AI agent. To build this chatbot, the company uses Databricks Mosaic AI to develop and train an AI agent that can learn from customer interactions and adapt to changing customer needs.

In order to improve the quality of responses, the AI agent can be fine-tuned on a variety of company-specific data. With Mosaic AI Model Training, the company can do continuous pre-training and instruction fine-tuning methods with the same set of tools. Continued pretraining can teach the model relevant terms and concepts including information such as product offerings, website content, glossaries and frequently asked questions, while instruction fine-tuning will use formatted examples of customer input and agent responses derived from prior call transcripts to improve agent responses. Many companies find that fine-tuning avoids the need for RAG architectures and simplifies model deployments. The AI agent can also be integrated with the company’s customer service platform, allowing it to access customer data and history to provide personalized responses and log the interaction via an agentic workflow.

Once the AI agent is trained, the company uses Databricks Mosaic AI Model Serving to deploy it in a production environment with a few simple steps. The AI agent can handle a large volume of customer inquiries in real time, providing quick and accurate responses to customer questions and concerns.

The benefits of this solution include:

• Improved customer satisfaction: The agent is able to respond quickly and accurately to customer inquiries, improving customer satisfaction and reducing the need for human customer support.
• Increased efficiency: The agent is able to handle a large volume of customer inquiries, freeing up human customer support agents to focus on more complex issues that require empathy and human judgment.
• Reduced costs: The agent can reduce the need for human customer support agents, reducing costs and improving the bottom line.
• Enhanced customer experience: The AI agent can provide personalized responses and recommendations, enhancing the customer experience and increasing customer loyalty.

By using Databricks Mosaic AI and NVIDIA AI to develop and deploy an AI agent, companies can create more efficient, effective, and personalized customer service chatbots, leading to improved customer satisfaction and loyalty. Industry leaders such as Allianz, Chevron Phillips, Experian, Kubrick, Santalucía Seguros, UK Power Networks, and many others are leveraging Databricks to enhance their customer experience with innovative generative AI solutions.

Enterprise-Ready AI Solutions

With the combined capabilities of Databricks Mosaic AI and NVIDIA accelerated computing, organizations can unlock new insights, improve decision-making, drive business innovation, and confidently move AI systems to production. With this collaboration, users can expect new features and functionality, including expanded support for NVIDIA accelerated computing.

Join Databricks (booth 1820) and NVIDIA (booth 1620) at AWS re:Invent 2024 to learn more about how you can leverage Databricks and NVIDIA technologies to transform your business. Join us for an evening of fun at the NVIDIA-sponsored Databricks networking reception on Tuesday, December 3, from 6-10 pm, at the Venetian Grand Lux Cafe. Register here.