Azure Databricks at Databricks Data + AI Summit 2024 featuring Industry Leaders and Pioneers

This is a collaborative post from Databricks and Microsoft. We thank Mohini Verma, Senior Product Marketing Manager, for her contributions.

Data + AI Summit 2024: Register now to join this in-person and virtual event June 10-13 and learn from the global data community.

Microsoft is a Legend Sponsor of the Databricks Data + AI Summit 2024, the premier event for the global data community. Join us to learn how data intelligence enables every organization to harness the power of generative AI on their own data. Hear from Microsoft leaders who will share how customers have successfully leveraged the Databricks Data Intelligence Platform on Azure for their business, integrating data, AI, and analytics on one common platform.

At Data + AI Summit, attendees will have the opportunity to hear from Microsoft data leaders sharing their insights on Tuesday, June 11th, Wednesday, June 12th, and Thursday June 13th during Microsoft, customer, and partner-led Breakout Sessions. Come listen and learn about the latest innovations and technologies, and hear thought-provoking discussions along with the ability for networking opportunities where you can connect with other data professionals in your industry.

The sessions below are a guide for everyone interested in Azure Databricks and span a range of topics—from scaling real-time data processing in healthcare to implementing confidential computing and leveraging multi-cloud analytics strategies. If you have questions about Azure Databricks or service integrations, connect with Azure Databricks specialists at Data + AI Summit at the Microsoft booth #119 on the Expo floor. 

Microsoft Customer Breakout Sessions

Scaling Real-Time Healthcare Data Processing for the Veterans Affairs
Wednesday, June 12, 2024, 5:10 PM - 5:50 PM

The Department of Veterans Affairs (VA), the U.S.’s largest health care system, supports over 9 million veterans across 172 medical centers and 1,200 clinics. VA averages  40-60 million records of daily patient transactions. The Electronic Health Record Modernization Data Syndication initiative aims to migrate VA data to the cloud with improved data accessibility and analysis capabilities. Central to the initiative’s success is the use of Azure Databricks and its Lakehouse architecture. The project features robust pipelines that ingest hundreds of terabytes of historical data into ADLS and employs structured streaming for real-time incremental data processing of 1,000+ tables, refreshing every 5 seconds. This streaming data is then shared with downstream users to support care delivery use cases. Significant optimization strategies such as Change Data Feed, Predictive IO, and Photon have reduced ETL time by over 85%, empowering the VA to deliver agile and responsive care to veterans.

Speakers:

• Kash Sabba, Sr. Consultant, Microsoft
• Spencer Schaefer, Chief AI Officer VISN 15, U.S. Department of Veteran Affairs

Microsoft-Led Breakout Sessions

Powering Scalable Analytics and AI with Azure Data Lake Storage
Tuesday, June 11, 2024 9:00 AM - 9:40 AM PM

Conventional AI models predict outcomes by analyzing data, while Generative AI (Gen AI) and large language models (LLMs) generate entirely new outputs. Scaling model building and inferencing pose challenges in maintaining precision, governance, and security for customer-facing applications. Azure Storage offers scalable, secure cloud storage that integrates with industry leading analytics engines such as Azure Databricks and Microsoft Fabric to provide a unified analytics platform for data engineering, data science, and machine learning.

Join us for an in-depth session, we'll delve into the underlying storage architecture that enables hyper-scale workloads to utilize Azure Data Lake Storage (ADLS) with analytics engines of their choice for tasks like data cleansing and curation within Spark pipelines for Big Data Analytics. You will discover how this curated data can then be accessed by data science and business intelligence teams and can be provided to machine learning teams for further training of GPT-X models. We will showcase architectural patterns, real-world workload performance, and storage behavioral characteristics of workloads running at scale on Azure Storage today.

Speakers:

• Jeff King, Principal Program Manager, Microsoft
• Saurabh Sensharma, Principal Product Manager, Azure Storage, Microsoft

Confidential Computing in Azure Databricks
Thursday, June 13, 2024, 12:30 PM - 1:10 PM

Join us for an insightful session as we explore the powerful combination of Azure Databricks and Confidential Computing. Azure Databricks offers a collaborative workspace tailored for data teams, seamlessly integrating with Azure infrastructure services, and facilitating the entire data lifecycle. Moreover, our discussion will highlight the pivotal role of Confidential Computing in safeguarding sensitive data within hardware-based trusted execution environments, ensuring data integrity and privacy. We will explore the cutting-edge features of Confidential VMs, which enhance guest protection without necessitating changes to application code, all powered by AMD SEV-SNP technology. Finally, get an exclusive sneak peek into the roadmap ahead, including upcoming support for Intel Confidential VMs on Azure Databricks and expansion plans for Confidential VM availability across various regions. 

Speakers:

• Lindsey Allen, GM Azure Databricks, Microsoft

Partner-Led Breakout Sessions

Sponsored by: KPMG | Best of Both Worlds: Microsoft Fabric & Databricks for Multi-Cloud Analytics

Thursday, June 13, 2024, 11:20 AM - 12:00 PM

This session will discuss how enterprises can leverage the strengths of Microsoft Fabric and Databricks to achieve the best by combining the "Multi-Cloud" and "Any Cloud" analytics strategies. Fabric provides a centralized data capability across on-premises and cloud sources while benefiting from the full-features of Microsoft 365, the co-pilot ecosystem, and OpenAI. Databricks offers cloud-agnostic "compute anywhere" data processing and model building: ensuring compute stays within a specific environment, avoiding vendor lock-in, while providing a secure model IP development platform. We will explore how critical capabilities of Delta Sharing and Unity Catalog enable governance of data, ML models, and GenAI across this environment and discuss first-hand learnings from harmonizing these platforms at KPMG.

Speakers:

• Sreekar Krishna, Principal, Advisory, KPMG LLP
• Tom Haslam, Principal, KPMG

We also invite you to visit the Microsoft booth on the Expo floor, where you'll get to talk 1:1 with Microsoft Azure specialists on how to address your critical business priority use cases with Azure Databricks.

Register now to join this event and become part of the data and AI community. Learn how companies are building their lakehouse architecture with Azure Databricks, creating a unified, open, and scalable data platform. Get started with Azure Databricks with a free trial.

Join the Data + AI community at the 2024 Summit and unlock the power of data, analytics, and AI with Azure Databricks.

For more details on the sessions and speakers, visit the Data + AI Summit 2024 homepage.