Skip to main content

In today's rapidly evolving technological landscape, the intersection of data and artificial intelligence (AI) has become a critical focus for organizations across industries. According to Foundry's recent CIO Tech Poll, IT leaders have overwhelmingly placed AI-enabled solutions at the top of their investment list for 2024, with only 8% expressing no interest in generative AI. This surge in AI prioritization underscores the need for a deeper understanding of how data and AI can work together to drive innovation and business value.

To explore this crucial topic, we recently hosted a panel discussion featuring industry experts from Informatica, Immuta, and Dataiku. The panel brought their unique perspectives from real-life customer scenarios on the need for high-quality data, new regulations, and laying the right data foundations for everyone. The conversation centered around two key facets:

  1. How technology providers are ensuring intelligence is built into platforms to address security, privacy, governance, and policy control.
  2. How to enable customers to lead AI initiatives effectively by leveraging the power of their own data.

Panel:
Robin Sutara - Field CTO, Databricks
Conor Jensen - Dataiku, Field CTO
Rik Tamm-Daniels - Informatica, GVP Ecosystem and Technology, Informatica
Chris Brown - Immuta, Public Sector CTO

Let's delve into the key insights shared by our esteemed panelists.

Leveraging AI for Data Management and Vice Versa

Rik Tamm-Daniels from Informatica highlighted the company's approach to integrating AI into data management processes:

  • Incorporating generative AI to accelerate data management
  • Simplifying user interactions with data using natural language interfaces within an intelligent data warehouse
  • Creating enterprise-grade AI applications that integrate not only public data and AI models but also first-party data

This approach emphasizes the importance of a strong data foundation in unlocking the full potential of AI technologies.

Ensuring Data Security and Access Control

Chris Brown from Immuta focused on the critical aspect of data security:

  • Identifying and understanding the location of sensitive data across various storage systems
  • Creating rules to ensure only authorized personnel have access to specific data sets
  • Implementing automated policies for data engineering and report creation

A customer success story shared by Chris illustrated how organizations are leveraging solutions from Dataiku, Immuta, and Databricks to enhance their data engineering capabilities while maintaining strict governance.

Democratizing Data Access and Usage

Conor Jensen from Dataiku emphasized the importance of making data accessible to everyone in an organization:

  • Enabling all employees to leverage data across various use cases
  • Addressing the challenge of limited access to valuable data within platforms
  • Striking a balance between ease of use and risk management in data access

The Role of Metadata in AI Applications

The panelists agreed on the critical role of metadata in integrating different data assets with Large Language Models (LLMs). Ensuring the right data is input into AI systems is crucial for generating reliable and trustworthy outputs, reinforcing the "garbage in, garbage out" principle.

Balancing Innovation and Risk

Robin Sutara from Databricks touched on the impact of generative AI on people, processes, and change management:

  • The need for alignment with business outcomes
  • The distinction between data governance and AI governance
  • The importance of bringing in the right tools for data and AI governance

She concluded with a powerful message: "Build for the future, but don't wait for the future." The combined solutions from Databricks, Immuta, Informatica, and Dataiku empower organizations to adapt to evolving technologies without constant rebuilding, ensuring continuous empowerment of data consumers.

Conclusion

As organizations continue to navigate the complex landscape of data and AI, insights from industry leaders become invaluable. By focusing on strong data foundations, breaking down data silos, robust security measures, and accessible yet governed data practices, businesses can harness the full potential of AI while mitigating associated risks.

To gain deeper insights into this fascinating discussion, we encourage you to watch the full panel discussion.

Try Databricks for free

Related posts

Announcing General Availability of Lakehouse Federation

Today, we are excited to announce that Lakehouse Federation in Unity Catalog is now Generally Available (GA) across AWS, Azure, and GCP! Lakehouse...

Apache Spark Key Terms, Explained

June 22, 2016 by Jules Damji and Denny Lee in
This article was originally posted on KDnuggets The Spark Summit Europe call for presentations is open, submit your idea today As observed in...

Databricks SQL Serverless is now available on Google Cloud Platform

Today, we are thrilled to announce that Databricks SQL Serverless is now Generally Available on Google Cloud Platform (GCP)! As a key component...
See all Data Strategy posts