Databricks Survey Gets To The Heart of the AI Dilemma: Nearly 90% of Organizations Investing in AI, Very Few Succeeding

San Francisco – July 24, 2018 – Databricks, the leader in unified analytics and founded by the original creators of Apache Spark™, today announced the results of a commissioned survey looking at the AI dilemma. Today, only one in three AI projects are succeeding, and, perhaps more importantly, it is taking businesses more than six months to go from concept to production. The primary reasons behind these challenges are that 96 percent of organizations face data-related problems like silos and inconsistent datasets, and 80 percent cite significant organizational friction like lack of collaboration between data scientists and data engineers. IT executives point to unified analytics as a solution for these challenges with 90 percent of respondents saying the approach of unifying data science and data engineering across the machine learning lifecycle will conquer the AI dilemma.

“Data is the fuel that powers AI.  Large amounts of reliable data that data scientists can iterate on is the key to AI success. But organizational silos between data science and engineering cripples the iterative model development process. And to make matters worse the divide between today’s data and AI technologies increases complexity throughout the lifecycle further slowing down AI,” said Bharath Gowda, vice president of product marketing at Databricks. “Unified analytics addresses this AI dilemma by providing an end-to-end analytics platform that unifies big data and AI, while fostering better collaboration between data science and engineering teams.”

View the complete survey results within the “Conquer the AI Dilemma by Unifying Data Science and Engineering” report found here: https://databricks.com/CIO-survey-report

The survey, commissioned by Databricks through IDG’s CIO Research Services, surveyed 200 IT executives at larger companies (1000+ employees) across the U.S. and Europe. The results speak to the complexity and organizational confusion being creating as companies pursue AI initiatives:

• 98 percent of those surveyed believe preparation and aggregation of large datasets in a timely fashion is a major challenge;
• 96 percent of respondents found data exploration and iterative model training challenging;
• 90 percent cited the deployment of models to production quickly and reliably as a significant challenge
• 87 percent of organizations invest in an average of seven different machine learning tools, adding to the organizational complexity

So, what will help these organizations conquer the AI dilemma? The surveyed executives said they need end-to-end solutions that combine data processing with machine learning capabilities. These streamlined solutions would simplify workflows, improve efficiency and ultimately accelerate business value.

In fact, nearly 80 percent of executives surveyed said they highly valued the notion of a unified analytics platform. Unified analytics makes AI more achievable for enterprise organizations by unifying data processing and AI technologies. Unified analytics solutions provide collaboration capabilities for data scientists and data engineers to work effectively across the entire AI development-to-production lifecycle. With more than 90 percent of large companies facing data-related challenges and increasing complexity driven by an explosion of machine learning tools, the need for platforms and processes that can remove technology and organizational silos is more pronounced than ever. Unified analytics provides an ideal approach for companies facing modern AI implementation barriers.

Databricks accelerates innovation by unifying data science, engineering, and business. Through a fully managed, cloud-based service built by the original creators of Apache Spark, the Databricks Unified Analytics Platform lowers the barrier for enterprises to innovate with AI and accelerates their innovation.

About Databricks

Databricks’ mission is to accelerate innovation for its customers by unifying Data Science, Engineering and Business. Databricks’ founders started the Spark research project at UC Berkeley that later became Apache Spark. Databricks provides a Unified Analytics Platform powered by Apache Spark for data science teams to collaborate with data engineering and lines of business to build data products. Users achieve faster time-to-value with Databricks by creating analytic workflows that go from ETL and interactive exploration to production. The company also makes it easier for its users to focus on their data by providing a fully managed, scalable, and secure cloud infrastructure that reduces operational complexity and total cost of ownership. Databricks, venture-backed by Andreessen Horowitz, NEA and Battery Ventures, among others, has a global customer base that includes Viacom, Shell and HP. For more information, visit www.databricks.com.

Apache, Apache Spark and Spark are trademarks of the Apache Software Foundation.

###

MEDIA CONTACT

Kristalle Cooks

Head of Communications

415-462-4907 x1002

Kristalle.Cooks@databricks.com