Productionize and Automate your Data Platform at Scale
Data-driven innovation is no longer optional to stay competitive in today's marketplace. Companies that can bring data, analytics, and ML-based products to market first will quickly beat their competition. While many companies have streamlined CI/CD (continuous integration and delivery) processes for application development, very few have well-defined processes for developing data and ML products. That’s...
Delivering and Managing a Cloud-Scale Enterprise Data Platform with Ease
Data is growing exponentially and organizations are building products to harness the data and provide services to their customers. However, this exponential growth cannot be sustained by an exponential growth in infrastructure spend or human capital cost. Today, there are over a hundred services available in each of the major clouds (AWS, Azure) that can...
Security that Unblocks the True Potential of your Data Lake
Over the last few years, Databricks has gained a lot of experience deploying data analytics at scale in the enterprise. In many cases, our customers have thousands of people using our product across different business units for a variety of different use cases —all of which involve accessing various data classifications from private and sensitive...
Developing Shiny Applications in Databricks
Join our live webinar hosted by Data Science Central on March 12 to learn more We are excited to announce that you can now develop and test Shiny applications in Databricks! Inside the RStudio Server hosted on Databricks clusters, you can now import the Shiny package and interactively develop Shiny applications. Once completed, you can...
Connect 90+ Data Sources to Your Data Lake with Azure Databricks and Azure Data Factory
Data lakes enable organizations to consistently deliver value and insight through secure and timely access to a wide variety of data sources. The first step on that journey is to orchestrate and automate ingestion with robust data pipelines. As data volume, variety, and velocity rapidly increase, there is a greater need for reliable and secure...
A Look into the Mid Market Sales Team
At Databricks, we are passionate about helping data teams solve the world’s toughest problems. Databricks helps organizations innovate faster and tackle challenges like treating chronic disease through faster drug discovery, improving energy efficiency, and protecting financial markets. With the help of our sales team, we are able to connect thousands of global customers to Databricks'...
Data Quality Monitoring on Streaming Data Using Spark Streaming and Delta Lake
Try this notebook to reproduce the steps outlined below In the era of accelerating everything, streaming data is no longer an outlier- instead, it is becoming the norm. We often no longer hear customers ask, "can I stream this data?" so much as "how fast can I stream this data?", and the pervasiveness of technologies...
Securely Accessing Azure Data Sources from Azure Databricks
Azure Databricks is a Unified Data Analytics Platform that is a part of the Microsoft Azure Cloud. Built upon the foundations of Delta Lake, MLFlow , Koalas and Apache Spark, Azure Databricks is a first party service on Microsoft Azure cloud that provides one-click setup, native integrations with other Azure services, interactive workspace, and enterprise-grade...
Check out the killer lineup of keynotes at Spark + AI Summit 2020
The Spark + AI Summit is already the world’s largest data and machine learning conference bringing together engineers, scientists, developers, analysts and leaders from around the world. This year is shaping up to be our biggest conference ever, with over 7,000 attendees expected to attend four days of training sessions, presentations and networking events. We’ve...
I Joined Databricks to Make Data Science a Little Less Scary
Big data and AI has always struck me as useful, but slightly scary. For example, it’s useful when Waze uses big data to help me outsmart a traffic jam. On the other hand, big data’s ad-targeting is so powerful that millions of people are scared that their smartphones are eavesdropping on them. Why Big Data...