New Databricks Delta Features Simplify Data Pipelines
Continued Innovation and Expanded Availability for the Next-gen Unified Analytics Engine Databricks Delta the next generation unified analytics engine, built on top of Apache Spark, and aimed at helping data engineers build robust production data pipelines at scale is continuing to make strides. Already a powerful approach to building data pipelines, new capabilities and performance...
AI Trends that Paved the Way in 2018
With 2018 behind us, it’s been amazing to see AI projects gain steam and make significant impact across industries. In fact, a recent survey by CIO.com cites that 90% of enterprises are actively investing in AI. What has fueled this innovation is the massive influx of organizations tapping into the potential of their data and...
High-Performance Modern Data Warehousing with Azure Databricks and Azure SQL Data Warehouse
As organizations move to the cloud, the architecture for a Modern Data Warehouse (MDW) allows a new level of performance and scalability. A modern data warehouse enables bringing together data at any scale easily, and to get insights through analytical dashboards, operational reports, or advanced analytics. But what does an MDW look like? The following...
Announcing Databricks Runtime 5.2
We are excited to announce the release of Databricks Runtime 5.2 which introduces several new features including the following: Delta Time Travel Fast Parquet Import Databricks Advisor Let’s unpack each of these features in more detail: Delta Time Travel Time Travel, released as an Experimental feature, adds the ability to query a snapshot of a table...
Introducing Delta Time Travel for Large Scale Data Lakes
Data versioning for reproducing experiments, rolling back, and auditing data We are thrilled to introduce time travel capabilities in Databricks Delta, the next-gen unified analytics engine built on top of Apache Spark, for all of our users. With this new feature, Delta automatically versions the big data that you store in your data lake, and...
Introducing Databricks Library Utilities for Notebooks
Databricks has introduced a new feature, Library Utilities for Notebooks, as part of Databricks Runtime version 5.1. It allows you to install and manage Python dependencies from within a notebook. This provides several important benefits: Install libraries when and where they're needed, from within a notebook. This eliminates the need to globally install libraries on...
Announcing Databricks Runtime 5.1
We are excited to announce the release of Databricks Runtime 5.1. This release includes several new features, performance and quality improvements. We highly recommend upgrading to Databricks Runtime 5.1 to benefit from all of these. Release highlights: Azure Active Directory Credential Passthrough to ADLS Gen1 (Preview) 7% performance gain overall for Databricks Azure New in...
Introducing Databricks Runtime 5.1 for Machine Learning
Last week, we released Databricks Runtime 5.1 Beta for Machine Learning. As part of our commitment to provide developers with the latest deep learning frameworks, this release includes the best of these libraries. In particular, our PyTorch addition makes it simple for a developer to simply import the appropriate Python torch modules and start coding,...
CIO Survey: Top 3 Challenges Adopting AI and How to Overcome Them
We recently hosted the webinar — CIO Survey: Enterprise Challenges to AI and How to Overcome Them — featuring Jen Garofalo, Research Director at IDG, the parent company to CIO.com, and Pat McDonough, VP of Customer Success at Databricks. This webinar covered key findings from a recent CIO.com survey of 200 executives on the topic...
Databricks Shows off AWS Competencies at AWS re:Invent
Powering innovation on AWS in life sciences, public sector, machine learning, and big data analytics. Databricks, founded by the original creators of Apache SparkTM, is proud to be a Platinum sponsor at AWS re:Invent. Stop by booth #416 to grab some cool schwag and see demos of MLflow, TensorFlow, Apache Spark, and other technologies that...
