What expedites the process of learning new concepts, languages or systems? When learning a new task, do you look for analogs from skills...

MLflow helps organizations manage the ML lifecycle through the ability to track experiment metrics, parameters, and artifacts, as well as deploy models to...

MLflow 1.12 features include extended PyTorch integration, SHAP model explainability, autologging MLflow entities for supported model flavors , and a number of UI...

In the last two years since its release, MLflow has seen a rapid adoption among enterprises and the data science community. With over...

Often, small things make a huge difference, hence the adage that "some of the best ideas are simple!" Over the course of a...

Technical conferences evolve over time. They expand beyond their initial focus, adding new technologies, attracting new attendees and broadening their range of sessions...

Developers attending a conference have high expectations: what knowledge gaps they’ll fill; what innovative ideas or inspirational thoughts they’ll take away; who to...

We are excited to announce new enterprise grade features for the MLflow Model Registry on Databricks. The Model Registry is now enabled by...

Over the years, technical conferences tend to expand beyond their initial focus, adding new technologies, types of attendees, and a broader range of...

Machine learning engineers and data scientists frequently train models to optimize a loss function. With optimization methods like gradient descent, we iteratively improve...

In February 2016, we introduced Databricks Community Edition , a free edition for big data developers to learn and get started quickly with...

Education and the pursuit of knowledge are lifelong journeys: they never complete; there is always something new to learn; a new professional certification...

You might have heard the famous saying, “Why software is eating the world .” But if software is eating the world, you may...

Read Rise of the Data Lakehouse to explore why lakehouses are the data architecture of the future with the father of the data...

In January 2013 when Stephen O’Grady, an analyst at RedMonk , published “The New Kingmakers: How Developers Conquered the World ,” the book’s...

To a good degree, this back-of-the-envelope flowchart, by Karen Hao of MIT Technology Review , charts to elucidate what constitutes the use of...

As part of the organizing committee and program chairs for this summit, we are delighted to share two achievements with you. First, we...

Last week we released MLflow v0.8.0 with multiple new features, including improved UI experience and support for deploying models directly via Docker containers...

Today, we are excited to introduce HorovodRunner in our Databricks Runtime 5.0 ML ! HorovodRunner provides a simple way to scale up your...

Today, we’re excited to announce MLflow v0.7.0 , released with new features, including a new MLflow R client API contributed by RStudio...

In part 2 of our series on MLflow blogs, we demonstrated how to use MLflow to track experiment results for a Keras network...

Today, we’re excited to announce MLflow v0.6.0 , released early in the week with new features. Now available on PyPI and Maven...

For much of Apache Spark’s history, its capacity to process data at scale and capability to unify disparate workloads has led Spark developers...

In the last blog post , we demonstrated the ease with which you can get started with MLflow , an open-source platform to...

Today, we’re excited to announce MLflow v0.5.0, MLflow v0.5.1, and MLflow v0.5.2, which were released last week with some new features. MLflow 0.5.2...

In October 2012, Harvard Business Review put a spotlight on the data science career with a dedicated issue and a catchy claim: Data...

On July 19, we held our monthly Bay Area Spark Meetup (BASM) at Databricks, HQ in San Francisco. At the Spark + AI...

Today, we’re excited to announce MLflow v0.3.0, which we released last week with some of the requested features from internal clients and open...

Within a couple of years of its release as an open-source machine learning and deep learning framework, TensorFlow has seen an amazing rate...

At Data + AI Summit in June, we announced MLflow , an open-source platform for the complete machine learning cycle. The platform’s philosophy...

London, as a financial center and cosmopolitan city, has its historical charm, cultural draw, and technical allure for everyone, whether you are an...

Apache Spark is tackling new frontiers through innovations by unifying new workloads. This enables developers to combine data and AI to develop intelligent...

By any measurement today, in the digital media, technical conferences and citations, or searches on Google trends , the frequency of terms like...

Within a couple of years of its release as an open-source machine learning and deep learning framework, TensorFlow has seen an amazing rate...

Apache Spark has become the de facto unified analytics engine for big data processing in a distributed environment. Yet we are seeing more...

Spark + AI Summit will be held in San Francisco on June 4-6, 2018. Check out the full agenda and get your ticket...

In collaboration with the local chapter of Women in Big Data Meetup and our continuing effort by Databricks diversity team to have more...

Early last month, we announced our agenda for Spark + AI Summit 2018 , with over 180 selected talks with 11 tracks and...

Today we are happy to announce the availability of Apache Spark 2.3.0 on Databricks as part of its Databricks Runtime 4.0. We want...

At Databricks we welcome the dawn of the New Year 2018 by reflecting on what we achieved collectively as a company and community...

When Fei-Fei Li , the director of Stanford’s AI Lab and now a chief scientist at Google Cloud, was asked in an interview...

Databricks and Microsoft have jointly developed a new cloud service called Microsoft Azure Databricks , which makes Apache Spark analytics fast, easy, and...

“Dublin is now a truly cosmopolitan capital, with an influx of people, energy, and ideas infusing the ever-beguiling, multi-layered city with fresh flavors...

This is the seventh post in a multi-part series about how you can perform complex streaming analytics using Apache Spark and Structured Streaming...

Introduction Big data practitioners often post recurring questions on Quora: What is data engineering? How to become a data scientist? What’s a data...

On September 7th, we held our monthly Bay Area Apache Spark Meetup (BASM) at HPE/Aruba Networks in Santa Clara. We had two Apache...

Since Apache Spark 1.3, Spark and its APIs have evolved to make them easier, faster, and smarter. The goal has been to unify...

On August 22, we held our monthly Bay Area Apache Spark Meetup (BASM) at Pinterest in San Francisco. In all, we had three...

Older anthologies collated a collection of contributions from various authors around a theme—bounded then as a journal or periodical. Newer anthologies, however, include...

At Databricks, we work with hundreds of companies, all pushing the bleeding edge in their respective industries. We want to share patterns for...

On August 15th, Data Science Central hosted a live webinar—Parallelize R Code Using Apache Spark—with Databricks’ Hossein Falaki . This webinar introduced SparkR...

On July 11, 2017, we announced the general availability of Apache Spark 2.2.0 as part of Databricks Runtime 3.0 (DBR) for the Unified...

On July 13th, we hosted a live webinar — Accelerate Data Science with Better Data Engineering on Databricks . This webinar focused on...

Read Rise of the Data Lakehouse to explore why lakehouses are the data architecture of the future with the father of the data...

Try this notebook on Databricks For developers, often the how is as important as the why . While our in-depth blog explains the...

We have assembled a selected collage of highlights from Databricks’ speakers at our 10th Spark Summit, a milestone for Apache Spark community and...

On May 16, we held our monthly Bay Area Apache Spark Meetup (BASM) at SalesforceIQ in Palo Alto. In all, we had three...

On May 4th, we hosted a live webinar — Deep Learning and Apache Spark: Workflows and Best Practices . Rather than comparing deep...

The number 10 is often used as a measuring yardstick to denote achievement, attainment or accomplishment: the 10th anniversary; a perfect score of...

On March 9th, we hosted a live webinar— Apache Spark MLlib 2.x: How to Productionize your Machine Learning Models —to address the following...

We’ve put together a short recap of the keynotes and highlights from Databricks’ speakers for Apache Spark enthusiasts who could not attend the...

Spark Summit East will be held in Boston on Feb 7-9, 2017. Check out the full agenda and get your ticket before it...

Spark Summit will be held in Boston on Feb 7-9, 2017. Check out the full agenda and get your ticket before it sells...

Spark Summit will be held in Dublin, Ireland on Oct 24-26, 2017. Check out the get your ticket before it sells out! Here’s...

Last week, we held a live webinar, Apache Spark MLlib 2.x: Migrating ML Workloads to DataFrames , to demonstrate the ease with which...

Spark Summit Talks and Apache Spark Roundup Databricks and partners set a new world record for CloudSort 2016 Benchmark using Apache Spark...

Read the recap from Day 1 of Spark Summit EU. Update: The videos of the presentations are now available. Find them below. Spark...

Update: The videos of the presentations are now available. Find them below. Spark Summit Keynotes Brussels’ October morning overcast or morning-commute traffic did...

Here’s our recap of what’s transpired with Apache Spark since our previous digest . Databricks Apache Spark Survey 2016 Report published and now...

In July 2016, we conducted our Apache Spark Survey to identify insights on how organizations are using Spark and highlight growth trends since...

Today, the Datanami Readers’ and Editors’ Choice Awards recognized the sweeping changes Apache Spark is bringing to the Big Data landscape with four...

Here’s our recap of what’s transpired with Apache Spark since our previous digest . Databricks CTO and Co-founder Matei Zaharia presented “Unifying big...

Generally, a session is an interaction between two or more entities. In computer parlance, its usage is prominent in the realm of networked...

Continuing with our bi-weekly digest series, here’s our recap of what’s transpired over the last two weeks with Apache Spark since our previous...

Try this notebook in Databricks For a speechwriter, JFK’s words “Ask not what your country can do for you. Ask what you can...

Today, we're kicking off a new series: the Databricks Bi-Weekly Digest. Our goal with this digest is to summarize Spark related content, compiled...

Of all the developers' delight, none is more attractive than a set of APIs that make developers productive, that is easy to use...

We are proud to introduce the Getting Started with Apache Spark on Databricks Guide . This step-by-step guide illustrates how to leverage the...

The Spark Summit Europe call for presentations is open, submit your idea today . Since our survey in 2015 , Apache Spark has...

This article was originally posted on KDnuggets The Spark Summit Europe call for presentations is open, submit your idea today As observed in...

The lure of San Francisco is indisputable as is its position as the preeminent high-tech hub. On day one of Spark Summit 2016...

Older anthologies collated a collection of contributions from various authors around a theme—bounded then as a journal or periodical. Newer anthologies include multiple...

Temples of Developer Knowledge “Developers are the new kingmakers,” wrote Stephen O’Grady in his book “ The New KingMakers: How Developers Conquered the...

On a rainy and foggy Saturday morning, April 30th, in McLean, VA., more than 275 Apache Spark enthusiasts, forsaking the comfort of Saturday...

Today, I joined Databricks, the company behind Apache Spark, as a Spark Community Evangelist. In the past, I've worked as an individual contributor...