Anthology of Technical Assets on Apache Spark's Structured Streaming
Older anthologies collated a collection of contributions from various authors around a theme—bounded then as a journal or periodical. Newer anthologies, however, include...
On-Demand Webinar and FAQ: Parallelize R Code Using Apache Spark
On August 15th, Data Science Central hosted a live webinar—Parallelize R Code Using Apache Spark—with Databricks’ Hossein Falaki . This webinar introduced SparkR...
Apache Spark’s Structured Streaming with Amazon Kinesis on Databricks
On July 11, 2017, we announced the general availability of Apache Spark 2.2.0 as part of Databricks Runtime 3.0 (DBR) for the Unified...
Breaking the “curse of dimensionality” in Genomics using “wide” Random Forests
This is a guest blog from members of CSIRO’s transformational bioinformatics team in Sydney, Australia. CSIRO, Australia’s government research agency, is in the...
Integrating Apache Airflow with Databricks
This blog post is part of our series of internal engineering blogs on Databricks platform, infrastructure management, integration, tooling, monitoring, and provisioning. Today...
Benchmarking Big Data SQL Platforms in the Cloud
For a deeper dive on these benchmarks, watch the webinar featuring Reynold Xin. Performance is often a key factor in choosing big data...
Introducing Apache Spark 2.2
Today we are happy to announce the availability of Apache Spark 2.2.0 on Databricks as part of the Databricks Runtime 3.0. This release...
Declarative Infrastructure with the Jsonnet Templating Language
This blog post is part of our series of internal engineering blogs on Databricks platform, infrastructure management, integration, tooling, monitoring, and provisioning. At...
Five Spark SQL Utility Functions to Extract and Explore Complex Data Types
Try this notebook on Databricks For developers, often the how is as important as the why . While our in-depth blog explains the...
A Vision for Making Deep Learning Simple
Try this notebook on Databricks When MapReduce was introduced 15 years ago, it showed the world a glimpse into the future. For the...