Reynold Xin - Databricks

Reynold Xin

Co-founder & Chief Architect
Top Contributor & Original Creator of Apache Spark, Databricks

Reynold is an Apache Spark PMC member and the top contributor to the project. He initiated and led efforts such as DataFrames and Project Tungsten. He is also a cofounder and Chief Architect at Databricks.


1. New Developments in the Open Source Ecosystem: Apache Spark 3.0 and KoalasSummit Europe 2019

In this talk, we will highlight major efforts happening in the Spark ecosystem. In particular, we will dive into the details of adaptive and static query optimizations in Spark 3.0 to make Spark easier to use and faster to run. We will also demonstrate how new features in Koalas, an open source library that provides Pandas-like API on top of Spark, helps data scientists gain insights from their data quicker.


Official Announcement of Koalas Open Source ProjectSummit 2019

Keynote from Spark + AI Summit 2019: Reynold Xin, Databricks, Brooke Wenig, Databricks

Project Hydrogen: Unifying State-of-the-art AI and Big Data in Apache SparkSummit 2018

Big data and AI are joined at the hip: the best AI applications require massive amounts of constantly updated training data to build state-of-the-art models AI has always been on of the most exciting applications of big data and Apache Spark. Increasingly Spark users want to integrate Spark with distributed deep learning and machine learning frameworks built for state-of-the-art training. This talk introduces a new project that substantially improves the performance and fault-recovery of distributed deep learning and machine learning frameworks on Spark.