Shell Oil Use Case: Parallelizing Large Simulations with Apache SparkR on Databricks
This blog post is a joint engineering effort between Shell’s Data Science Team (Wayne W. Jones and Dennis Vallinga) and Databricks (Hossein Falaki). Introduction Apache Spark 2.0 introduced a new family of APIs in SparkR, the R interface to Apache Spark to enable users to parallelize existing R functions. The new dapply, gapply and spark.lapply...
