Build Your Next Apache Spark Job in .NET Using Mobius - Databricks

Build Your Next Apache Spark Job in .NET Using Mobius

Download Slides

Apache Spark offers the flexibility to build data processing jobs in any language that is natively supported in Spark (namely Scala, Java, Python or R) and it is one of the key factors driving its explosive growth. The latest addition to the external language bindings to Spark is Mobius which adds C# as another first-class Spark programming language. Mobius is an open source project that aims to bring Spark’s rich set of capabilities to the .NET community. With Mobius, developers can build Spark jobs in C# and reuse their existing .NET libraries with Apache Spark. Mobius is open-sourced at http://github.com/Microsoft/Mobius. In this talk, we will cover the following: (1) implementing Apache Spark jobs in C# and F# using Mobius (2) using RDD, DataFrame and Streaming API in Mobius (3) best practices using Mobius in production with a Spark cluster – in Windows or Linux, on-premises or in your favorite Cloud