Deploying dbt on Databricks Just Got Even Simpler
At Databricks, nothing makes us happier than making our users more productive, which is why we are delighted to announce a native adapter…
Python Autocomplete Improvements for Databricks Notebooks
At Databricks, we strive to provide a world-class development experience for data scientists and engineers, and new features are constantly getting added to…
Interoperability between Koalas and Apache Spark
Koalas is an open source project which provides a drop-in replacement for pandas, enabling efficient scaling out to hundreds of worker nodes for…
Introducing Koalas 1.0
Koalas was first introduced last year to provide data scientists using pandas with a way to scale their existing big data workloads by…
10 Minutes from pandas to Koalas on Apache Spark
This is a guest community post from Haejoon Lee, a software engineer at Mobigen in South Korea and a Koalas contributor. pandas is…
Introducing New Built-in and Higher-Order Functions for Complex Data Types in Apache Spark 2.4
Try this notebook in Databricks Apache Spark 2.4 introduces 29 new built-in functions for manipulating complex types (for example, array type), including higher-order…
