Databricks and Jupyter: Announcing ipywidgets in the Databricks Notebook

Today, we are excited to announce a deeper integration between the Databricks Notebook and the ecosystem established by Project Jupyter, a leader in the scientific computing community that has been responsible for the definition of open standards and software for interactive computing. With the release of Databricks Runtime 11.0 (DBR 11.0), the Databricks Notebook now supports ipywidgets (a.k.a., Jupyter Widgets) and the foundational Python execution engine powering the Jupyter ecosystem, the IPython kernel.

At Databricks, we are committed to making the Lakehouse the ultimate destination for creating and sharing data insights. We want to make it as simple as possible for users of all backgrounds to turn the data in their Lakehouse into business value, and we believe a major part of this is enabling users to easily enrich their analyses and data assets with interactivity. Our integration of ipywidgets represents a big step toward realizing this vision, and we look forward to seeing what our users create with them!

ipywidgets

The ipywidgets package, included in DBR 11.0 as a public preview on AWS and Azure and coming to GCP with DBR 11.1, enables a user to add graphical controls to their notebooks to visualize and interact with data. For example, we can use ipywidgets' interact function to automatically construct a graphical user interface to explore how different inputs change its output.

Using the many components that come with ipywidgets (sliders, buttons, checkboxes, dropdowns, tabs, and more), you can build custom user interfaces to modify variables, execute code, and visualize results directly in your notebooks. This is just the beginning, however; the real power of ipywidgets is the framework it provides for building more complex controls and interactions. Now that the Databricks Notebook supports ipywidgets, you can also use more advanced widgets like the plotly charting widget and the ipyleaflet map widget that enable you to immersively visualize and interact with data by visually selecting data points or drawing regions on a map.

As an example, here is a notebook that uses ipyleaflet to visualize farmers market locations from a Databricks dataset.

Ipywidgets will become the recommended way to create interactive controls when using Python in the Databricks Notebook. The Databricks Notebook in DBR 11.0 brings to public preview support for the core ipywidget controls and the plotly, ipyleaflet, and ipyslickgrid custom widget packages. Note that when you are passing parameters into a notebook or into jobs, we still recommend using the Databricks widgets syntax.

You can find more examples in the Databricks documentation or the official ipywidgets documentation, and you can find a variety of advanced ipywidgets examples in the official directory of ipywidgets examples. We are excited to add support for more of these advanced widgets in the coming months. One we are especially excited about is bamboolib, and we will have more to say about it and its integration into the Databricks Notebook very soon.

IPython Kernel

As part of DBR 11.0, Databricks also adopts the IPython kernel execution engine for its notebooks, replacing the custom Python execution engine Databricks has used for many years. Using the IPython kernel more closely aligns the Databricks Notebook with the Jupyter standards and ecosystem, in particular powering ipywidgets in the Notebook, and we are excited to contribute improvements to the project.

Databricks supports Project Jupyter

As a company which was built on open source technologies and has established open source projects like mlflow and Delta Lake, Databricks understands the importance of healthy open source communities. This is why we have become a Project Jupyter institutional partner, sponsoring Jupyter (and ipywidgets) development, and it is why Databricks engineers contribute improvements and bugfixes to Jupyter projects. We are excited to grow our involvement in the Jupyter ecosystem and continue bringing its capabilities to users of the Databricks Notebook.

Try it out

To try out ipywidgets in the Databricks Notebook on either AWS or Azure, all you need to do is choose a compute resource running DBR 11.0 or greater and import the ipywidgets package. It will also be accessible on GCP with the release of DBR 11.1 or greater. See our documentation for more information and examples.

If you would like to see further Jupyter ecosystem features and widgets added to Databricks, please let us know!