Efficient Point in Polygon Joins via PySpark and BNG Geospatial Indexing
This is a collaborative post by Ordnance Survey, Microsoft and Databricks. We thank Charis Doidge, Senior Data Engineer, and Steve Kingston, Senior Data...
Pandas API on Upcoming Apache Spark™ 3.2
We're thrilled to announce that the pandas API will be part of the upcoming Apache Spark™ 3.2 release. pandas is a powerful, flexible...
Databricks SQL: Delivering a Production SQL Development Experience on the Data Lake
Databricks SQL is now generally available on AWS and Azure. Databricks SQL (DB SQL) is a simple and powerful SQL analytics platform for...
Shiny and Environments for R Notebooks
At Databricks, we want the Lakehouse ecosystem widely accessible to all data practitioners, and R is a great interface language for this purpose...
Catalog and Discover Your Databricks Notebooks Faster
This is a collaborative post from Databricks and Elsevier. We thank Darin McBeath, Director Disruptive Technologies -- Elsevier, for his contributions. As a...
Extracting Oncology Insights From Real-World Clinical Data With NLP
Preview the solution accelerator notebooks referenced in this blog online or get started right away by downloading and importing the notebooks into your...
Managing Model Ensembles With MLflow
In machine learning, an ensemble is a collection of diverse models that provide more predictive power together than any single model would on...
How YipitData Extracts Insights From Alternative Data Using Delta Lake
Get an early preview of O'Reilly's new ebook for the step-by-step guidance you need to start using Delta Lake. This is a guest...
Timeliness and Reliability in the Transmission of Regulatory Reports
Managing risk and regulatory compliance is an increasingly complex and costly endeavour. Regulatory change has increased 500% since the 2008 global financial crisis...
Large Scale ETL and Lakehouse Implementation at Asurion
This is a guest post from Tomasz Magdanski, Sr Director of Engineering, Asurion. With its insurance and installation, repair, replacement and 24/7 support...