How (Not) to Tune Your Model With Hyperopt
Hyperopt is a powerful tool for tuning ML models with Apache Spark. Read on to learn how to define and execute (and debug)...
7 Reasons to Learn PyTorch on Databricks
What expedites the process of learning new concepts, languages or systems? When learning a new task, do you look for analogs from skills...
Benchmark: Koalas (PySpark) and Dask
Koalas is a data science library that implements the pandas APIs on top of Apache Spark so data scientists can use their favorite...
Fine-Grained Time Series Forecasting at Scale With Facebook Prophet and Apache Spark: Updated for Spark 3
Advances in time series forecasting are enabling retailers to generate more reliable demand forecasts. The challenge now is to produce these forecasts in...
Upgrade Production Workloads to Be Safer, Easier, and Faster With Databricks Runtime 7.3 LTS
What a difference a year makes. One year ago, Databricks Runtime version (DBR) 6.4 was released -- followed by 8 more DBR releases...
Analyzing Algorand Blockchain Data With Databricks Delta (Part 2)
Get an early preview of O'Reilly's new ebook for the step-by-step guidance you need to start using Delta Lake. This post was written...
Introducing Apache Spark™ 3.1
We are excited to announce the availability of Apache Spark 3.1 on Databricks as part of Databricks Runtime 8.0 . We want to...
Amplify Insights into Your Industry With Geospatial Analytics
Data science is becoming commonplace and most companies are leveraging analytics and business intelligence to help make data-driven business decisions. But are you...
Accelerating ML Experimentation in MLflow
This fall, I interned with the ML team, which is responsible for building the tools and services that make it easy to do...
How Data Lakehouses Solve Common Issues With Data Warehouses
Read Rise of the Data Lakehouse to explore why lakehouses are the data architecture of the future with the father of the data...