Adaptive Query Execution: Speeding Up Spark SQL at Runtime
Read Rise of the Data Lakehouse to explore why lakehouses are the data architecture of the future with the father of the data...
Announcing a New Redash Connector for Databricks
We’re happy to introduce a new, open source connector with Redash , a cloud-based SQL analytics service, to make it easy to query...
Automating away engineering on-call workflows at Databricks
A Summer of Self-healing This summer I interned with the Cloud Infrastructure team. The team is responsible for building scalable infrastructure to support...
Modernizing Risk Management Part 1: Streaming data-ingestion, rapid model development and Monte-Carlo Simulations at Scale
Part 2 of this accelerator here . Managing risk within the financial services , especially within the banking sector, has increased in complexity...
New Pandas UDFs and Python Type Hints in the Upcoming Release of Apache Spark 3.0
Pandas user-defined functions (UDFs) are one of the most significant enhancements in Apache Spark TM for data science. They bring many benefits, such...
Manage and Scale Machine Learning Models for IoT Devices
A common data science internet of things (IoT) use case involves training machine learning models on real-time data coming from an army of...
Schema Evolution in Merge Operations and Operational Metrics in Delta Lake
Get an early preview of O'Reilly's new ebook for the step-by-step guidance you need to start using Delta Lake. Try this notebook to...
Shrink Training Time and Cost Using NVIDIA GPU-Accelerated XGBoost and Apache Spark™ on Databricks
Guest Blog by Niranjan Nataraja and Karthikeyan Rajendran of Nvidia. Niranjan Nataraja is a lead data scientist at Nvidia and specializes in building...
A Convolutional Neural Network Implementation For Car Classification
Convolutional Neural Networks (CNN) are state-of-the-art Neural Network architectures that are primarily used for computer vision tasks. CNN can be applied to a...
Fighting Cyber Threats in the Public Sector with Scalable Analytics and AI
Join our webinar Real-time Threat Detection, Analytics and AI in the Public Sector to learn more and see a live demo. In 2019...