Identifying Financial Fraud With Geospatial Clustering
For most financial service institutions (FSI), fraud prevention often implies a complex ecosystem made of various components –- a mixture of traditional rules-based controls and artificial intelligence (AI) and a patchwork of on-premises systems, proprietary frameworks and open source cloud technologies. Combined with strict regulatory requirements (such as model explainability), high governance frameworks, low latency...
Efficiently Building ML Models for Predictive Maintenance in the Oil and Gas Industry With Databricks
Guest authored post by Halliburton’s Varun Tyagi, Data Scientist, and Daili Zhang, Principal Data Scientist, as part of the Databricks Guest Blog Program Halliburton is an oil field services company with a 100-year-long proven track record of best-in-class oilfield offerings. With operations in over 70 countries, Halliburton provides services related to exploration, development and production...
Benchmark: Koalas (PySpark) and Dask
Koalas is a data science library that implements the pandas APIs on top of Apache Spark so data scientists can use their favorite APIs on datasets of all sizes. This blog post compares the performance of Dask’s implementation of the pandas API and Koalas on PySpark. Using a repeatable benchmark, we have found that Koalas...
Fine-Grained Time Series Forecasting at Scale With Facebook Prophet and Apache Spark: Updated for Spark 3
Advances in time series forecasting are enabling retailers to generate more reliable demand forecasts. The challenge now is to produce these forecasts in a timely manner and at a level of granularity that allows the business to make precise adjustments to product inventories. Leveraging Apache Spark™ and Facebook Prophet, more and more enterprises facing these...
Data + AI Summit Is Back
Data + AI Summit, the global event for the data community, returns May 24-28. We are thrilled to announce that registration for this free virtual event is now open! The future is open Data and AI are rapidly opening up new possibilities and solving stubborn problems. Unifying data opens up collaboration and solutions that were...
Data Democratization: A Key to Building a Healthy Data Culture
Building a thriving data culture is a strategic priority for many organizations, but only 24% of enterprises have managed to forge a data culture. What is a thriving data culture anyway? In its purest form, it’s when the entire organization - from the C-suite to the front-line workers - are making informed business decisions every...
Introducing Delta Time Travel for Future Data Sets
We are thrilled to introduce enhanced time travel capabilities in Databricks Delta, the next-gen unified analytics engine built on top of Apache Spark, for all of our users. With this new feature, Delta can automatically extrapolate big datasets stored in your data lake, enabling access to any future version of your data today. This temporal...
Top Questions from Customers About Data Management
Last week, we hosted a virtual event highlighting Delta Lake, an open source storage layer that brings reliability, performance and security to your data lake. We had amazing engagement from the audience, with almost 200 thoughtful questions submitted! While we can’t answer all in this blog, we thought we should share answers to some of...
Databricks Notebook Dark Theme
This became the most-requested feature in Databricks’ history, and now it’s here: a dark theme for the Databricks notebook! We’re excited for you to try it out. To turn it on, open a notebook and navigate to the View menu > Notebook Theme > Dark Theme. Enjoy!
Creating Growth and Advancement Opportunities: Introducing the Women in Tech Mentorship Program
At Databricks, we recognize the importance of offering professional growth and advancement opportunities for all communities and are committed to fostering a work environment where every employee can contribute the best work of their careers. This week, we’re wrapping up our celebration of Women’s History Month. In alignment with our commitment to employee growth, we...
