Company Blog › Product

Over the last decade, security incident and event management tools (SIEMs) have become a standard in enterprise security operations. SIEMs have always had their detractors. But the explosion of cloud footprints is prompting the question, are SIEMs the right strategy in the cloud-scale world? Security leaders from HSBC don’t think so. In a recent talk,...

Apache Hadoop was created more than 15 years ago as an open source, distributed storage and compute platform designed for large data sets and large-scale batch processing. Early on, it was cheaper than traditional data storage solutions. At the time, businesses didn't need to run it on particular hardware. The Hadoop ecosystem also consists of...

Leveraging data (internal and external) and customer analytics to innovate and create competitive advantages is more powerful than it has ever been. This popular practice is fueled by the growing volume of operational and customer data and technological advancements that make extracting value from data even faster and more accessible. Driving more value from all...

A single patient produces approximately 80 megabytes of medical data every year. Multiply that across thousands of patients over their lifetime, and you’re looking at petabytes of patient data that contains valuable insights. Unlocking these insights can help streamline clinical operations, accelerate drug R&D and improve patient health outcomes. But first, the data needs to...

This is a guest authored post by Madhu Kotian, Vice President of Engineering (Investment Products Data, CRM, Apps and Reporting) at Northwestern Mutual.   Digital Transformation has been front and center in most contemporary big data corporate initiatives, especially in companies with a heavy legacy footprint. One of the underpinning components in digital transformation is...

The internal logging infrastructure at Databricks has evolved over the years and we have learned a few lessons along the way about how to maintain a highly available log pipeline across multiple clouds and geographies. This blog will give you some insight as to how we collect and administer real-time metrics using our Lakehouse platform,...

Fraud is a costly and growing problem – research estimates that $1 of fraud costs companies 3.36x in chargeback, replacement and operational cost. Adding to the pain, according to experts, there are not enough regulations to protect small businesses from chargebacks and losses from fraud. Despite significant advancements in credit card fraud, risk management techniques...

READ DOCUMENTATION As companies undertake more business intelligence (BI) and artificial intelligence (AI) initiatives, the need for simple, clear and reliable orchestration of data processing tasks has increased. Previously, Databricks customers had to choose whether to run these tasks all in one notebook or use another workflow tool and add to the overall complexity of...

The global health crisis accelerated the adoption of omnichannel shopping and fulfillment.  Consumers spent $861.12 billion online with US merchants in 2020, up an incredible 44% compared to the previous year, which marks the highest annual growth in U.S. e-commerce in at least two decades. To keep up pace with this shift and more effectively...

Inspiration doesn’t always come from our peers. Some of the best ideas come from innovators in other industries.This is especially true when it comes to data science and AI/ML innovations being applied to business processes and all aspects of operational decision-making. Consider fraud: Who knows how to analyze millions of data points in real-time better...