Protecting the Securities Market with Predictive Fraud Detection
FINRA (Financial Industry Regulatory Authority), a regulatory body charged with protecting the U.S. securities market, spoke at the Spark + AI Summit on how they use Databricks Unified Analytics Platform to analyze up to a 100 billion stock market events per day for fraud detection and prevention. This is a summary of their story from Summit....
Efficient Databricks Deployment Automation with Terraform
Managing cloud infrastructure and provisioning resources can be a headache that DevOps engineers are all too familiar with. Even the most capable cloud admins can get bogged down with managing a bewildering number of interconnected cloud resources - including data streams, storage, compute power, and analytics tools. Take, for example, the following scenario: a customer...
Tangible Impacts of AI on the Business
With 2019 in full swing, the excitement for data and AI driven innovation continues. Over the past few years, we’ve seen leading innovators - like Riot Games, Regeneron and Shell - become early adopters of the latest machine learning and AI technologies, building and deploying AI applications into production. But with great promise comes great...
Challenges of AI in the Mainstream
Artificial Intelligence (AI) adoption is picking up steam. Enterprises are looking for new ways to deploy AI to transform their business and gain a competitive edge. With the power of machine learning (ML) fueled by the insights mined from massive volumes of data, AI is no longer only for the 1%, but quickly becoming an...
The Biggest AI Trends from 2019
The progress made in the field of machine learning and AI over the past several years has been tremendous. We’ve moved from science projects and edge use cases, to core businesses and competitive advantages, with more companies across various industries looking for opportunities to tap into their data and leverage AI. In 2018, we saw...
AI Trends that Paved the Way in 2018
With 2018 behind us, it’s been amazing to see AI projects gain steam and make significant impact across industries. In fact, a recent survey by CIO.com cites that 90% of enterprises are actively investing in AI. What has fueled this innovation is the massive influx of organizations tapping into the potential of their data and...
Apparate: Managing Libraries in Databricks with CI/CD
This is a guest blog from Hanna Torrence, Data Scientist at ShopRunner. Introduction As leveraging data becomes a more vital component of organizations' tech stacks, it becomes increasingly important for data teams to make use of software engineering best-practices. The Databricks platform provides excellent tools for exploratory Apache Spark workflows in notebooks as well as...
Open Sourcing Databricks Integration Tools at Edmunds
This is a guest post from Shaun Elliott, Data Engineering Tech Lead and Sam Shuster, Staff Engineer at Edmunds. What is Databricks and How is it Useful for Edmunds? Databricks is a cloud-based, fully managed, big data and analytics processing platform that leverages Apache SparkTM and the JVM. The big selling point of the Databricks Unified...
Introducing Flint: A time-series library for Apache Spark
This is a joint guest community blog by Li Jin at Two Sigma and Kevin Rasmussen at Databricks; they share how to use Flint with Apache Spark. Introduction The volume of data that data scientists face these days increases relentlessly, and we now find that a traditional, single-machine solution is no longer adequate to the demands...
Announcing Databricks Runtime 4.2!
We’re excited to announce Databricks Runtime 4.2, powered by Apache Spark™. Version 4.2 includes updated Spark internals, new features, and major performance upgrades to Databricks Delta, as well as general quality improvements to the platform. We are moving quickly toward the Databricks Delta general availability (GA) release and we recommend you upgrade to Databricks Runtime...
