Simplifying Streaming Stock Analysis using Delta Lake and Apache Spark: On-Demand Webinar and FAQ Now Available!
On June 13th, we hosted a live webinar — Simplifying Streaming Stock Analysis using Delta Lake and Apache Spark — with Junta Nakai, Industry Leader - Financial Services at Databricks, John O’Dwyer, Solution Architect at Databricks, and Denny Lee, Technical Product Marketing Manager at Databricks. This is the first webinar in a series of financial...
Detecting Bias with SHAP
StackOverflow's annual developer survey concluded earlier this year, and they have graciously published the (anonymized) 2019 results for analysis. They're a rich view into the experience of software developers around the world -- what's their favorite editor? how many years of experience? tabs or spaces? and crucially, salary. Software engineers' salaries are good, and sometimes...
Databricks Connect: Bringing the capabilities of hosted Apache Spark™ to applications and microservices
In this blog post we introduce Databricks Connect, a new library that allows you to leverage native Apache Spark APIs from any Notebook, IDE, or custom application. Overview Over the last several years, many custom application connectors have been written for Apache Spark. This includes tools like spark-submit, REST job servers, notebook gateways, and so...
Enhanced Hyperparameter Tuning and Optimized AWS Storage with Databricks Runtime 5.4 ML
We are excited to announce the release of Databricks Runtime 5.4 ML (Azure | AWS). This release includes two Public Preview features to improve data science productivity, optimized storage in AWS for developing distributed applications, and a number of Python library upgrades. To get started, you simply select the Databricks Runtime 5.4 ML from the...
Protecting the Securities Market with Predictive Fraud Detection
FINRA (Financial Industry Regulatory Authority), a regulatory body charged with protecting the U.S. securities market, spoke at the Spark + AI Summit on how they use Databricks Unified Analytics Platform to analyze up to a 100 billion stock market events per day for fraud detection and prevention. This is a summary of their story from Summit....
Introducing Databricks Runtime 5.4 with Conda (Beta)
We are excited to introduce a new runtime: Databricks Runtime 5.4 with Conda (Beta). This runtime uses Conda to manage Python libraries and environments. Many of our Python users prefer to manage their Python environments and libraries with Conda, which quickly is emerging as a standard. Conda takes a holistic approach to package management by...
Databricks is a Diamond Partner at Snowflake Summit
Here at Databricks, we are excited to participate in the first Snowflake Summit as a Diamond Partner. The event takes place June 3-6 at the Hilton San Francisco Union Square and is another great opportunity to share how Databricks and Snowflake have partnered together to provide: Massively scalable data pipelines. Pipelines running on Databricks can...
Databricks and Informatica Accelerate Development and Complete Data Governance for Intelligent Data Pipelines
The value of analytics and machine learning to organizations is well understood. Our recent CIO survey showed that 90% of organizations are investing in analytics, machine learning and AI. But we’ve also noted that the biggest barrier is getting the right data in the right place and in the right format. So we’ve partnered with...
New videos from Databricks Academy: Introduction to Natural Language Processing—Latent Semantic Analysis
Databricks' commitment to education is at the center of the work we do. Through Instructor-Led Training, Certification, and Self-Paced Training, Databricks Academy provides strong pathways for users to learn Apache Spark™ and Databricks to push their knowledge to the next level. Our latest offering is a series of short videos introducing the Natural Language Processing...
How Tilting Point Does Streaming Ingestion into Delta Lake
Diego Link is VP of Engineering at Tilting Point Tilting Point is a new-generation games partner that provides top development studios with expert resources, services, and operational support to optimize high quality live games for success. Through its user acquisition fund and its world-class technology platform, Tilting Point funds and runs performance marketing management and...
