Databricks and Informatica Accelerate Development and Complete Data Governance for Intelligent Data Pipelines
The value of analytics and machine learning to organizations is well understood. Our recent CIO survey showed that 90% of organizations are investing in analytics, machine learning and AI. But we’ve also noted that the biggest barrier is getting the right data in the right place and in the right format. So we’ve partnered with...
New videos from Databricks Academy: Introduction to Natural Language Processing—Latent Semantic Analysis
Databricks' commitment to education is at the center of the work we do. Through Instructor-Led Training, Certification, and Self-Paced Training, Databricks Academy provides strong pathways for users to learn Apache Spark™ and Databricks to push their knowledge to the next level. Our latest offering is a series of short videos introducing the Natural Language Processing...
How Tilting Point Does Streaming Ingestion into Delta Lake
Diego Link is VP of Engineering at Tilting Point Tilting Point is a new-generation games partner that provides top development studios with expert resources, services, and operational support to optimize high quality live games for success. Through its user acquisition fund and its world-class technology platform, Tilting Point funds and runs performance marketing management and...
AWS + Databricks – Developer Day Events
Every enterprise today wants to accelerate innovation by building AI into their business. However, most companies struggle with preparing large datasets for analytics, managing the proliferation of ML frameworks, and moving models in development to production. AWS and Databricks are presenting a series of Dev Day events where we will cover best practices for enterprises...
Advanced Analytics with HyperLogLog Functions in Apache Spark
This is a community guest blog from Sim Simeonov, the founder & CTO of Swoop and IPM.ai. Pre-aggregation is a common technique in the high-performance analytics toolbox. For example, 10 billion rows of website visitation data per hour may be reducible to 10 million rows of visit counts, aggregated by the superset of dimensions used...
Spark + AI Summit 2019 Product Announcements and Recap. Watch the keynote recordings today!
Spark + AI Summit 2019, the world’s largest data and machine learning conference for the Apache Spark™ Community, brought nearly 5000 registered data scientists, engineers, and business leaders to San Francisco's Moscone Center to find out what’s coming next. Watch the keynote recordings today and learn more about the latest product announcements for Apache Spark,...
Efficient Databricks Deployment Automation with Terraform
Managing cloud infrastructure and provisioning resources can be a headache that DevOps engineers are all too familiar with. Even the most capable cloud admins can get bogged down with managing a bewildering number of interconnected cloud resources - including data streams, storage, compute power, and analytics tools. Take, for example, the following scenario: a customer...
Detecting Financial Fraud at Scale with Decision Trees and MLflow on Databricks
Detecting fraudulent patterns at scale is a challenge, no matter the use case. The massive amounts of data to sift through, the complexity of the constantly evolving techniques, and the very small number of actual examples of fraudulent behavior are comparable to finding a needle in a haystack while not knowing what the needle looks...
Understanding Dynamic Time Warping
This blog is part 1 of our two-part series Using Dynamic Time Warping and MLflow to Detect Sales Trends. To go to part 2, go to Using Dynamic Time Warping and MLflow to Detect Sales Trends. The phrase “dynamic time warping,” at first read, might evoke images of Marty McFly driving his DeLorean at 88...
Using Dynamic Time Warping and MLflow to Detect Sales Trends
This blog is part 2 of our two-part series Using Dynamic Time Warping and MLflow to Detect Sales Trends. The phrase “dynamic time warping,” at first read, might evoke images of Marty McFly driving his DeLorean at 88 MPH in the Back to the Future series. Alas, dynamic time warping does not involve time travel; instead, it’s...
