Simplifying Genomics Pipelines at Scale with Databricks Delta
This blog is the first blog in our “Genomics Analysis at Scale” series. In this series, we will demonstrate how the Databricks Unified Analytics Platform for Genomics enables customers to analyze population-scale genomic data. Starting from the output of our genomics pipeline, this series will provide a tutorial on using Databricks to run sample quality...
High-Performance Modern Data Warehousing with Azure Databricks and Azure SQL Data Warehouse
As organizations move to the cloud, the architecture for a Modern Data Warehouse (MDW) allows a new level of performance and scalability. A modern data warehouse enables bringing together data at any scale easily, and to get insights through analytical dashboards, operational reports, or advanced analytics. But what does an MDW look like? The following...
Applying your Convolutional Neural Network: On-Demand Webinar and FAQ Now Available!
On October 25th, we hosted a live webinar—Applying your Convolutional Neural Network—with Denny Lee, Technical Product Marketing Manager at Databricks. This is the third webinar of a free deep learning fundamental series from Databricks. In this webinar, we dived deeper into Convolutional Neural Networks (CNNs), a particular type of neural networks that assume that inputs...
Simplifying Change Data Capture with Databricks Delta
Note: We also recommend you read Efficient Upserts into Data Lakes with Databricks Delta which explains the use of MERGE command to do efficient upserts and deletes. A common use case that we run into at Databricks is that customers looking to perform change data capture (CDC) from one or many sources into a set...
Training your Neural Network: On-Demand Webinar and FAQ Now Available!
On October 9th, we hosted a live webinar—Training your Neural Network—on Data Science Central with Denny Lee, Technical Product Marketing Manager at Databricks. This is the second webinar of a free deep learning fundamental series from Databricks. In this webinar, we covered the principles for training your neural network including activation and loss functions, batch...
MLflow v0.7.0 Features New R API by RStudio
Today, we’re excited to announce MLflow v0.7.0, released with new features, including a new MLflow R client API contributed by RStudio. A testament to MLflow’s design goal of an open platform with adoption in the community, RStudio’s contribution extends the MLflow platform to a larger R community of data scientists who use RStudio and R...
Introduction to Neural Networks: On-Demand Webinar and FAQ Now Available!
On September 27th, we hosted a live webinar—Introduction to Neural Networks—with Denny Lee, Technical Product Marketing Manager at Databricks. This is the first webinar of a free deep learning fundamental series from Databricks. In this webinar, we covered the fundamentals of deep learning to better understand what gives neural networks their expressive power: the potential...
How to Use MLflow To Reproduce Results and Retrain Saved Keras ML Models
In part 2 of our series on MLflow blogs, we demonstrated how to use MLflow to track experiment results for a Keras network model using binary classification. We classified reviews from an IMDB dataset as positive or negative. And we created one baseline model and two experiments. For each model, we tracked its respective training...
Simplify Market Basket Analysis using FP-growth on Databricks
When providing recommendations to shoppers on what to purchase, you are often looking for items that are frequently purchased together (e.g. peanut butter and jelly). A key technique to uncover associations between different items is known as market basket analysis. In your recommendation engine toolbox, the association rules generated by market basket analysis (e.g. if...
Identify Suspicious Behavior in Video with Databricks Runtime for Machine Learning
With the exponential growth of cameras and visual recordings, it is becoming increasingly important to operationalize and automate the process of video identification and categorization. Applications ranging from identifying the correct cat video to visually categorizing objects are becoming more prevalent. With millions of users around the world generating and consuming billions of minutes of...
