Securely Accessing Azure Data Sources from Azure Databricks
Azure Databricks is a Unified Data Analytics Platform that is a part of the Microsoft Azure Cloud. Built upon the foundations of Delta Lake, MLFlow , Koalas and Apache Spark, Azure Databricks is a first party service on Microsoft Azure cloud that provides one-click setup, native integrations with other Azure services, interactive workspace, and enterprise-grade...
Check out the killer lineup of keynotes at Spark + AI Summit 2020
The Spark + AI Summit is already the world’s largest data and machine learning conference bringing together engineers, scientists, developers, analysts and leaders from around the world. This year is shaping up to be our biggest conference ever, with over 7,000 attendees expected to attend four days of training sessions, presentations and networking events. We’ve...
Introducing Databricks Ingest: Easy and Efficient Data Ingestion from Different Sources into Delta Lake
We are excited to introduce a new feature - Auto Loader - and a set of partner integrations, in a public preview, that allows Databricks users to incrementally ingest data into Delta Lake from a variety of data sources. Auto Loader is an optimized cloud file source for Apache Spark that loads data continuously and...
New Data Ingestion Network for Databricks: The Partner Ecosystem for Applications, Database, and Big Data Integrations into Delta Lake
Organizations have a wealth of information siloed in various sources, and pulling this data together for BI, reporting and machine learning applications is one of the biggest obstacles to realizing business value from data. The data sources vary from operational databases such as Oracle, MySQL, etc. to SaaS applications like Salesforce, Marketo, etc. Ingesting all...
On-Demand Webinar: Granular Demand Forecasting At Scale
We recently hosted a live webinar — How Starbucks Forecasts Demand at Scale with Facebook Prophet and Databricks — During this webinar we learnt why Demand Forecasting is critical to Retail/ CPG firms and how it enables 22 other use cases. Brendan O’Shaughnessy, Data Science Manager at Starbucks walked us through how Starbucks does demand...
Celebrating Black History Month | Black @ Databricks
Celebrating the launch of our two newest ERG Groups:Black at Databricks and Latinx at our 2020 Company Retreat With the start of Black History Month, Databricks launched our newest Employee Resource Group, Black at Databricks. The mission of this group is to provide support to Black employees and allies by providing a place to build...
Actionable Insight for Engineers and Scientists at Big Data Scale with Databricks and MathWorks
Today, Databricks announced that it is launching a new partnership with MathWorks, the leading developer of mathematical computing software, MATLAB, and Simulink products that are used by engineers and scientists globally. From medical devices to jets and autonomous cars, millions of engineers and scientists use MATLAB and Simulink to build and test autonomous systems using...
How to Display Model Metrics in Dashboards using the MLflow Search API
Machine learning engineers and data scientists frequently train models to optimize a loss function. With optimization methods like gradient descent, we iteratively improve upon our loss, eventually arriving at a minimum. Have you ever thought: Can I optimize my own productivity as a data scientist? Or can I visually see the progress of my training...
Databricks Named A Leader in Gartner Magic Quadrant for Data Science and Machine Learning Platforms
Gartner has released its 2020 Data Science and Machine Learning Platforms Magic Quadrant, and we are excited to announce that Databricks has been recognized as a Leader. Gartner evaluated 17 vendors for their completeness of vision and ability to execute. We are confident the following attributes contributed to the company's success: Our unique ability to...
Building Reliable Data Pipelines for Machine Learning Webinar Recap
This is a guest blog from Ryan Fox Squire | Product & Data Science at SafeGraph At SafeGraph we are big fans of Databricks. We use Databricks every day for ad hoc analysis, prototyping, and many of our production pipelines. SafeGraph is a data company - we sell accurate and reliable data so our customers...