Powering a Better Future With ESG Data and Analytics
By now, most professionals know that the future of business goes hand in hand with social responsibility, environmental stewardship and corporate ethics. It should come as no surprise, then, that Sustainability and Environmental, Social, and Governance (ESG) have become top priorities for consumers, investors and regulators alike. But creating a better future for all stakeholders...
Azure Databricks Achieves DoD Impact Level 5 (IL5) on Microsoft Azure Government
We are excited to announce that Azure Databricks has received a Provisional Authorization (PA) by the Defense Information Systems Agency (DISA) at Impact Level 5 (IL5), as published in the Department of Defense Cloud Computing Security Requirements Guide (DoD CC SRG). The authorization closely follows our FedRAMP High authorization and further validates Azure Databricks security...
Amplify Insights into Your Industry With Geospatial Analytics
Data science is becoming commonplace and most companies are leveraging analytics and business intelligence to help make data-driven business decisions. But are you supercharging your analytics and decision-making with geospatial data? Location intelligence, and specifically geospatial analytics, can help uncover important regional trends and behavior that impact your business. This goes beyond looking at location...
Accelerating ML Experimentation in MLflow
This fall, I interned with the ML team, which is responsible for building the tools and services that make it easy to do machine learning on Databricks. During my internship, I implemented several ease-of-use features in MLflow, an open-source machine learning lifecycle management project, and made enhancements to the Reproduce Run capability on the Databricks...
Automatically Evolve Your Nested Column Schema, Stream From a Delta Table Version, and Check Your Constraints
We recently announced the release of Delta Lake 0.8.0, which introduces schema evolution and performance improvements in merge and operational metrics in table history. The key features in this release are: Unlimited MATCHED and NOT MATCHED clauses for merge operations in Scala, Java, and Python. Merge operations now support any number of whenMatched and whenNotMatched...
How Lakehouses Solve Common Issues With Data Warehouses
Data analysts, data scientists, and artificial intelligence experts are often frustrated with the fundamental lack of high-quality, reliable and up-to-date data available for their work. Some of these frustrations are due to known drawbacks of the two-tier data architecture we see prevalent in the vast majority of Fortune 500 companies today. The open lakehouse architecture and underlying technology can dramatically improve the productivity of data teams and thus the efficiency of the businesses employing them.
Security Cluster Connectivity Is Generally Available on Azure Databricks
This is a collaborative post co-authored by Principal Product Manager Premal Shah, Microsoft, and Principal Enterprise Readiness Manager Abhinav Garg, Databricks We're excited to announce the general availability of Secure Cluster Connectivity (also commonly known as No Public IP) on Azure Databricks. This release applies to Microsoft Azure Public Cloud and Azure Government regions, in...
Ray & MLflow: Taking Distributed Machine Learning Applications to Production
This is a guest blog from software engineers Amog Kamsetty and Archit Kulkarni of Anyscale and contributors to Ray.io In this blog post, we're announcing two new integrations with Ray and MLflow: Ray Tune+MLflow Tracking and Ray Serve+MLflow Models, which together make it much easier to build machine learning (ML) models and take them to...
Data Exfiltration Protection With Databricks on AWS
In this blog, you will learn a series of steps you can take to harden your Databricks deployment from a network security standpoint, reducing the risk of Data exfiltration happening in your organization. Data Exfiltration is every company's worst nightmare, and in some cases, even the largest companies never recover from it. It’s one of...
Accenture and Databricks Partner Together to Streamline Large Scale Machine Learning Deployments
Today, we’re excited to announce Databricks’ partnership with Accenture to provide high-value Databricks services and reusable components to enterprise clients globally. Specializing in data strategy and design, data platform modernization and AI, the Accenture data and artificial intelligence (AI) team leverages Databricks’ Unified Data Analytics Platform to streamline proven methodologies for large-scale machine learning deployments....
