All posts

From Static Data Warehouse to Scalable Insights and AI On-Demand Government agencies today are dealing with a wider variety of data at a much larger scale. From satellite imagery to sensor data to citizen records, petabytes of semi- and unstructured data is collected each day. Unfortunately, traditional data warehouses are failing to provide government agencies...

Adding a Lower Tier, Lower Price Option Since its inception a guiding principle for Databricks has been the unification of data science and engineering helping to bring together silos and raising the level of collaboration and productivity among the analytics professionals. And our customers have been reaping the benefits of this to solve some of...

As more and more analytics move to the cloud, customers are faced with the challenge of how to control which users have access to what data. Cloud providers like AWS provide a rich set of features for Identity and Access Management (IAM) such as IAM users, roles, and policies. These features allow customers to securely...

Azure Databricks Unified Analytics Platform is the result of a joint product/engineering effort between Databricks and Microsoft. It’s available as a managed first-party service on Azure Public Cloud. Along with one-click setup (manual/automated), managed clusters (including Delta), and collaborative workspaces, the platform has native integration with other Azure first-party services, such as Azure Blob Storage,...

With a tsunami of data, scale of computing resources available, and rapid development of easy-to-learn open source Machine Learning frameworks, data science and machine learning concepts are much easier to learn and implement today than they were a decade ago. As a result, across all industries, practitioners are using cutting-edge ML algorithms to solve tough...

Simplify building big data pipelines for change data capture (CDC) and GDPR use cases. Databricks Delta, the next-generation engine built on top of Apache Spark™, now supports the MERGE command, which allows you to efficiently upsert and delete records in your data lakes. MERGE dramatically simplifies how a number of common data pipelines can be...

With 2019 in full swing, the excitement for data and AI driven innovation continues. Over the past few years, we’ve seen leading innovators - like Riot Games, Regeneron and Shell - become early adopters of the latest machine learning and AI technologies, building and deploying AI applications into production. But with great promise comes great...