Solving the World’s Toughest Problems with the Growing Open Source Ecosystem and Databricks
We started Databricks in 2013 in a tiny little office in Berkeley with the belief that data has the potential to solve the world’s toughest problems. We entered 2020 as a global organization with over 1000 employees and a customer base spanning from two-person startups to Fortune 10s. In this blog post, let’s take a...
Engineering Interviews — A Hiring Manager’s Guide to Standing Out
Ask any engineering leader at a growth stage company what their top priority is, and they’ll likely say hiring. When we think about how big a decision taking a job is for both the company and candidate, the few hours of interviews seems pretty short. We want to make sure our job interview process makes...
Azure Databricks Achieves HITRUST CSF® Certification
We are excited to announce that Azure Databricks is now certified for the HITRUST Common Security Framework (HITRUST CSF®). Azure Databricks is already trusted by organizations such as Electrolux, Shell, renewables.AI, and Devon Energy for their business-critical use cases. The HITRUST CSF certification provides customers the assurance that Azure Databricks meets a level of security...
Automate deployment and testing with Databricks Notebook + MLflow
Today many data science (DS) organizations are accelerating the agile analytics development process using Databricks notebooks. Fully leveraging the distributed computing power of Apache Spark™, these organizations are able to interact easily with data at multi-terabytes scale, from exploration to fast prototype and all the way to productionize sophisticated machine learning (ML) models. As fast...
Better Machine Learning through Active Learning
Try this notebook to reproduce the steps outlined below Machine learning models can seem like magical savants. They can distinguish hot dogs from not-hot-dogs, but that's long since an easy trick. My aunt's parrot can do that too. But machine-learned models power voice-activated assistants that effortlessly understand noisy human speech, and cars that drive themselves...
How Databricks makes distributed sites successful
Credit: Pexels.com Databricks recently announced signing a lease for new office space in Toronto. After Amsterdam, Toronto is our second geographically-distributed engineering site and a major expansion milestone for the company. This is a good time to reflect on how we work hard to make distributed offices successful and make sure employees do their best...
YipitData Example Highlights Benefits of Databricks integration with AWS Glue
At Databricks, we have partnered with the team at Amazon Web Services (AWS) to provide a seamless integration with the AWS Glue Metastore. Databricks can easily use Glue as the metastore, even across multiple workspaces. YipitData, a longtime Databricks customer, has taken full advantage of this feature, storing all their metadata in AWS Glue. ...
Make Your Data Lake CCPA Compliant with a Unified Approach to Data and Analytics
With more digital data being captured every day, there has been a rise of various regulatory standards such as the General Data Protection Regulation (GDPR) and recently the California Consumer Privacy Act (CCPA). These privacy laws and standards are aimed at protecting consumers from businesses that improperly collect, use, or share their personal information, and...
End-to-end Data Governance with Databricks and Immuta
Businesses are consuming data at a staggering rate but when it comes to getting insights from this data they grapple with secure data access and data sharing, and ensure compliance. With new customer data privacy regulations like GDPR and the upcoming CCPA, the leash on data security policies is getting tighter, slowing down analytics and...
Databricks Demonstrates AWS Platform Integrations at re:Invent 2019
Databricks was proud to be a Platinum sponsor at re:Invent. The past year has been an exciting one for our partnership with AWS, as we built new integrations and deepened existing ones with so many AWS services. re:Invent was a great opportunity to showcase how our joint customers have benefitted from those integrations and we...