The data subject request (DSR) is one of the most challenging components of GDPR. It provides individuals in the EEA (European Economic Area) the right to request what personal data has been collected, how that data is being used and to have that data exported, restricted, changed, and erased. For many organizations that rely on data lakes to store their big data, sifting through millions of files to find and modify individual records for a DSR within prescribed GDPR timelines (typically 30 days) is a time consuming, and quite possibly, impossible task.
Fortunately there’s a path forward. Databricks Delta, a powerful offering within the Databricks Unified Analytics Platform, improves data lake management and makes it possible to quickly find and surgically remove individual records.
Experience up to 100x performance improvements when querying your data with data optimizations such as auto indexing, caching and compacting. Finding records buried in your data lake is no longer a challenge.
Many data lakes do not allow users to remove individual records. Databricks Delta enables you to easily and reliably modify or erase individual records by pulling data into organized tables. Underlying raw files can then be erased for compliance.
Databricks Delta provides detailed transaction logs so you can you track changes made to your data to satisfy GDPR data subject requests and share as needed for compliance reporting.