The data subject request (DSR) is one of the most challenging components of GDPR. It provides individuals in the EEA (European Economic Area) the right to request what personal data has been collected, how that data is being used and to have that data exported, restricted, changed, and erased. For many organizations that rely on data lakes to store their big data, sifting through millions of files to find and modify individual records for a DSR within prescribed GDPR timelines (typically 30 days) is a time consuming, and quite possibly, impossible task.
Fortunately there’s a path forward. Databricks Delta, a powerful offering within the Databricks Unified Analytics Platform, improves data lake management and makes it possible to quickly find and surgically remove individual records.