How Australia’s National Health Services Directory Improved Data Quality, Reliability, and Integrity with Databricks Delta and Structured Streaming

Download Slides

Healthdirect Australia delivers telehealth and digital health services in partnership with the Australian federal, state and territory governments. Healthdirect helps address key priorities and challenges within the health, aging and social service sectors. In Australia, we run the National Health Services Directory (NHSD) that stores every Healthcare Organisation, Health Practitioner, their Physical and Virtual Locations, Health Care Services and eHealth Secure Messaging Endpoints. This is forecast to need to scale to handle over 10 terabytes of data covering time-driven activity-based healthcare transactions.

Globally we share similar problems with Healthcare data such as incomplete integration across the Health Sector participants, data supply chain inefficiency and duplication. At Healthdirect we use Apache Spark and Databricks Delta’s fine-grained table features and data versioning to solve duplication and eliminate data redundancy. This has enabled us to develop and provide high-quality data through federation and interoperability services whilst providing the analytics to improve Health Services demand forecasting and clinical outcomes in service lines, such as Aged Care and Preventative Health.


Try Databricks
See More Spark + AI Summit in San Francisco 2019 Videos

« back
About Peter James

Peter leads the Architecture practice at Healthdirect Australia and has over 20 years’ experience in delivering data intensive architectures across banking, finance, telecommunications and Health. He has built distributed systems using big data and analytics with machine learning capabilities. Peter is also an active participant in Marine Citizen Science programs applying big data and computer vision techniques to map and identify coral species on Australia's reefs.