How HP is optimizing the 3D Printing supply chain using Delta Sharing

Javier Lagares is a Principal Data Engineer at HP, where he leads the development of data-driven solutions for the 3D printing business. With a strong background in data engineering and architecture, Javier is passionate about leveraging data to drive business outcomes and improve customer experience.

As companies increasingly rely on data to drive decision-making across all industries, HP’s 3D Print division is leading the charge with a transformative initiative: sharing near real-time equipment telemetry data with their customers that use 3D Printers as part of their own manufacturing process. This data sharing capability allows HP’s customers to monitor the performance and usage of their leased equipment and proactively take business or maintenance actions using data insights. By using Delta Sharing and other Databricks features like Unity Catalog, Mosaic AI and AI/BI dashboards, HP has built a secure and scalable solution that provides comprehensive insights into machine performance, material usage, and maintenance needs.

Empowering customers with data

HP is committed to providing its customers with detailed, actionable insights into their equipment. This initiative enables customers to view and analyze their printers’ telemetry data, making it possible to optimize their operations, reduce downtime, and gain a clear understanding of resource usage to reduce costs.

Led by Javier Lagares - principal data engineer, HP implemented a comprehensive data solution on the Databricks Data Intelligence Platform. This solution captures telemetry data from an array of sensors embedded in HP’s equipment, and combines it with data from several other devices including not only printers, but also processing stations (which handle tasks that prepare a 3D printed object for its final use), unpacking stations (which unpack parts, remove powder, and recycles materials post-3D printing), curing stations (which harden and stabilize parts after printing), among others. These sensors collect critical information for each processing station, such as the jobs processed and materials used in the jobs, among several other data points. This data essentially forms the “DNA” of each printer, allowing HP and its customers to gain valuable insights into the entire print job lifecycle.

Architecting the Solution with Databricks

HP’s solution is built on Databricks’ medallion architecture, which is designed to enable scalable data ingestion, transformation, and sharing; and includes four main stages:

1. Landing Layer: The landing layer serves as a communication layer with the devices. The data in this layer is processed exactly once. If data from this layer is not acknowledged as received by HP, a preemptive process kicks in to stop related devices.
2. Bronze Layer: The bronze layer unpacks the information from the landing layer into multiple tables for further consumption.
3. Silver Layer: In this stage, the raw telemetry data is cleansed, filtered, and transformed for usability. The data is prepared by applying transformations to remove redundancies, correct anomalies, and format it into a more usable structure.
4. Gold Layer: The Gold layer offers the curated datasets, which are then shared with HP’s customers via Delta Sharing, enabling them to analyze and utilize this high-value data directly.

The medallion architecture enables HP to process and share telemetry data ensuring that customers always have up-to-date information at their disposal. 

Sharing Data Securely with Delta Sharing

Once the telemetry data has been processed, HP shares it with customers using Databricks’ Delta Sharing open sharing capabilities (D2O). These features allow HP’s customers to consume the data; irrespective of whether they use Databricks or not. The recipients can directly connect to shared data using Delta Sharing connectors that support various systems like pandas, Tableau, Power BI, Apache Spark, Rust, or others that support the open protocol. When a customer is registered, they receive an HP endpoint, which they can use to access datasets using any Delta Sharing connectors or APIs. 

To make the process seamless for customers, HP provides comprehensive documentation for each data asset, detailing how to consume the data. HP also includes sample code that leverages D2O libraries to help customers quickly integrate the data into their pipelines, as well as a pre-built PowerBI dashboard that reads data through the Delta Sharing connector and customers can use if they prefer not to create their own visualizations. This level of support ensures that HP’s customers can easily begin using and analyzing their telemetry data without any prior expertise in Delta Sharing or the underlying data architecture.

Predictive Maintenance through Mosaic AI

One of the most impactful features of HP’s solution is the inclusion of predictive maintenance alerts. Using machine learning models that leverage Mosaic AI capabilities, HP can accurately predict maintenance needs based on telemetry data. These predictive maintenance alerts are provided to HP’s Customer support to help them provide a better customer experience by reaching out to customers to schedule equipment servicing ensuring their printers remain in optimal working condition.

These machine learning models analyze telemetry data patterns, which can indicate early signs of potential issues. By providing these alerts, HP enhances customer satisfaction by allowing customers to plan ahead and reduce interruptions to their production processes.

Enhanced Data Security with Unity Catalog and ApiGee

With customer data privacy and data security being top priorities, HP leverages Unity Catalog to enforce data governance and apply customer segmentation. Unity Catalog enables HP to control access to data assets at a granular level, allowing them to segregate data for each customer and apply custom data access policies based on specific contract terms. This dynamic model ensures that each customer only has access to their own data.

Additionally, HP has implemented ApiGee to further enhance data security. ApiGee acts as a masking layer for all HP internal URLs, making the data-sharing process 100% transparent to customers. The use of ApiGee ensures that customers only interact with secure endpoints, adhering to HP’s data security framework.

Monitoring and Enhancing Customer Experience with AI/BI Dashboards

HP also leverages Databricks AI/BI dashboards to monitor the usage of its telemetry solution. These dashboards provide metrics on data access frequency, geographic location, and the number of users accessing the data, among other interesting data points, giving HP’s team a comprehensive view of how their solution is actually used.

This usage data allows HP to understand customer consumption patterns and identify areas for improvement. For example, if HP notices a specific feature that is frequently accessed, they might prioritize enhancements to that feature in future product iterations. Additionally, the AI/BI dashboards offer insights into geographic trends, enabling HP to tailor their support and service offerings to meet regional customer needs.

Conclusion

The ability to access and analyze telemetry data in near real-time is a game-changer for 3D printing. HP’s solution not only offers valuable insights into machine performance and the 3D part generation process but also enables customers to take a proactive approach to equipment maintenance, maximizing the value of their leased 3D printers. By continuing to invest in advanced data-sharing technologies, HP is setting the standard for data-driven innovation in the 3D printing industry.

As HP continues to refine and expand its data-sharing capabilities, they are a clear leader in a future where data insights become integral to industrial processes and customer engagement. With each new enhancement, HP demonstrates its commitment to leveraging technology for customer empowerment, setting a high bar for data sharing and machine monitoring.