Delivering integrity and efficiency for the U.S. Postal Service

Inability to scale analytics with the data perpetuates stagnation

As one of the most trusted government agencies in the country, the U.S. Postal Service (USPS) depends on a network of people and technology to collect, transport, process and deliver nearly 130 billion pieces of mail to over 163 million delivery points per year. The United States Postal Service Office of Inspector General (OIG) was established as an independent oversight agency to help maintain confidence in the postal system and improve the bottom line through audits and investigations. For instance, a key focus of the agency is to detect and prevent postal crimes such as mail dumping, which is when postal employees intentionally discard or delay mail rather than deliver it to its intended recipients. By monitoring various USPS data points, the OIG can identify indicators of employees or routes that are involved with dumped mail.

Prior to its use of a data lakehouse architecture, the OIG’s on-premises infrastructure was highly complex to manage and costly to scale. This became increasingly problematic as data volumes increased at such a rate that the office faced challenges extracting insights and developing timely solutions. The agency struggled to handle the influx of customer and delivery-related data, provide a centralized view for all teams, and support reliable and performant data pipelines for downstream analytics and machine learning — a requirement that became heavily evident during the 2020 election season, which saw a historic spike in voting by mail due to COVID-19.

With a data team of more than 100 people who needed to work together or respond to anomalous activities in a timely manner, the OIG looked to the cloud and a new data architecture that would offer all its data teams easy access to any data and unlock new analytical and machine learning capabilities to further its efforts to improve mail delivery efficiency and accountability.

Lakehouse solves efficiency challenges and opens new doors

A combination of internal and external factors challenged the OIG to simplify the management of all its data at scale while also facilitating analytics and machine learning. The USPS OIG wanted to leverage industry standards for data management and analytics in-house, which prompted its transition to the cloud. As it explored data infrastructure options, the USPS OIG found that a data lakehouse is the only environment that offers a common place to do ETL, analytics and machine learning under the same umbrella.

With a data lakehouse, the OIG has removed the barriers that once blocked its ability to deliver reliable and timely data for analytics and machine learning. With an open lakehouse architecture, all the data the agency pulls from USPS is much clearer and easier to use across teams. It has also unlocked a level of operational efficiency and performance at an unprecedented scale. Data engineers can simplify ETL pipeline development and improve data reliability, data analysts can use SQL to collaboratively query and share insights with the built-in visualizations and dashboards, and data scientists can use AutoML to jump-start new machine learning projects by automating tasks and accelerating workflows.

For example, one solution called Informed Delivery provides eligible residential customers with a digital preview of their household’s incoming mail. This allows households to see and track their inbound mail more easily. Behind the scenes, OIG stores images of mail alongside traditional data on things like sender, receiver, package contents, weight and value. By combining this data, it can mitigate fraudulent activity in the shape of mail theft or dumping, while improving the customer experience.

As a branch of the federal government and a long-established institution without a reputation for being modern in its approach to workflows, the Postal Service expects to see an incredibly positive effect on customer satisfaction and the agency’s reputation through its adoption of the lakehouse.

Delivering insights that matter, with confidence

Migrating to a cloud based data lakehouse architecture not only paid dividends from an innovation standpoint, but it also made an impact on costs. Compared to its previous architecture, the agency experienced a 40% reduction in total cost of ownership. These savings were primarily due to the elimination of legacy data infrastructure, unnecessary storage and licensing costs, and productivity improvements.

Simultaneously, the OIG’s ability to harness its data for analytics and machine learning use cases was reduced from months to days, because of the data engineering efficiencies introduced by the lakehouse. In fact, it has been able to scale its data ingestion from 1 production pipeline in its legacy system to over 90 production pipelines in the data lakehouse. This has helped the agency deliver more value to its various auditors and investigators in uncovering anomalous activity, while also giving the data team the satisfaction of being able to create customer-facing solutions in ways that were impossible with the previous system.

Society has experienced a lot of change on many levels in the last couple of years, and the USPS OIG's primary goal as an organization is to be able to respond to those changes and support the various stakeholders who love its products but need more of them. The data lakehouse has enabled the USPS OIG to do that on a level it never dreamed of.

Now that the OIG can operate in a more forward-thinking and agile fashion, the team is taking on several additional use cases. With the lakehouse as its data foundation, the OIG is well-positioned to continue leveraging data, analytics and AI to deliver value efficiently and with confidence.