Creating pathways to student success
faster time to insights
“The Databricks Data Intelligence Platform has allowed us to centralize all our data for our teams to collaboratively uncover insights, and develop student-centric services that have a positive impact on retention rates, and ultimately lead to more successful student outcomes.”
As an online university focused on a student-centric approach to education, Western Governors University strives to make higher education accessible for as many people as possible. With enrollment skyrocketing to over 140,000 students, a legacy data warehouse environment and data silos slowed WGU’s ability to extract insights from the sheer volume of data on their students. They needed a way to unify the onslaught of disparate data, as well as simplify data engineering and infrastructure in order to accelerate innovation. With the help of the Databricks Data Intelligence Platform, the university is expected to hit its 10x growth target and greatly accelerate the development and deployment of new analytics to help administrators tailor curriculum and services to the needs of their students.
Data deluge in online education
Western Governors University’s core goal is to be the world’s most student-centric university. For them, this means providing a higher education to everyone, including underserved and underrepresented learners. They strive to minimize financial impact while maximizing life results by remaining a not-for-profit entity and encouraging students to only borrow the minimum amount of money they need to acquire an education.
As one can imagine, this credo has attracted a large number of education seekers, requiring WGU to quickly address their scalability issues. With large volumes of data coming in from disparate sources, the WGU data team dealt with the challenges of data accessibility, which complicated the usability of downstream analytics for making informed decisions based on Situation-Behavior-Impact (SBI) insights.
For example, they might experience an unacceptable dropout rate due to COVID-19. Through a better understanding of how COVID is impacting their students, they are able to identify key considerations for course extensions for these students to ensure they aren’t left behind. As Will Slade, Staff Data Engineer at WGU, explained, “Our goal as a data team is to hypothesize about what might improve student outcomes, implement the behavior and then measure the impact.”
With an objective to improve the student learning experience to drive retention and graduation rates as they continue to scale, WGU knew it was time to find a new solution that would unify their data, simplify infrastructure management and seamlessly feed data insights to decision makers.
Unified lakehouse unlocks student-centric insights
Since implementing the Databricks Data Intelligence Platform, WGU has seen significant improvement in all the areas where they needed it, plus more. “Databricks has helped us to bring our disparate data sources together into a single lakehouse,” said Slade. “The speed and scalability of our data living in Delta Lake has provided the flexibility we need to take the university in the direction of 10x growth.”
Delta Lake provides a high-powered data layer that allows them to build reliable and scalable data pipelines for their analytics and ML workloads. With the power of unified data, WGU now has several ML models in production, and several more in early adoption. These models can predict things like student dropout patterns, longer-term retention probabilities and course load completion, collectively contributing to the improvement of the student experience. The data team is also developing a predictive model with intervention paths to help retain students and get them closer to graduation.
“Databricks has enabled us to move from the mindset of reporting on what has already happened with our students to see what we can learn about how we’re doing, to reporting on what we can predict will happen with our students, to see if we might be able to reach out and intervene to improve outcomes,” explained Slade. “That shift is all the difference when it comes to our future successes as an online education provider.”
From a collaboration standpoint, the ability to log in to the platform from anywhere and work together in the same notebook has been game changing, especially in the modern world, where remote work is the norm.
“Databricks has changed the way we explore and analyze the various types of student data at our disposal,” added Slade. “Some of us have come from analytics backgrounds, so it’s amazing to be able to use Python, Scala, SQL, R, and know that we’re all writing the same code, just in a different wrapper. However our brain can best analyze a given problem, we can simply switch to that language, pass in whatever variables we need for that scope and make the necessary transformations to the data.”
With Delta Lake, the WGU data team has been able to move BI workloads off of their data warehouse, providing centralized access to student-centric data such as financial profiles, student performance and dropout rates. Now they can easily analyze this data and quickly identify opportunities to transform the learning experience and, in turn, impact the success rates of their student population.
More insights, higher student success
With Databricks, the WGU team has been able to transform their hard-to-scale data warehouse into a vast lake that seamlessly houses 80+ TB of data. And since moving their ETL jobs to Databricks, they’ve reduced their (core) overnight processing time from 10+ hours to between 3 and 4 — accelerating time to market of new insights that can dramatically help their student body.
As for the future of WGU, Databricks will play a large role in ingesting more and more data about students to leverage in their prescriptive models. One project the team is currently looking at includes natural language processing (NLP) on social media data, all in all speaking greatly to the institution’s goal of being student-centric.
“Databricks provides us with not only the tooling but the confidence to leverage our data to make education better,” concluded Slade. “It’s gratifying to play a role in helping our diverse community of learners meet their academic and life goals.”
