How Blitz and Databricks are Powering a New Era of Competitive Gaming

Understanding the best strategy when dealing with millions of possible combinations

How do you take the gameplay of millions of daily users in order to recommend the strategy with the highest probability of success for any given situation?

Blitz simplifies the process of mastering difficult games, helping gamers improve at every step of their journey to the top. The Blitz App, recognized as the best cross-games coaching app in the market, leverages telemetry data from over 8 million active users to provide our players the best-in-class recommendations and feedback via overlays, performance insights, and stats for games like League of Legends, Valorant, and Fortnite.

Using machine learning to personalize at scale

For years, we at Blitz had a linear approach based on having our team of experts spend months or even years going deep into each game analyzing what the best players were doing in order to teach this strategy to beginners and competitive gamers on a 1-to-many basis. But for Riot Games' Teamfight Tactics (TFT), we undertook a new approach, grounded in Machine Learning, to deliver the new Blitz Analyzed Comps that offer personalized gameplay recommendations based not on overall strategies, but rather on the specific scenarios a player is facing at the moment to increase the likelihood of success.

What made TFT unique compared to other games was not just the sheer volume of possible compositions (theoretically up to 608 compositions), but the relatively short duration of TFT sets (usually just two months) such that the average player would not have time to master even the most common compositions unless they played dozens of games per day.

With ML, we are able to analyze more scenarios than ever so that we can provide 1:1 recommendations and coaching at scale and much faster than ever before. By expanding our focus from specialized, vertical solutions to encompass a more versatile, horizontal approach, we are laying the groundwork for a data-driven transformation that meets each of our gamers where they are in their journey. And what used to take months or even years to deliver general gameplay strategies, is now more targeted to a player's scenario and can go to market within weeks of a game's release.

So how does this all work? The large volume of data from our player base allows us to offer insights on what might be good or bad choices in the game, helping players understand what they might do to improve. This work is an ongoing process, and we're continuously exploring ways to leverage data and ML to provide more accurate and tailored suggestions for our players.

How we got here: The ability to be 'close to the data'

Data has always played a central role in The Blitz App, guiding our approach from day one and fueling all of our features. As we've expanded, we've worked to stay ahead of our competitors by fetching unique data through our expertise in reverse engineering the data generated by games like League of Legends, Valorant, and Fortnite. This allows us to gain insights into game mechanics, player behavior, and performance metrics that are not readily available through official APIs or documentation.

In addition to our original game data sources, we have generated a wealth of telemetry events from the app, which we collect and process for various BI purposes, further enhancing our data-driven insights to better understand our users. It's not simply about fine-tuning individual game strategies anymore; it's about supporting new games in a matter of weeks.

At the core of this shift is the Databricks Data Intelligence Platform, the backbone of our product, providing the open and scalable data platform necessary for such an expansive vision. We are continuously building a data platform that will soon process data for all competitive games in the market, with Databricks playing a crucial role in ensuring efficiency, cost-effectiveness, and real-time capabilities. The introduction of the Blitz Analyzed Comps for Riot Games' Teamfight Tactics is paving the way for a new era where our App, powered by Databricks, becomes a universal tool relevant for every game and every player.

Databricks has proven to be a versatile partner in our mission to leverage data as efficiently as we can. The ability to be "close to the data", querying and visualizing either some really specific insights or the big picture at a glance, has streamlined our process, providing valuable insights quickly.

Migrating from Snowflake to the Databricks Data Intelligence Platform

The decision to migrate from Snowflake to Databricks was prompted by a series of limitations and challenges we faced with Snowflake. The inability to explicitly partition data, particularly given our practice of splitting data by date, meant that we were leaving substantial optimization opportunities on the table. While Snowflake was starting to roll out Snowpark at the time, it still fell short of our evolving needs, especially regarding machine learning integration and workflow tools. The overall cost was another significant concern, as Snowflake's structure was leading to increased spending without corresponding benefits.

The shift to the lakehouse signaled a move towards a more modern and efficient way of managing data aggregation, particularly within our core games such as TFT and Valorant. Migrating the backend aggregation pipeline for Valorant was not only a strategic decision for cost-saving but also a transition towards contemporary methods of handling data. Historically, each game we supported required a custom-made aggregation backend with multiple cloud VM nodes to handle incoming requests.

Thanks to Databricks, we now enjoy much simpler and unified data pipelines, improved control, and interaction with our data, along with the ability to create alerts and charts that our previous backend setup could not offer. These enhancements have led to more accurate insights, substantial reductions in infrastructure costs, and the added benefit of Databricks' auto-scaling ability.

Databricks Data Intelligence Platform has proven to be a remedy to the challenges we encountered with Snowflake. Its flexible data partitioning capabilities allow us to optimize data management, aligning with our specific needs. The seamless integration with machine learning, coupled with tools like MLflow, has provided a more robust and agile environment for experimenting and model training. Databricks' cost-effective structure has also been a welcome relief, demonstrating value without compromising functionality.

Perhaps most impressive has been the continuous evolution of Databricks, with the release of timely and relevant new features, consistently providing us with the "Ah, I really needed this" feeling. For example, consuming data from Databricks used to require utilizing a read-many database or query engines such as Presto. However, with the introduction of innovative features like Serverless DBSQL and the Online Store, Databricks has streamlined the process, reducing the number of interconnected components needed to manage a wide array of data serving use cases.

The Databricks-Blitz partnership

We have been leveraging Databricks Data Intelligence Platform for approximately two years now, continually enhancing our best practices along two primary axes:

• Game data features: While browsing our app, we actively fetch game data on behalf of the user, serving as a mediator to access crucial game information that might otherwise be difficult or impossible for the player to retrieve on their own. Fetching is done through a sophisticated and advanced scraping backend that manages all Game API requests to acquire game data. This data is not only vital for providing personalized feedback to players but also serves our analytics, eliminating the need to fetch data ourselves. We utilize this information via aggregates that power up the statistics pages as well as some features and overlays within our app, thereby enriching the user experience.
• Business intelligence (BI): The telemetry data, or app usage, forms another significant portion of our data sources. From the very start, we have created a workflow that allows the frontend team for the app to easily export events generated by a player during a typical journey. This process enables us to gain a deeper understanding of our players, monitor our success, and make informed, data-driven product decisions.

For us, it has proved incredibly powerful and efficient to use a single platform for ETL, warehousing, BI, data exploration, and ML. Its rich set of features - including the ability to seamlessly scale according to the load, the auto-loader that enables real-time data ingestion, and the different connectors for our app team to fetch and serve aggregates - have empowered us to tackle end-to-end use cases in a way that wasn't possible before.

Additionally, we've been harnessing Databricks' capabilities to build custom components that make our work more effective and nimble on our end as well - from creating a generic App event ingestion pipeline that streamlines data collection to developing generic aggregation pipelines that simplify data processing when supporting new games. The synergy between Databricks' broad features and our specific needs is forging a pathway to continuous innovation, agility, and success in the competitive gaming market. The decision to work with Databricks was guided by their unmatched scalability, flexibility, and alignment with our vision for the future of gaming.

Databricks has been particularly transformative for our work on Teamfight Tactics (TFT). The ease of transitioning from development to production on the same platform, including writing production notebooks, hosting jobs, and handling Git, has enhanced our agility.

For our TFT AI features specifically, Databricks' native MLflow integration has been a game-changer, allowing us to simply train iterative numerous models and check all the experiments directly within Databricks. This seamless integration has not only made our process more efficient but also empowered us to innovate and adapt, contributing to our success in providing top-notch experiences for TFT players.

Auto-Loader has also been making our engineers' lives easier, from the data team and beyond. We have developed a pipeline that makes it super simple for software engineers in the company to be able to query telemetry event data in real-time. This drastically reduced the operation time needed for the data team to perform ad hoc requests. It has been really pleasant to see that even our software engineers have shown a great interest in Databricks and have even built dashboards on top of SQL queries.

Receiving all that telemetry data has also been a strong asset when it comes to detecting eventual downtime across the app. We have been able to automatically detect any big difference in data that could eventually be a sign of something wrong. All of those alerts end up in our Slack channel and are handled by different teams, helping us ensure we meet our SLAs.

The future of Blitz and Databricks

Blitz aspires to become the all-in-one hub for players engaged in multiple games. Recognizing that most players enjoy more than one game and would prefer not to clutter their devices with individual apps, Blitz aims to consolidate all of a player's game match histories, stats, and powerful features into a single, unique application.

Our vision goes beyond merely being a convenience; we strive to deliver the most accurate, best-in-class statistics available. With a continually growing user base, our commitment to excellence propels us toward offering unrivaled precision in our data insights.

Expanding horizontally and scaling across more games presents a unique set of challenges, particularly in reducing downtimes, maintenance, and operation time and costs. We strive to remain agile and efficient, unwilling to be held "hostage" to our pipelines and recognizing that maintenance time detracts from developing new games and features. Moreover, minimizing the time it takes to go to market for new games and releases is a crucial objective. Despite the unique nature of each game, we capitalize on common concepts within our pipelines to make them as generic as possible.

Databricks' ongoing innovations have significantly eased the burden of managing massive volumes of data, complex infrastructure and machine learning — setting us up with the means to allocate our resources toward new games and features that meet the demands of our customers.

To learn more about how we use clustering and conditional probabilities to provide 1:1 recommendations for TFT players, check out our post on Medium.

Get started with Databricks today!