Conversational Analytics for Games

“So often I’m asked to produce a dashboard but the request isn’t always clear, even after having a conversation with the person. This means that I may build something for them, it may miss the mark and we have to start over, or iterate upon it. What I find interesting here is 1) they may be able to find the answers on their own and 2) perhaps more importantly, they can get close to what they were looking for and we can use that as part of their request for something from my team.” - Data Leader at a AAA studio

Introduction

When we announced the next evolution to Databricks, The Data Intelligence Platform in November 2023, we shared plans to integrate machine learning, GenAI and other capabilities into our platform. With this we would further your productivity and the value you could generate from your data. Through this blog we’ll explore one of these features whose goal is to democratize the generation of data insight, AI/BI Genie.

When we built AI/BI Genie, our goal was to empower non-technical data users to uncover insights without relying solely on technical experts. In the business intelligence space, this concept is commonly known as self-service analytics. However, we believe this term alone does not fully capture the value that AI/BI Genie delivers. To better reflect the user experience and the unique benefits Genie provides, we consider ‘conversational analytics’ a more fitting description.

Through conversational analytics, AI/BI Genie provides users the ability to engage and discover new insights through a chat-like interface that sits directly on top of their enterprise data. Instead of being limited to what a dashboard might show them, they can ask their own questions and dive deeper into details. You point it at a series of tables, the user asks questions of the data, the space interprets the question and presents results. With these capabilities, users are able to explore their ideas as though they were working side-by-side with a data analyst. It can even turn the results into visualizations for the user. For a detailed view of our Genie spaces solution, check out their documentation here.

In this game focused walkthrough we use a simulated dataset for avatars in a Viking centric roleplaying game. With it we create a Genie space, recreate a user journey and explore the value it can bring to a games company. When you see how easy it is to set up a Genie space, we encourage you to try it with one of your own datasets. It will be more meaningful and, as long as you are Unity Catalog enabled, it’ll be easier to set up than the sample space.

Creating a Genie Space

As a reminder, you need to be on Unity Catalog, or have the data sets you’re going to use in UC, for this to work. On the left side of your Databricks workspace you will see “Genie.” Click there, “New” in the top right corner, give it a title, assign it a warehouse and add tables. You now have a Genie space and have set up conversational analytics for these datasets.

When creating a Genie space, and selecting the tables to include, think about the end user. Who is going to be asking questions? What types of questions are they going to ask? What tables might support those questions? You may include Silver and/or Gold tables for the domain in question. Avoid datasets completely disconnected from each other as it will negatively impact your results.

For this blog we are using a dataset that includes information about characters in a fictitious Viking MMORPG over a period of three years. The datasets include high level session information, character level progression, character class, zones they entered during a session and other related details like guild membership. As we explore this Genie space we thought through the lens of a game designer who is new to this capability and is interested in understanding character engagement across the open world.

Getting Started with the Genie Space

When you enter the space there’s a button that will analyze the datasets and provide you with some example questions you could answer with data included in this space. By default these are auto-generated by the Genie space based on the data sets included but you could add your own examples when creating the space. These prompts help a new user understand the types of things this space may be able to answer to help them get started. Your user can also ask the space, as we do in the next diagram, to describe the data it has access to. It will generate details about the tables, their columns and what it might all mean.

Interrogating and Iterating on Your Data

We then experimented with the Genie space as if we were brand new to the capability, and were a game designer looking to learn more about the character experience across our game world.

Mentioned above, to start our effort we asked the space to tell us a bit about the tables themselves. We see this is a relatively simple space with a single table, but there’s still lots to investigate here about how our players are engaging. This table has information about their avatars, level, race, class, guilds and what zones they were during different sessions. Let’s imagine we were wanting to understand more about social gameplay and guild makeup. To get used to the interface we start with a simple question: “Show me the top 10 guilds.” The space returns the guilds and understands that “top 10” probably means by the number of characters in the guild, so it gives us a table with the ordering and the number of characters.

This example shows us a few important things

1. The space can infer details from our question: “Top 10,” while explaining what it thought you meant. This is helpful because what if you meant Top 10 based on time played?
2. The space also does its best to provide enough detail for you to understand why it thought this was the case. We just asked for the Top 10 guilds, it could have just returned their names in order but by including the character count we can trust the result more.

We’ve found that Kind Brown Brigands is the top guild, it’d be interesting to know a bit more about that guild. What is the makeup of classes, what does the data tell us about the characters in this guild, what other insight can we glean? So we start to ask questions of the data like: “What are the top classes within Kind Brown Brigands?”

We get a table response and the space infers, since this shows the distribution of 5 classes, that it might be easier to read as a chart, specifically a bar chart. We didn’t ask for it, but it was provided. I’m curious what else I could garner from this, I don’t really know where I want to go, so let’s ask something open ended. “What else can you tell me about the Goði in Kind Brown Brigands. Genie spaces aren’t magic, but it does its best and gives us all the details that it can.

Looking at this data it makes me wonder about the distribution of max level Goði that are elves. We’re planning some end game content and a remarketing effort aligned with it. It would be good to know how many players that would impact. So we ask about the percentage of Goði elf players who are at max level.

Our answer isn’t quite right though. As you can see in the explanation it says that 0.28% are level 60, but we can see in the table above (and I would know anyway, it is my game!) that max level is 70. This is why, in addition to the answer, the context and natural language explanation of what it is providing is so important. For insight generation to be democratized and for non-core data users to explore datasets on their own they need feedback that they can understand. With the above mistake made, we can modify our question and say “Max level is 70, not 60, please re-run the above.” This is cool because we didn’t have to write the full prompt again, it remembers what we asked and modifies the query that it ultimately used.

About 4.5%, that’s not a ton but aligns with the number of people who typically make it to the endgame for most video games, so I’ll buy that. Knowing that under ½ percent are level 60, and 4.5% are 70, it makes me wonder, what’s the level distribution like across the whole group?

This is enlightening. Outside of level one and 70 there’s a relatively flat distribution across all levels. It looks like level one is about twice that of level 70, so 10%? Even in this guild there’s a lot of people creating a new character and either churning or realizing immediately that it wasn’t for them. Makes me wonder, is that normal? Is this an issue with this specific guild, class or character type? Let’s look broader, across all classes and guilds, what’s the level distribution. We find something very interesting.

That was not common, in fact the highest concentration across levels for the population as a whole is level 70. It hits me, there’s 10M records being shown here, but I know my population is only a few thousand. What could be going wrong? I’ve gotten close, this might be a time that I ask a data analyst to give me a hand. I remember that this dataset has a timestamp, I wonder if it is showing each person once, or is it counting each session and their level during that session. Let’s try one last thing and ask: “Character name should be a unique value, can you ensure that level distribution only considers the max level for each character?”

We see a very different result now. Most of the population made it to level 71-80, about half stopped between 61 and 70.

In Conclusion

This was just a short example of how AI/BI Genie offers conversational analytics to help game designers explore their data, find insight and then take action on it. While this example focused on player engagement data but the same process and capabilities could be applied to all game domains including: marketing, campaign optimization, server operations, match outcomes and LiveOperations. We hope it has inspired you to try AI/BI Genie on one of your datasets.

At the start we shared a quote from one studio and have included one from a AA console game maker, and a mobile studio:

• “A lot of the requests I receive are Adhoc, can you find this number for me. These requests end up iterative in nature because I’ll give them the answer and then they want to know the why behind the number. Looking at this tool I could put together a Genie space with data related to their question and give them the power to find the answer and then see the details behind it. Not only will this save my team’s time but it should also delight my end users.” - AA console game maker’s Data Analytics Director
• “I have end users who understand the data we are collecting really well, some of them know a bit of SQL, though they aren’t experts, but most of them don’t. These users, typically product managers, game designers or LiveOps teams want more access to data and the ability to find answers on their own. With this capability I could create a space focused on their domain and provide them a self service tool that will speed their ability to react to change and free my team up for bigger projects.” - Mobile game Director of Data Operations

With conversational analytics like AI/BI Genie, you empower your teams to find emergent insight, increase satisfaction from your internal customers and free data teams to work on high value projects. This post shows how easy it is to create a Genie space, create one with your own dataset, and learn something new about your players in the process.

If you’d like to learn more about Genie spaces, how to activate them and how to use them, check out the official documentation or reach out to your Databricks account team. You can find the dataset we used here. For insight into additional games data and AI use cases, see our other Solution Accelerators and our Guide to Games Data and AI.