Managed Sportlogiq to Databricks Data Ingestion Pipelines for NHL Teams: A Game-Changing Alliance

Overview

In the competitive world of professional hockey, NHL teams are always seeking to optimize their performance. Advanced analytics has become increasingly important in this quest. Third-party data vendors employ cutting-edge technologies, like computer vision and machine learning, to process large amounts of raw data and video footage. Their aim is to extract detailed insights from each game. A comprehensive analysis of these details often makes the difference between winning and losing.

One notable vendor in this field is Sportlogiq, a company based in Montreal. They utilize patented computer vision and machine learning technologies to capture and analyze data that would typically be beyond the scope of human observation. Sportlogiq provides comprehensive analytics services and tracking data to various entities in the NHL, including sports teams, leagues, media outlets, and performance enhancement companies.

However, for NHL teams to conduct SQL analytics and run Machine Learning models on specialized metrics, such as a player's decision-making ability under pressure, they need to integrate Sportlogiq's game event data with other relevant datasets. This includes player and puck tracking data from vendors like SMT, scouting information from RinkNet, team and league rankings from EliteProspects, and salary cap details from CapFriendly, among others. To effectively merge and manipulate this diverse range of data, teams require a data analytics platform that is not only versatile but also emphasizes simplicity, scalability, and collaborative functionality.

In this blog, we'll explore how Databricks, in partnership with Sportlogiq and Koantek, developed an automated and managed data ingestion pipeline. This pipeline is designed to seamlessly ingest and integrate Sportlogiq's data with other relevant, isolated data owned by NHL teams, all within the Databricks Platform. This collaboration makes Sportlogiq's data more accessible and easier to use on Databricks, ensuring that NHL teams can easily harness this wealth of information. The unique combination of Sportlogiq's data, Databricks' platform capabilities, and Koantek's operational know-how represents a groundbreaking advancement in sports analytics. It provides NHL teams with an unmatched competitive advantage in leveraging data-driven insights.

The Challenge

Almost every NHL team subscribes to Sportlogiq for game event data and various hockey analytics services. Most teams access this data and accompanying video through the Sportlogiq website (iCE). The more technically proficient teams go a step further, ingesting this data into their own analytics environments using the Sportlogiq API. This allows them to generate their own data and AI insights. There is a significant opportunity for teams to move beyond merely keeping pace with their competitors. By fully leveraging this data, they can differentiate themselves and set a new standard for in-game analytics and performance. However, several barriers prevent teams from fully utilizing this data to its greatest potential.

1. Limited Resources and Expertise: Many teams face challenges due to insufficient resources and expertise. This includes a lack of skills in data engineering, platform administration, data science, and advanced analytics. These limitations hinder their capacity to develop and sustain complex data analytics environments.
2. Bandwidth and Time Constraints: Even teams that have some capability often encounter issues with limited bandwidth or time constraints. In the fast-paced and demanding world of professional sports, finding the time and focus to develop and maintain data pipelines can be challenging. These tasks often conflict with other essential activities within the team.
3. Lack of a Robust Data Platform: Some teams possess the necessary skills and willingness to delve into advanced analytics, but they lack a suitable platform for effectively processing and analyzing the data. Without a robust and integrated data and AI platform, they are unable to fully unlock the potential of Sportlogiq's data and other similar data sources.

The Solution

Koantek and Databricks have created a comprehensive solution designed to assist teams at any stage of their Data & AI journey. This solution is tailored to help overcome the barriers mentioned above.

Key Offerings of Koantek's Solution:

Nightly Refreshable, Maintainable Pipelines: Koantek's pipelines utilize the Sportlogiq API, allowing teams to import all the data into Databricks or just the specific subset that they need. This ensures that teams have access to the most recent data for their analyses and decision-making processes. With this system, there's no need for manual updates, which greatly lowers the chances of delays or errors in the data.

Data Model for SQL Analytics: Koantek offers a pre-built data model on the Lakehouse designed for SQL analytics. This model lets analysts start querying and analyzing data right away, without the need to navigate the complexities of ETL (Extract, Transform, Load) processes or data ingestion difficulties. The tables within this data model are well-managed through the Databricks Unity Catalog, which ensures easy discovery, documentation, security, and tracking of data lineage.

Feature Store Tables for ML: Sportlogiq data can equally be utilized to train and construct AI models. Data scientists typically invest a significant (sometimes prohibitive) amount of time in data preparation and feature engineering. Koantek's solution offers pre-built feature store tables that structure and prepare the data in the ways most likely to be repeatedly leveraged by NHL data science team AI Models. Examples include player metrics by position and by shift, team performance by game, etc. Having pre-existing feature tables allows teams to bypass these preliminary steps in AI/ML and allows data scientists to concentrate on creating sophisticated machine learning models and insights.

Databricks Lakehouse Buildout: For teams that are not yet using Databricks, Koantek can set up and guide teams to onboard on the Lakehouse. This includes setting up Sportlogiq pipelines to provide a comprehensive, end-to-end solution that allows teams to establish their Lakehouse platform following best practices including source control management, continuous integration and continuous deployment (CI/CD), Infrastructure as Code (IaC) and automation using Databricks Asset Bundles (DAB).

Customization and Support: Koantek's offerings extend beyond just setting up technology; they also include customization, personalization, and continuous support. Recognizing that every NHL team has its own proprietary needs and tactics, Koantek collaborates closely with teams to customize data pipelines and build new analytic and AI use cases to meet their particular demands. This tailored approach guarantees that teams get the most benefit from their data analytics endeavors.

Effortless Integration and Usage: At the heart of Koantek's services is the simplification of advanced data analytics for NHL teams. Koantek manages the complexities of data pipeline management and integration with Databricks, enabling teams to concentrate on their strengths—analyzing data to improve team performance and strategies.

Impact on NHL Teams

This solution offers NHL teams a range of powerful opportunities that can transform their approach to game strategy, player development, game preparation, and scouting. These opportunities include:

1. Improved Game Analytics:
   • Data-Driven Game Strategies: By using insights from Koantek's managed data model, teams can craft superior game strategies grounded in statistical analysis and predictive modeling, giving them an advantage over competitors.
   • Real-time Tactical Adjustments: Advanced analytics integration allows teams to make better-informed decisions on tactics like line changes and shot quality assessments. This results in swift adaptation to the ever-changing dynamics on the ice and a higher likelihood of winning.
   • In-depth Post-Game Analysis: Detailed analytics provide a deeper understanding of each game's outcomes, enabling teams to conduct thorough post-game reviews and continuous improvement.
2. Enhanced Player Development and Performance:
   • Personalized Training Programs: By employing detailed analytics, teams can develop customized training programs tailored to the unique strengths and weaknesses of each player. This approach leads to an overall enhancement in performance.
   • Injury Prevention and Management: Advanced data analysis aids in identifying potential injury risks. This enables teams to put in place effective strategies for injury prevention and to optimize player health and longevity.
3. Efficient Scouting and Recruitment:
   • Data-Driven Player Evaluation: The capability to analyze detailed player performance data simplifies the scouting process. This allows teams to identify and recruit players who align best with their strategic requirements.
   • Opponent Analysis: Advanced analytics provide insights into the strategies and weaknesses of opponents. This equips teams with the necessary knowledge to exploit these aspects during matches.

The Future Vision

Combining Sportlogiq Data with Additional Sources:

• Integration of Multiple Data Sources: Understanding that NHL teams draw upon a wide range of data sources, Koantek envisions expanding its managed data pipelines to incorporate and synchronize data from various providers. This comprehensive approach will enable teams to access a broader analytics landscape, merging traditional statistics with advanced metrics.
• Unified Analytics Platform: Koantek aims to consolidate numerous data streams into the Databricks Lakehouse, aspiring to establish a unified analytics platform. This platform will empower teams to uncover correlations and insights previously unreachable due to data compartmentalization, fostering more intricate and strategic decision-making processes.

Beyond NHL: Adapting to Other Sports Leagues:

• Customized Solutions for Different Sports: The innovative approach of Koantek is not just confined to hockey. The company plans to adapt its managed data pipeline solutions to suit other sports leagues, each having its own distinct data requirements and analytical challenges.
• Global Sports Analytics Revolution: This expansion is set to empower teams in various sports around the world, heralding a new era in sports analytics. In this era, data-driven decisions will become standard, significantly enhancing performance, strategy, and fan engagement.

Conclusion:

With Koantek's managed data pipelines on Databricks, NHL teams can now sidestep the substantial investment in data pipeline infrastructure. Instead, they can focus on utilizing Sportlogiq data to boost their team's performance. This partnership signifies a new era in sports analytics, where data emerges as a crucial element in game strategy. What's even more exciting is that you can get this pipeline enabled and have a continuous stream of Sportlogiq data flow into your own Databricks workspace for further analysis today! So, why wait? Join the future of sports analytics with the Databricks Data Intelligence Platform.

To speak with the sports team at Databricks contact Harrison Flax. To get started with the NHL Sportlogiq data pipeline, contact Edward Edgeworth.