Maximizing the potential of work with the power of data
Asana uses Databricks to improve data team productivity
Used by over 100,000 organizations and millions of teams worldwide, Asana is on a mission to help humanity thrive by enabling the world’s teams to work together effortlessly. Within Asana, teams of data scientists focus on providing product and business insights to inform product innovations, ad spend optimization and more. With massive volumes of business and product engagement data generated daily, Asana’s data science organization struggled to keep up with an over-reliance on hard-to-scale Jupyter open source notebooks and the complexity of managing cloud-native services. Today, Asana uses the Databricks Data Intelligence Platform to accelerate transformative outcomes efficiently and at scale. Databricks Notebooks have removed collaboration barriers to jump-start productivity and efficiency while adding much-needed governance and security to protect sensitive data.
Legacy notebooks and data science bottlenecks
Software companies must empower their data science organizations to drive product innovations, compete in the market and deliver business impact. Asana understands that smart downstream decisions depend on the upstream capabilities of data teams — and the Asana team needed help. The most prominent issue internal teams faced stemmed from self-hosted, open source Jupyter Notebooks that were not collaborative and were complex to scale. As a result, operational inefficiency persisted.
To collaborate, data scientists had to copy and paste individual notebooks to give others their own view, creating versioning issues, risks of overwrites and operational inefficiencies in general.
Brian Estlin, Engineering Manager at Asana, explained, “Doing different types of analyses was challenging because our Jupyter Notebooks weren’t collaborative and would often crash. That wasted a lot of time and effort.” When the negative impact of resource-intensive and low-performance tools started adding up, Asana decided it was time to start phasing out old technologies, replacing them with modern solutions built to support their complex needs.
Collaborating securely with the Databricks Data Intelligence Platform
After exploring the Databricks Data Intelligence Platform and experiencing the open and collaborative nature of the integrated Notebooks — Asana migrated to Databricks and never turned back. Asana replaced their Jupyter Notebooks with Databricks Notebooks for their built-in collaboration and fine-grained access controls. Davidheiser describes the highlights, “We love the fact that we can comment on, view and share notebooks in real time and with confidence. Sitting on a call together and working on the same notebook collaboratively is really powerful. Also, the organizational capabilities with fine-grained access controls to decide who you do or don’t want to see the data is critical given the sensitivity of the data our clients entrust us to manage.”
Equipped with faster and simpler data ingestion and query performance, Asana was able to optimize pipelines for downstream analytics as well. “Databricks is able to query over months of product engagement data much faster than we were previously able to. That unlocked types of analyses that were previously much more expensive — either from a human or compute time standpoint,” explains Nathan Lawrence, Engineering Manager at Asana. By reducing the complications commonly holding organizations back, Asana began discovering new levels of pace and efficiency to improve support for a range of use cases, including informing roadmap decisions based on feature adoption and customer needs, identifying key marketing channels to engage with prospects and optimizing ad spend to drive marketing ROI.
Fast, accurate and accessible data drives innovation
With the Databricks Data Intelligence Platform, Asana is building a modern data infrastructure that supports the data science organization now and into the future. Paramount to their success is the enthusiastic adoption of Databricks within their data and engineering teams. A year after launching, an internal survey run across the data organization showed that 78% of data scientists reported feeling positive about the Databricks Platform, higher than any other data tool deployed at the company.
From a business perspective, Asana has substantially reduced costs by increasing productivity and collaboration, allowing data teams to serve other departments quickly for informed, data-driven decision-making. Data modeling within Delta Lake is dramatically more straightforward with ACID transactions and merge updates on existing datasets. With the help of Delta Lake, Asana’s data team members are saving 2–3 hours of execution time every week with faster and more capable queries. With seamless Tableau integration through Databricks, Asana is also able to create self-serve analytics dashboards for product managers and engineers to better understand their teams’ impact and optimization needs. Nathan Lawrence summarizes Asana’s success, saying, “The biggest win is efficiency from a computation and organizational standpoint. We can scale data science and infrastructure engineering to be more effective while simultaneously maintaining the same headcount. It’s a people and organizational force multiplier.”
Of course, Asana isn’t stopping there. They are now implementing Unity Catalog to bolster security and granular access controls over team-specific data within team-specific catalogs. Delta Live Tables is being incorporated to reduce the need for manual processes which created friction in modifying key pipelines, enabling easier maintenance and landing data at lower latency. Asana also plans to enable Databricks Assistant which takes natural language prompts to generate queries and fix errors directly within the Notebooks — empowering their data scientists to be more self-sufficient and productive.
Looking ahead, the Asana data team is confident in their roadmap for the future due to the operational efficiency and cost-cutting provided by the Databricks Data Intelligence Platform. As James Davidheiser says, “Working with Databricks just makes the operation of everything much easier. We are working better together and operationalizing data insights that are having a material impact on our business. It’s an incredibly powerful and compelling platform.”
