Databricks Enables Metacog to Accelerate Time to Market With Continuous Integration

SAN FRANCISCO, CA--(Marketwired - Mar 31, 2016) - Databricks, the company behind Apache Spark, today announced that Metacog, a cloud-based analytics platform for adaptive and competency-based learning, has selected Databricks as its enterprise Spark solution. Databricks affords Metacog the ability to automate Spark operations for numerous measureable benefits.

Metacog provides analytics to education institutions, corporations, and government entities by monitoring and analyzing how individuals tackle open-ended performance tasks to assess whether learning goals have been met. The company's business model hinges on accurately scoring hundreds of different assessments per customer in an automated fashion by machine learning. Given the scoring criteria for each assessment is unique to different learning goals, Metacog needed an efficient system to build and deploy production for a large number of machine learning models.

As result of the quantity of Metacog's data and speed at which it needs to be processed and analyzed, Metacog was attracted to Apache Spark as a big data engine for its flexibility in performing ETL and developing machine learning algorithms. Yet the company suffered release delays and software defects because thoroughly testing code on Spark clusters proved to be too complex and time-consuming.

With the implementation of Databricks, Metacog created an environment to fully automate their test and release processes, allowing them to achieve a number of benefits:

• Doubled release cadence: Metacog accelerated their time-to-market by reducing release cycles from one month to, at most, two weeks with the continuous integration system.
• Managing infrastructure costs: Metacog used Databricks to automatically shut down unused Spark clusters (e.g., over the weekend), achieving significant infrastructure savings of approximately 28 percent.
• Improve developer productivity: Databricks enabled the Metacog team to free up 20 percent of their time, reallocating this to product development rather than upgrading and maintaining their Spark production environment.

"Metacog's success in using our platform validates two key Databricks values; not only are we simplifying operations to achieve measurable improvements in business metrics, but we are doing so while helping them to manage infrastructure complexity and costs. This kind of customer use case is encouraging and incredibly rewarding," said Kavitha Mariappan, Vice President of Marketing at Databricks.

"Not having a way to automate our Spark production environment resulted in many software defects and long release cycles. Databricks enabled us to build a continuous integration environment easily," said Doug Stein, Chief Technology Officer at Metacog. "Now every member of the team can automatically test code with Spark clusters, resulting in faster detection and remediation of bugs and much shorter release cycles."

Download the Metacog case study here: https://www.databricks.com/
For more information on Databricks visit: https://www.databricks.com/product/databricks

About Databricks:

Databricks' vision is to make big data simple for the enterprise. The company was founded by the team that created and continues to drive Apache Spark, a powerful open source data processing engine built for sophisticated analytics, ease of use, and speed. Databricks is the largest contributor to the open source Spark project providing 10x more code than any other company. The company has also trained over 20,000 users on Apache Spark, and has the largest number of customers deploying Spark to date. Databricks is venture-backed by Andreessen Horowitz and NEA. For more information, contact [email protected].