During the past several years, Spark has significantly changed the landscape of big data computing. It improves applications’ performance dramatically. However, there still remains several challenges, e.g. high GC overhead. In this talk, I will introduce Tachyon, a distributed in-memory storage system. In addition, I will talk about how Tachyon can further improve Spark’s performance and the integration between the two systems.
Haoyuan Li is founder and CEO of Alluxio Inc.(formerly Tachyon Nexus). Before founding the company, he was working on his Ph.D. at UC Berkeley AMPLab, where he co-created Alluxio, a memory-speed virtual distributed storage. Haoyuan is also a founding committer of Apache Spark. Before the AMPLab, he worked at Conviva and Google. Haoyuan has an MS from Cornell University and a BS from Peking University.