Xiaoyan Guo is an associate principal research scientist at EMC. His big data expertise covers verticals like video surveillance, social intelligence, internet of vehicle, etc. As a passionate data scientist, his most recent work focuses on large scale video analytics to process and understand huge volumes of video data. Additionally, he works on advanced and applied research on big data partitioning and Containers as a Service (CaaS). Xiaoyan holds a M.E in computer science from Renmin University of China.
Video surveillance is critical to public security. The fast-growing scale of online cameras fosters automated and real-time video analytics across multiple video streams. We build a large-scale video security analytics platform, with Spark at the core, which online recognizes, classifies, stores and indexes humans/faces in cameras, so as to real-time detect suspected humans/faces in pre-defined blacklist, identify abnormal/unsafe user behaviors based on our home-grown security analytics model, as well as answer ad-hoc security questions such as "search all locations a suspect appeared". This talk will share our recent efforts and experience with Spark ecosystem to realize such platform. We equip Spark with rich video processing/analytics capabilities. We leverage Spark Streaming to real-time recognize and classify humans/faces, and utilize deep learning, with DeepDist, to extract visual features. We enable fast human/face search and security analytics at scale with the power of Spark execution engine, novel visual feature indexing, and multi-tier video data storage atop Tachyon, SparkSQL and HDFS.