Online Security Analytics on Large Scale Video Surveillance System - Databricks

Online Security Analytics on Large Scale Video Surveillance System

Download Slides

Video surveillance is critical to public security. The fast-growing scale of online cameras fosters automated and real-time video analytics across multiple video streams.

We build a large-scale video security analytics platform, with Spark at the core, which online recognizes, classifies, stores and indexes humans/faces in cameras, so as to real-time detect suspected humans/faces in pre-defined blacklist, identify abnormal/unsafe user behaviors based on our home-grown security analytics model, as well as answer ad-hoc security questions such as “search all locations a suspect appeared”.

This talk will share our recent efforts and experience with Spark ecosystem to realize such platform. We equip Spark with rich video processing/analytics capabilities. We leverage Spark Streaming to real-time recognize and classify humans/faces, and utilize deep learning, with DeepDist, to extract visual features. We enable fast human/face search and security analytics at scale with the power of Spark execution engine, novel visual feature indexing, and multi-tier video data storage atop Tachyon, SparkSQL and HDFS.

Related Articles:

  • Fighting Cybercrime: A Joint Task Force of Real-Time Data and Human Analytics
  • Deep Learning in Security—An Empirical Example in User and Entity Behavior Analytics (UEBA)
  • Improving Threat Detection in a Big Data World


  • « back
    About Yu Cao

    Yu is a senior manager at EMC, where he works at a combo role of data scientist, solution architect, and technical leader on customer specific Big Data solution innovation for verticals like social intelligence, smart-grid, video surveillance and healthcare, as well as applied research like infrastructure (IaaS, PaaS, SDDC) analytics and software-defined storage intelligence. He also works on collaborative relationships between EMC and the academic community across APJ region. He holds a Ph.D. in computer science from National University of Singapore.

    About Xiaoyan Guo

    Xiaoyan Guo is an associate principal research scientist at EMC. His big data expertise covers verticals like video surveillance, social intelligence, internet of vehicle, etc. As a passionate data scientist, his most recent work focuses on large scale video analytics to process and understand huge volumes of video data. Additionally, he works on advanced and applied research on big data partitioning and Containers as a Service (CaaS). Xiaoyan holds a M.E in computer science from Renmin University of China.