User Tools

Site Tools


10a.002.TAU_WP3 - Early Anomaly Recognition System

Project - Summary

The rapid outbreak of the Coronavirus Disease 2019 (COVID-19) has imposed restrictions on people’s movement and daily life [1]. Reducing the spread of the virus mandates constraining social interactions, traveling, and access to public areas and events [1]. These limitations arise to mainly advocate social distancing; the practice of increasing physical space among people to minimize virus transmission [2]. Monitoring and maintaining social distancing are carried out by governmental bodies and agencies using mass surveillance systems and closed-circuit television (CCTV) cameras [3]. Nonetheless, this task is cumbersome and suffers from subjective interpretations and human error due to fatigue; hence, computer vision and machine learning tools are convenient for automation [4]. In addition, they enable crowd behavior to be monitored and anomalies such as congested regions, curfew infractions, and illegal gatherings to be recognized. The widespread of mass surveillance and its integration with Machine Learning is hindered by ethical concerns, including possible breach of privacy and potential abuse [3]. Therefore, privacy-preserving surveillance and Machine Learning solutions are paramount to their ethical adoption and application [5]. The design of vision-based social distance estimation and crowd monitoring system deals with the following challenges [4]: (1) geometry understanding, in terms of ground plane identification and homography estimation; (2) multiple people detection and localization; and (3) statistical/temporal characterization for social distance infractions, e.g., short-term violations are irrelevant. Currently, Machine Learning-based solutions identify social distance infringements using off-the-shelf person detection and tracking models [4]. In general, the models’ performance is conjoined with privacy; they yield high performance by carrying and processing person-specific information to develop robustness against occlusions and missing data [4]. In addition, they localize human subjects via bounding boxes that can be over-sized or incomplete which results in significant distance estimation errors [6]. Therefore, we developed a privacy-preserving adaptive social distance estimation and crowd monitoring system that can be implemented on top of any existing CCTV infrastructure. Specifically, we designed a novel person localization strategy through pose estimation, built a privacy-preserving adaptive smoothing and tracking model to mitigate occlusions and noisy/missing measurements, computed inter-personal distances in the real-world coordinates, detected social distance infractions, and identified overcrowded regions in a scene. Performance evaluation was carried out by testing the system’s ability in person detection, localization, density estimation, anomaly recognition, and high-risk areas identification. We compared the proposed system to the latest techniques and examined the performance gain delivered by the localization and smoothing/tracking algorithms. Experimental results indicated a considerable improvement, across different metrics, when utilizing the developed system. In addition, they showed its potential and functionality for applications other than social distancing. In brief, the main contributions of this project are as follows: (1) developing a robust person localization strategy using pose estimation techniques; (2) forming an adaptive smoothing and tracking paradigm to mitigate the problem of occlusions and missing data without compromising privacy; (3) designing a real-time privacy-preserving social distance estimation and crowd monitoring solution with potential to cover other application areas and tasks.

Project - Team

Team Member Role Email Phone Number Academic Site/IAB
Moncef Gabbouj PI 358 40 073 6613 Tampere University
Serkan Kiranyaz Co-PI 97 43 063 5600 Tampere University
Mohammad Al-Sa'd Researcher 358 41 799 3159 Tampere University
Matti Vakkuri Project Mentor N/A Haltian
Tomi Teikko Project Mentor N/A N/A Haltian
Gunnar Hansen Project Mentor N/A N/A Haltian

Project - Novelty of Approach

  • Detecting and identifying threats and abnormal behaviors in video feeds have been a hot topic ever since computer vision algorithms became popular thanks to the advances made in deep learning; however, no convincing real-time solution has been provided up to date. In this project, we designed a real-time crowd monitoring and anomaly detection systems that can be deployed on CPU and GPU machines.
  • Current anomaly detection solutions excel given specific anomalies and conditions. Therefore, we leveraged various optimized techniques to yield a comprehensive social distance estimation and crowd monitoring system.
  • Current anomaly detection systems yield high performance by carrying and processing person-specific information to develop robustness against occlusions and missing data. In addition, they localize human subjects via bounding boxes that can be over-sized or incomplete which results in significant distance estimation errors. We solved these shortcomings by using a pose estimation technique to detect people because it is independent of the subject’s height, width, and orientation and carries no person-specific information; hence, it preserves privacy.

Project - Deliverables

1 Identifying and collecting suitable datasets
2 Proof of concept of the early warning system
3 System integration, verification and publication of the results

Project - Benefits to IAB

The COVID-19 outbreak imposed immediate needs for such a comprehensive surveillance and tracking system for indoors and outdoors. The application domains include both surveillance and empathic buildings development. Moreover, apart from the project direct application to anomaly detection, social distance estimation, and crowd monitoring, the designed system occupancy/crowd density map functionality extends its application domain beyond the COVID-19 pandemic to cover other areas. For instance, it can help re-configure or re-design common physical layouts and relocate facilities in businesses to optimally reduce congestion. Additionally, it can facilitate the analysis of customer’s browsing habits in shops and quantifying the effectiveness of marketing kiosks.

Project - Documents

projects/year10/10a.002.tau_wp3.txt · Last modified: 2022/10/03 14:57 by sally.johnson