User Tools

Site Tools


10a.001.UL_TAU - AnonyTrack: A Privacy-Aware People Localization and Tracking Within Buildings with Multi-Modal Learning

Project - Summary

Indoor Positioning and Localization are essential to provide location-based services in smart building environments. The choice of indoor position technology is usually a tradeoff between various factors such as cost, localization performance, and privacy. The general idea of multi-modal localization is to combine more than one modality to provide better localization performance. In this project, we propose multi-modal approach that combines BLE and computer vision techniques with the goal to track mobile phones anonymously. We apply Siamese network-based method to map BLE and computer vision trajectories. The proposed system overcomes the limitation of BLE technologies in terms of improving its location precision as a result of using trajectories generated from computer vision. We evaluated the localization performance of the proposed multi-modal approach through rigorous experiments on small restricted indoor spaces with various movement dynamics. Our proposed MIPS system achieves an average accuracy of 95.16\% and an error of less than 39.53 cm for all five scenarios with a tracking window of 5 seconds.

Project - Team

Team Member Role Email Phone Number Academic Site/IAB
Raju Gottumukkala PI (318) 680-5886 UL Lafayette
Moncef Gabbouj Co-PI 358 40 073 6613 Tampere University
Satya Katradgadda Research Scientist N/A N/A UL Lafayette
Ravi Teja Bhupatiraju Research Scientist N/A N/A UL Lafayette
Seyed Majid Hosseini Student N/A N/A UL Lafayette
Jake Guidry Student N/A N/A UL Lafayette
Soon Jynn Chu Student N/A N/A UL Lafayette
Matti Vakkuri Project Mentor 358 40 5126 894 Haltian
Robert Boland Project Mentor (215) 837-2642 CSL Behring
John Thompson Project Mentor N/A N/A CSL Behring

Project - Novelty of Approach

Conventional indoor tracking systems can be divided into two different categories non-video and video-based tracking models. Computer vision tracking systems trade of the privacy to accuracy and are not able to connect to the individual. However, the proposed model is able balancing cost, privacy, accuracy, speed) to solve the problem of real-time privacy-aware tracking and localization of individuals. To the best of our knowledge, there is no literature combining BLE low energy signal and video for indoor localization. The proposed model is robust in the absence of one or some modalities.

Moreover, the system can track the mobile devices over time and self-correct when new tracking IDs are generated due to ID switching or Occlusions due to the Siamese mapping.

Project - Deliverables

1 Location analytics for people tracking
2 Develop a multimodal location detection model to detect location of users
3 Develop a comprehensive dataset for multimodal human tracking with multiple sensors and occlusions
4 Share our results with IAB partners
5 Publish our analysis and the laboratory datasets

Project - Benefits to IAB

This is the first work integrating BLE and computer vision for indoor localization and tracking using machine learning using Siamese networks. The BLE approach uses standard machine learning-based fingerprinting, and the computer vision-based IPS uses YOLOv3-tiny-based object detection and tracking approaches. Our proposed decision level fusion uses Spatio-temporal trajectory information to match the location estimates using Siamese networks.

  • The proposed MIPS uses multi-modal localization and tracking with commercial BLE beacons, an overhead camera, and a mobile phone-based app. The edge device can process the data streams close to real-time and can anonymously communicate with the user while not uniquely identifying the user.
  • We experimentally validated the feasibility of integrating BLE signals and video streams for indoor localization in small spaces with multiple users. This system was extensively tested for various scenarios to understand the performance implications of increasing the number of people and mobility dynamics.

Project - Documents

projects/year10/10a.001.ul.txt · Last modified: 2022/10/04 09:11 by sally.johnson