GeoComp demos

Lidar based people surveillance with tracking, biometric identification and activity recognition

We proposed new approaches for gait and activity analysis based on data streams of a Rotating Multi Beam (RMB) Lidar sensor. The introduced algorithms are embedded into an integrated 4D vision and visualization system, which is able to analyze and interactively display real scenarios in natural outdoor environments with walking pedestrians. The main focus of the investigations are gait based person re-identification during tracking, and recognition of specific activity patterns such as bending, waving, making phone calls and checking the time looking at wristwatches. The descriptors for training and recognition are observed and extracted from realistic outdoor surveillance scenarios, where multiple pedestrians are walking in the field of interest following possibly intersecting trajectories, thus the observations might often be affected by occlusions or background noise. Since there is no public database available for such scenarios, we created and published a new Lidar-based outdoor gait and activity dataset on our website, that contains point cloud sequences of 28 different persons extracted and aggregated from 35 minutes-long measurements. The presented results for Velodyne HDL64 [1,2,3,5], and Velodyne VLP16 [4] sensors confirm that both efficient gait-based identification and activity recognition is achievable in the sparse point clouds of a single RMB Lidar sensor. After extracting the people trajectories, we synthesized a free-viewpoint video, where moving avatar models follow the trajectories of the observed pedestrians in real time, ensuring that the leg movements of the animated avatars are synchronized with the real gait cycles observed in the Lidar stream.

 

Tracking multiple pedestrians in 3D [1], and silhouette projection to 2D [2]

 

Person re-identification using biometric gait features [2,3,4,5]

 

Recognizing various activities from point cloud sequences [4,5]

Benchmark dataset

Download Ground Truth data: SZTAKI-LGA Database

References:

[5]Cs. Benedek, B. Gálai, B. Nagy and Z. Jankó: ”Lidar-based Gait Analysis and Activity Recognition in a 4D Surveillance System,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 28, no. 1, pp. 101-113, 2018, IF: 4.046

[4]B. Gálai, and Cs. Benedek: "Gait Recognition with Compact Lidar Sensors", International Conference on Computer Vision Theory and Applications (VISAPP), Porto, Portugal, 27 February - 1 March, 2017

[3] B. Gálai and C. Benedek: "Feature selection for Lidar-based gait recognition", International Workshop on Computational Intelligence for Multimedia Understanding (IWCIM), Prague, Czech Republic, October 29-30, 2015

[2] C. Benedek, B. Nagy, B. Gálai and Z. Jankó: "Lidar-based Gait Analysis in People Tracking and 4D Visualization", European Signal Processing Conference (EUSIPCO), Nice, France, August 31-September 4, 2015

[1] C. Benedek, "3D People Surveillance on Range Data Sequences of a Rotating Lidar," Pattern Recognition Letters, Special Issue on Depth Image Analysis vol. 50, pp. 149–158, 2014