Heat-Map-based Algorithm for Group Activity Recognition

Goal

Classify group activities in surveillance videos.
Example: The camera sees a bunch of people, what are they doing? Gathering? Separating? Standing still? Or chasing?

Method

Step 1: Treat each object as a moving heat source, create heat map.
Example: If this is what the camera sees (single object):

tracked video
Then this is what its heat map looks like:
heatmap 2d heatmap 3d
Different types of activity form different heat map shapes, for example 2-object activities:
heatmaps
Step 2: Align heat map.
Example: Original heat maps:
before alignment
Aligned heat maps:
after alignment
Step 3: Classification!
Example: kNN, PCA-LDA, SVM, sparse...

References

[1] Hang Chu, Weiyao Lin*, Jianxin Wu, Xingtong Zhou, Yuanzhe Chen, and Hongxiang Li, A New Heat-Map-based Algorithm for Human Group Activity Recognition, ACM Multimedia (ACM MM), 2012. [pdf]
[2] Hang Chu*, Weiyao Lin*, Jianxin Wu, Bin Sheng, and Zhenzhong Chen, A Heat-Map-based Algorithm for Recognizing Group Activities in Videos, IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2013. [pdf]