Tracking multiple articulating humans from a single camera

Multi-target tracking aims at locating multiple targets in frames, maintaining their identities and estimating their motion trajectories over time, which has been an active research topic in the field of computer vision in past few decades. With a large amount of video data generated at every moment, it is highly demanded to improve artificial visual intelligence capacities which can automatically analyze videos and understand scenes without human involvement. It has tremendous applications such as surveillance, autonomous vehicles and human-computer interaction. ❧ Among multi-target tracking problems, monocular multi-target tracking using video data from a single camera view is critical and fundamental to others with multiple camera views. In particular, humans are often the most concerned targets as daily activities and events in real scenes usually involve human participants. In this thesis, we target at improving monocular multi-target tracking on not only pedestrians but also articulating humans. ❧ Even though some fairly significant advances have been made on pedestrian tracking in recent years, the problem of tracking multiple humans with large articulating poses or in crowded scenes is still far from solved. Unlike well-studied pedestrian detection, generic articulated human detection in a wild scene remains a challenging problem. As multi-target tracking problem typically requires detection input from offline-trained detectors, large miss detection and false alarm rates on articulated humans make the existing tracking approaches less effective on articulating humans than pedestrians. In common crowded scenes, where human bodies are partially visible and frequent occlusions exist, appearance and motion cues are weakened; however, social context cues become more informative and can be explored to help tracking, as humans may move in groups, follow others' paths or repulse from obstacles. ❧ In order to track each single target with large pose variations, we first introduced a part-based appearance model with superpixel-based features and proposed an instance-specific tracker. For common crowded scenes, we proposed to online learn discriminative appearance features of each target and consider social context to improve tracking performance. ❧ Different from the linear simplification in classic tracking optimization formulations by ignoring motion dependency among targets, we proposed a new optimization framework that can consider various pairwise social dependencies to improve tracking. To ensure the tracking efficiency, we introduced an approach to convert the new binary quadratic programming formulation to a semidefinite programming problem under convex relaxation for efficient solution. Besides, we propose to infer simple motion dependency factors online efficiently. In scenarios where no trajectory dependency can be explored, our solution is the same and as efficient as classic linear optimization formulations. In general, the new formulation provides a way to consider various high order context to improve multi-target tracking. ❧ To address the problem of tracking multiple articulating humans, we proposed a hybrid approach. Our method incorporates offline learned category-level detector with online learned instance-specific detector as a hybrid system. To deal with humans in large pose articulation, which can not be reliably detected by off-line trained detectors, we propose an online learned instance-specific patch-based detector, consisting of layered patch classifiers. With extrapolated tracklets by online learned detectors, we use the discriminative color filters learned online to compute the appearance affinity score for further global association. ❧ Experimental evaluation on both standard pedestrian datasets and articulated human datasets shows significant improvement compared to state-of-the-art multi-human tracking methods.