A Tube-and-Droplet-based Approach for Representing and Analyzing Motion Trajectories


Weiyao Lin, Yang Zhou, Hongteng Xu, Junchi Yan, Mingliang Xu, Jianxin Wu, Zicheng Liu


Department of Electronic Engineering,

Shanghai Jiao Tong University, China






Trajectory analysis is essential in many applications. In this paper, we address the problem of representing motion trajectories in a highly informative way, and consequently utilize it for analyzing trajectories. Our approach first leverages the complete information from given trajectories to construct a thermal transfer field which provides a context-rich way to describe the global motion pattern in a scene. Then, a 3D tube is derived which depicts an input trajectory by integrating its surrounding motion patterns contained in the thermal transfer field. The 3D tube effectively: 1) maintains the movement information of a trajectory, 2) embeds the complete contextual motion pattern around a trajectory, 3) visualizes information about a trajectory in a clear and unified way. We further introduce a droplet-based process. It derives a droplet vector from a 3D tube, so as to characterize the high-dimensional 3D tube information in a simple but effective way. Finally, we apply our tube-and-droplet representation to trajectory analysis applications including trajectory clustering, trajectory classification & abnormality detection, and 3D action recognition. Experimental comparisons with state-of-the-art algorithms demonstrate the effectiveness of our approach.




Basic idea



An example of ambiguous trajectories: The orange, red, and green curves labeled by CP1-CP3 indicate three trajectory classes. The black curves labeled by Trajectory A and B are two trajectories from CP1 and CP2, respectively. (b) Complete contextual motion patterns of trajectories A and B in (a): Contextual motion patterns are described by the motion flows of all trajectory points in the neighborhood of A and B (Note: We use color to differentiate motion flows from different trajectory classes only for a clearer illustration. In our approach, we do not differentiate motion flows' classes and directly use all motion flows in the neighborhood of an input trajectory to model its contextual motion pattern).(c) Use our 3D tube to model A and B's complete contextual motion patterns. (Best viewed in color)




Framework of the proposed approach




This figure shows the framework of the proposed approch.It constructs a scene-specific thermal transfer field via trajectory training data; For a test trajectory sample, it builds a 3D tube based on the constructed thermal transfer field, and generates a feature vector via a droplet process. The obtained feature vector is applicable to various trajectory analysis applications.



3D tubes example



Examples of 3D tubes and water droplet results. First row: results by expanding trajectories into 3D spatio-temporal curves; Second row: 3D tube representations for the black input trajectories A-D in; Third row: water droplet results derived from 3D tubes. (Note: In the middle row of (a)-(d), the thickness of a tube is represented by different colors where yellow indicates thick and red indicates narrow. Best viewed in color)




Extend to 3D action recognition



An example of 3D skeleton sequence from MSR-Action3D dataset



Left: The 3D trajectory of 'horizontal wave' action of a 'left hand' body point. Right: The skeleton sequence for 'horizontal wave' action.



An example of thermal transfer fields for 3D trajectories



(a) Given 3D trajectories. (c)-(h) Constructed thermal transfer fields for the given 3D trajectories in (a). Note that (c)-(h) correspond to the thermal transfer fields in directions y+, x+, z+, y-, x-, z-, respectively. (b) An illustration of details inside a 3D thermal transfer field when cutting a high-value volume in a transfer field.



An example of equipotential surfaces (left) and the resulting water droplet sphere (right) for 3D trajectories.



After thermal transfer fields are constructed, a tube and a droplet vector is constructed to depict the information of an input trajectory. Due to the 3D feature of input trajectories, the constructed tube is extended to a combination of 3D equipotential surfaces instead of 2D equipotential lines, as in the left. Similarly, the obtained droplet is also extended from a surface to a sphere which is represented by a center point and a set of boundary points located on a closed surface surrounding it (right).






(1) Trajectory Clustering


Table 1

Table1 compares the cluster learning accuracy for different methods on VMT dataset, where the cluster learning accuracy measures the total percentage of trajectories being correctly clustered.


Table 2

Cluster Learning Accuracy with different noise or trajectory break levels on VMT Dataset.



(2) Trajectory Classification & Abnormality Detection




(a) ROC curves of different methods when discriminating normal/abnormal trajectories under different abnormality detection thresholds. (Best viewed in color).Confusion matrices of different methods when classifying normal trajectories and detecting abnormal ones (denoted as ‘AB’). (b) GPRF [6]. (c) DTW [37]. (d) Ours.


Table 3

Trajectory classification & abnormality detection results on CROSS (%)




(3) 3D Action Recognition


Table 4

Recognition accuracy comparison on MSR-Action3D dataset (%)





This dataset includes 300 real-scene trajectories where 200 trajectories are for normal activities and the other 100 trajec- tories are abnormal ones. The normal trajectories includes seven classes, with about 30 trajectories for each class.



Left: Example trajectories of the TRAFFIC dataset (yellow curves: normal trajectories, red curves: abnormal trajectories). Right: Major motion routes of different trajectory classes. The yellow and red curves indicate routes for normal and abnormal trajectory classes (‘U’,‘D’,‘L’,‘R’ refers to ‘upward’,‘downward’,‘leftward’,‘rightward’ respectively. For example, ‘LU’ means moving leftward and then upward).







3D-tube Code



Lin, Weiyao, et al. "A Tube-and-Droplet-based Approach for Representing and Analyzing Motion Trajectories." IEEE Transactions on Pattern Analysis and Machine Intelligence, DOI: 10.1109/TPAMI.2016.2608884, 2016.





Institute of Media, Information, and Network (MIN Lab)