**Weiyao Lin, Yang Zhou, Hongteng Xu, Junchi Yan, Mingliang Xu, Jianxin Wu, Zicheng Liu**

Department of Electronic Engineering,

Shanghai Jiao Tong University, China

wylin@sjtu.edu.cn

Trajectory analysis is essential in many applications. In this paper, we address the problem of representing motion trajectories in a highly informative way, and consequently utilize it for analyzing trajectories. Our approach first leverages the complete information from given trajectories to construct a thermal transfer field which provides a context-rich way to describe the global motion pattern in a scene. Then, a 3D tube is derived which depicts an input trajectory by integrating its surrounding motion patterns contained in the thermal transfer field. The 3D tube effectively: 1) maintains the movement information of a trajectory, 2) embeds the complete contextual motion pattern around a trajectory, 3) visualizes information about a trajectory in a clear and unified way. We further introduce a droplet-based process. It derives a droplet vector from a 3D tube, so as to characterize the high-dimensional 3D tube information in a simple but effective way. Finally, we apply our tube-and-droplet representation to trajectory analysis applications including trajectory clustering, trajectory classification & abnormality detection, and 3D action recognition. Experimental comparisons with state-of-the-art algorithms demonstrate the effectiveness of our approach.

An example of ambiguous trajectories: The orange, red, and green curves labeled by CP_{1}-CP_{3} indicate three trajectory classes. The black curves labeled by Trajectory A and B are two trajectories from CP_{1} and CP_{2}, respectively. (b) Complete contextual motion patterns of trajectories A and B in (a): Contextual motion patterns are described by the motion flows of all trajectory points in the neighborhood of A and B (Note: We use color to differentiate motion flows from different trajectory classes only for a clearer illustration. In our approach, we do not differentiate motion flows' classes and directly use all motion flows in the neighborhood of an input trajectory to model its contextual motion pattern).(c) Use our 3D tube to model A and B's complete contextual motion patterns. (Best viewed in color)

This figure shows the framework of the proposed approch.It constructs a scene-specific thermal transfer field via trajectory training data; For a test trajectory sample, it builds a 3D tube based on the constructed thermal transfer field, and generates a feature vector via a droplet process. The obtained feature vector is applicable to various trajectory analysis applications.

Examples of 3D tubes and water droplet results. First row: results by expanding trajectories into 3D spatio-temporal curves; Second row: 3D tube representations for the black input trajectories A-D in; Third row: water droplet results derived from 3D tubes. (Note: In the middle row of (a)-(d), the thickness of a tube is represented by different colors where yellow indicates thick and red indicates narrow. Best viewed in color)

Left: The 3D trajectory of 'horizontal wave' action of a 'left hand' body point. Right: The skeleton sequence for 'horizontal wave' action.

(a) Given 3D trajectories. (c)-(h) Constructed thermal transfer fields for the given 3D trajectories in (a). Note that (c)-(h) correspond to the thermal transfer fields in directions y^{+}, x^{+}, z^{+}, y^{-}, x^{-}, z^{-}, respectively. (b) An illustration of details inside a 3D thermal transfer field when cutting a high-value volume in a transfer field.

After thermal transfer fields are constructed, a tube and a droplet vector is constructed to depict the information of an input trajectory. Due to the 3D feature of input trajectories, the constructed tube is extended to a combination of 3D equipotential surfaces instead of 2D equipotential lines, as in the left. Similarly, the obtained droplet is also extended from a surface to a sphere which is represented by a center point and a set of boundary points located on a closed surface surrounding it (right).

**Table 1**

**Table 2**

(a) ROC curves of different methods when discriminating normal/abnormal trajectories under different abnormality detection thresholds. (Best viewed in color).Confusion matrices of different methods when classifying normal trajectories and detecting abnormal ones (denoted as ‘AB’). (b) GPRF [6]. (c) DTW [37]. (d) Ours.

**Table 3**

**Table 4**

## Dataset

## This dataset includes 300 real-scene trajectories where 200 trajectories are for normal activities and the other 100 trajec- tories are abnormal ones. The normal trajectories includes seven classes, with about 30 trajectories for each class.

## Left: Example trajectories of the TRAFFIC dataset (yellow curves: normal trajectories, red curves: abnormal trajectories). Right: Major motion routes of different trajectory classes. The yellow and red curves indicate routes for normal and abnormal trajectory classes (‘U’,‘D’,‘L’,‘R’ refers to ‘upward’,‘downward’,‘leftward’,‘rightward’ respectively. For example, ‘LU’ means moving leftward and then upward).

TRAFFIC Dataset

## Code

3D-tube Code

## Citation

## Lin, Weiyao, et al. "A Tube-and-Droplet-based Approach for Representing and Analyzing Motion Trajectories." IEEE Transactions on Pattern Analysis and Machine Intelligence, DOI: 10.1109/TPAMI.2016.2608884, 2016.

Institute of Media, Information, and Network (MIN Lab)

沪交ICP备20160059