A Tube-and-Droplet-based Approach for Representing and Analyzing Motion Trajectories

 

Weiyao Lin, Yang Zhou, Hongteng Xu, Junchi Yan, Mingliang Xu, Jianxin Wu, Zicheng Liu

 

Department of Electronic Engineering,

Shanghai Jiao Tong University, China

 

wylin@sjtu.edu.cn

 

 

Abstract

Trajectory analysis is essential in many applications. In this paper, we address the problem of representing motion trajectories in a highly informative way, and consequently utilize it for analyzing trajectories. Our approach first leverages the complete information from given trajectories to construct a thermal transfer field which provides a context-rich way to describe the global motion pattern in a scene. Then, a 3D tube is derived which depicts an input trajectory by integrating its surrounding motion patterns contained in the thermal transfer field. The 3D tube effectively: 1) maintains the movement information of a trajectory, 2) embeds the complete contextual motion pattern around a trajectory, 3) visualizes information about a trajectory in a clear and unified way. We further introduce a droplet-based process. It derives a droplet vector from a 3D tube, so as to characterize the high-dimensional 3D tube information in a simple but effective way. Finally, we apply our tube-and-droplet representation to trajectory analysis applications including trajectory clustering, trajectory classification & abnormality detection, and 3D action recognition. Experimental comparisons with state-of-the-art algorithms demonstrate the effectiveness of our approach.

 

 

 

Basic idea

 

 

An example of ambiguous trajectories: The orange, red, and green curves labeled by CP₁-CP₃ indicate three trajectory classes. The black curves labeled by Trajectory A and B are two trajectories from CP₁ and CP₂, respectively. (b) Complete contextual motion patterns of trajectories A and B in (a): Contextual motion patterns are described by the motion flows of all trajectory points in the neighborhood of A and B (Note: We use color to differentiate motion flows from different trajectory classes only for a clearer illustration. In our approach, we do not differentiate motion flows' classes and directly use all motion flows in the neighborhood of an input trajectory to model its contextual motion pattern).(c) Use our 3D tube to model A and B's complete contextual motion patterns. (Best viewed in color)

 

 

 

Framework of the proposed approach

 

 

This figure shows the framework of the proposed approch.It constructs a scene-specific thermal transfer field via trajectory training data; For a test trajectory sample, it builds a 3D tube based on the constructed thermal transfer field, and generates a feature vector via a droplet process. The obtained feature vector is applicable to various trajectory analysis applications.

 

 

 

 

Examples of 3D tubes and water droplet results. First row: results by expanding trajectories into 3D spatio-temporal curves; Second row: 3D tube representations for the black input trajectories A-D in; Third row: water droplet results derived from 3D tubes. (Note: In the middle row of (a)-(d), the thickness of a tube is represented by different colors where yellow indicates thick and red indicates narrow. Best viewed in color)

 

 

 

Extend to 3D action recognition

 

 

An example of 3D skeleton sequence from MSR-Action3D dataset

 

 

Left: The 3D trajectory of 'horizontal wave' action of a 'left hand' body point. Right: The skeleton sequence for 'horizontal wave' action.

 

 

An example of thermal transfer fields for 3D trajectories

 

 

(a) Given 3D trajectories. (c)-(h) Constructed thermal transfer fields for the given 3D trajectories in (a). Note that (c)-(h) correspond to the thermal transfer fields in directions y⁺, x⁺, z⁺, y^-, x^-, z^-, respectively. (b) An illustration of details inside a 3D thermal transfer field when cutting a high-value volume in a transfer field.

 

 

An example of equipotential surfaces (left) and the resulting water droplet sphere (right) for 3D trajectories.

 

 

After thermal transfer fields are constructed, a tube and a droplet vector is constructed to depict the information of an input trajectory. Due to the 3D feature of input trajectories, the constructed tube is extended to a combination of 3D equipotential surfaces instead of 2D equipotential lines, as in the left. Similarly, the obtained droplet is also extended from a surface to a sphere which is represented by a center point and a set of boundary points located on a closed surface surrounding it (right).

 

 

 

Results

 

(1) Trajectory Clustering

 

Table 1

Table1 compares the cluster learning accuracy for different methods on VMT dataset, where the cluster learning accuracy measures the total percentage of trajectories being correctly clustered.

 

Table 2

Cluster Learning Accuracy with different noise or trajectory break levels on VMT Dataset.

 

 

(2) Trajectory Classification & Abnormality Detection

 

 

 

(a) ROC curves of different methods when discriminating normal/abnormal trajectories under different abnormality detection thresholds. (Best viewed in color).Confusion matrices of different methods when classifying normal trajectories and detecting abnormal ones (denoted as ‘AB’). (b) GPRF [6]. (c) DTW [37]. (d) Ours.

 

Table 3

 

 

 

(3) 3D Action Recognition

 

Table 4

Recognition accuracy comparison on MSR-Action3D dataset (%)