Choreography Dataset

More information: https://flowers.inria.fr/choreo/doc/

<strong>Presentation</strong>

This database contains choreography motions recorded through a kinect device. These motions have a combinatorial structure: from a given set of primitive dance motions, choreographies are constructed as simultaneous execution of some of these primitive motions.

Primitive dance motions are chosen from a total set of 48 motions and are spanned over one or two limbs, either the legs (e.g. <em>walk</em>, <em>squat</em>), left or right arm (e.g. <em>wave hand</em>, <em>punch</em>) or both arms (e.g. <em>clap in hands</em>, <em>paddle</em>).

Complex choreographies are produced as the simultaneous demonstration of two or three of these primitive motion: either one for legs and one for both arm, or one for legs and one for each arm.

Each example (or record) contained in the dataset consists in two elements:

the motion data,
labels identifying which primitive motions are combined to produce the choreography.

3 separate sets of examples are included in this dataset:

<strong>primitive</strong>: in each example, only one primitive motion is demonstrated, the set of labels associated to each example is thus a singleton (326 examples)
<strong>mixed small</strong>: demonstrations of complex choreographies composed of primitive motions taken in a subset of 16 possible motions (137 examples).
<strong>mixed full</strong>: demonstrations of complex choreographies composed of primitive motions taken in all the possible motions (277 examples).

<strong>Description of the data</strong>

The data has been acquired through a kinect camera and the OpenNI drivers, which yields a stream of values of markers on the body.

Each example from the dataset is associated to a sequence of 3D positions of each of the 24 markers. Thus for a sequence of length T, the example would corresponds to T*24*3 values.

The kinect device recognizes the following list of markers:

<em>head</em>, <em>neck</em>, <em>waist</em>, <em>left_hip</em>, <em>left_shoulder</em>, <em>left_elbow</em>, <em>left_hand</em>, <em>left_knee</em>, <em>left_foot</em>, <em>left_collar</em>, <em>left_wrist</em>, <em>left_fingertip</em>, <em>left_ankle</em>, <em>right_hip</em>, <em>right_shoulder</em>, <em>right_elbow</em>, <em>right_hand</em>, <em>right_knee</em>, <em>right_foot</em>, <em>right_collar</em>, <em>right_wrist</em>, <em>right_hand</em>, <em>right_fingertip</em>, <em>right_ankle</em>

These markers are however not tracked with the same accuracy and it might be better to filter to keep only a subset of these markers. For examples the following list is a good start:

<em>head</em>, <em>neck</em>, <em>left_hip</em>, <em>left_shoulder</em>, <em>left_elbow</em>, <em>left_hand</em>, <em>left_knee</em>, <em>left_foot</em>, <em>right_hip</em>, <em>right_shoulder</em>, <em>right_elbow</em>, <em>right_hand</em>, <em>right_knee</em>, <em>right_hand</em>, <em>right_foot</em>

Labels are provided as lists of one (<em>primitive</em> set), or two or three (other sets) identifiers.

A list of primitives and their descriptions can be found at the end of this document.

<strong>Format</strong>

This data is accessible in three data formats:

<em>text</em>
<em>numpy</em>
<em>Matlab</em>

<strong>The text format</strong>

The set of examples consists in:

a json file describing metadata and labels,
a directory containing one text file for each example.

These are distributed in a compressed archive (<em>tar.gz</em>).

An example of a json file is given below. They all have a similar structure.

{ "marker-names": [ "head", "neck", ... ], "data-dir": "mixed_partial_data", "name": "mixed_partial", "records": [ { "data-id": 0, "labels": [ 20, 26 ] }, { "data-id": 1, "labels": [ 19, 28 ] }, ... ] }

It contains the following data:

<em>name</em>: name of the set of examples,
<em>marker-names</em>: list of name of the markers in the same order as they appear in data,
<em>data-dir</em>: path to the data directory,
<em>records</em>: list of records. Each record contains: - a <em>data-id</em> fields, - a <em>labels</em> field containing a list of label as integers.

For each record listed in th json file there exists a text file in the ‘data-dir’ directory, which name is the ‘data-id’ plus a ‘.txt’ extension.

The text files contains the sequence of positions of the marker. Each set of values at a given time is given as a line of space separated floating numbers (formated as ‘5.948645401000976562e+01’).

Each line contains 3 successive values for each marker which are there 3D coordinates, as provided by the OpenNI framework during capture. Thus each line contains 3M values with M the number of markers.

<strong>The numpy format</strong>

In this format each set of examples is described by two files: a json file and a compressed numpy data file (.npz).

The json file is very similar to the one from the text format, the only difference is that the ‘data-dir’ element is replaced by a ‘data-file’ element containing the path to the data file.

The data file is a numpy compressed data file storing one array for each example. The name of the array is given by the ‘data-id’ element. Each data array (one for each record) is of shape (T, M, 3) where T is the length of the example and M the number of markers.

The following code can be used to load a set of example in python:

import os import json import numpy as np FILE = 'path/to/mixed_full.json' with open(FILE, 'r') as meta_file: meta = json.load(meta_file) # meta is a dictionary containing data from the json file path_to_data = os.path.join(os.path.dirname(FILE), meta['data-file']) loaded_data = np.load(path_to_data) data = [] labels = [] for r in meta['records']: data.append(loaded_data[str(r['data-id'])]) # numpy array labels.append(r['labels']) # list of labels as integers print "Loaded %d examples for ``%s`` set." % (len(data), meta['name']) print "Each data example is a (T, %d, 3) array." % len(meta['marker-names']) print "The second dimension corresponds to markers:" print "\t- %s" % '\n\t- '.join(meta['marker-names']) return (data, labels, meta['marker-names'])

<strong>The Matlab format</strong>

In the Matlab format, a set of examples is described by a single ‘.mat’ file containing the following elements:

a ‘name’ variable (string) containing the name of the set of examples,
a ‘marker_names’ variable containing a list of marker names (strings),
a ‘data’ variable containing a list of data arrays (one for each record) of size (T, M, 3) where T is the length of the example and M the number of markers,
a ‘labels’ variable which is a list of list of labels (one list of labels for each example).