Skip to content

Body Tracking and trx file specifications

Jump to bottom
Allen Lee edited this page Feb 7, 2019 · 17 revisions

How to input body tracking data:

  1. Select 'Has body tracking' when creating a project
  2. Select the tracking file in trx format (see below) for each movie when loading it into APT. Body tracking must include the approximate x,y position of the body center and may include the body orientation (theta).

In APT pre-calculated body tracking data can be used for:

1. Labeling

To keep the currently selected target in the center of field of view (View Menu> Center on target) and to keep the target in the same orientation in the field of view for ease of labeling (View Menu > Rotate video so target always pointed up).

2. Cropping and/or rotating images for tracking and training

Using pre-calculated body tracking allows APT to crop and/or rotate the image around the tracked animals to improve tracking performance. Conceptually, this creates a target animal ROI by cropping a piece of the image centered around the tracked x,y position, with the cropped region optionally rotated to be aligned with the animal's body axis. These transformations can improve tracking in a few ways:

  1. Cropping reduces the size of the image the algorithm needs to search over. We have found this to be very important. Centering the cropped image on a single animal is also essential for part tracking of individuals in movies with multiple animals.
  2. Orientation alignment removes "orientation" from what the tracker needs to learn by pre-rotating the image to align with the body axis of the animal.

NB APT can also do image cropping around a fixed target position using a bounding box without body tracking. See Cropping Movies

How to set tracking parameters to use pre-calculated body tracking:

In Tracking Parameters > Image Processing > Body Tracking > Target ROI (You must have selected 'has Body Tracking' when creating your project to see these options)

If you have x,y, but not theta tracked:

  1. Unselect Align using trajectory theta
  2. Set a Radius (pixels) that will create an ROI (region of interest) box big enough to contain your entire target animal in all poses. This parameter sets the radius (one-half the side length) of this square ROI. The center of the ROI is defined to be (trx.x,trx.y) for the target trajectory.

If you have x,y, theta tracked:

  1. Select Align using trajectory theta
  2. Set a Radius (pixels)

For CPR tracking algorithm only set these additional parameters:

In CPR > Rotational Invariance

  • If you have animal(s) whose orientation is constant relative to the camera within and across movies (such as head fixation): Set Movie Orientation to fixed
  • In all other cases, where you have either freely moving animal(s) or constrained animal(s) whose orientation nonetheless can vary within or across movies, set Movie Orientation to arbitrary

3. Neighbor Masking

Note, Neighbor-masking functionality is experimental.

Trx File Format

APT expects body tracking data in the trx file format which is a MATLAB .mat file containing a structure array, trx. The trx is an array of structs with an element for each animal (i.e., length(trx) is the number of animals/trajectories). The struct for each trajectory, trx(traj), has the following member variables:

  • x : x-coordinate of the animal in pixels (1 x nframes). Thus, trx(traj).x(i) is the x-position of animal in the ith frame of animal's trajectory (movie frame trx(traj).firstframe + i - 1).
  • y : y-coordinate of the animal in pixels (1 x nframes).
  • theta : Orientation of the animal (head) in radians (1 x nframes).
  • a : 1/4 of the major-axis length in pixels (1 x nframes).
  • b : 1/4 of the minor-axis length in pixels (1 x nframes).
  • nframes : Number of frames in the trajectory of the current animal (scalar).
  • firstframe : First frame of the animal's trajectory (scalar).
  • endframe : Last frame of the animal's trajectory (scalar).
  • off : Offset for computing index into x, y, etc. Always equal to 1 - firstframe (scalar).

If theta, a, or b don't exist for you data they can be replaced with NaNs. In these cases: (theta missing) theta alignment, and (theta,a,b missing) Neighbor Masking can not be used.

See below for an example of trx struct for a movie with 1500 frames and two tracked trajectories. The first trajectory has data for the entire movie. The second trajectory is tracked for the second half of the movie (frames 750:1500).

trx(1) = 
          x: [1×1500 double]
          y: [1×1500 double]
          a: [1×1500 double]
          b: [1×1500 double]
      theta: [1×1500 double]
 firstframe: 1
   endframe: 1500
    nframes: 1500
        off: 0 


 trx(2) = 
          x: [1×751 double]
          y: [1×751 double]
          a: [1×751 double]
          b: [1×751 double]
      theta: [1×751 double]
 firstframe: 750
   endframe: 1500
    nframes: 751
        off: -749
Clone this wiki locally