Skip to content

Multiview Calibrated Labeling

Jump to bottom
Allen Lee edited this page Dec 5, 2018 · 18 revisions

Background

APT enables labeling videos taken from multiple calibrated cameras.

From a single labeled point in any given view, epipolar lines can be generated and visualized in all other views.

From any pair of labeled points in a pair of views, best-guess estimates can be made for the projected points on all other views.

Requirements

  • Support is built into APT for two-camera configurations, but any number of views is supported by the framework. We have done work for example with a 3-camera rig.
  • Movies for all views must be synchronized and have the same number of frames. (Update 2018: the latter requirement may now be relaxed.)
  • Currently, explicit support is provided for pairs of cameras calibrated using:

Usage

To perform calibrated labeling, first take calibration images and perform camera calibration using one of the methods listed above. The final result of the calibration process should be a saved MAT-file containing calibration information for your cameras/rig.

Create a new project, entering the appropriate number of views for your rig. Add "moviesets", or nview-tuples of matched movies.

Use the Label>Load Calibration File menu option to load your previously-generated calibration file. You may choose to have this calibration apply to all movies in the project, or just the current movie. (In the latter case, you will need to load calibration data for every movie in the project. See below if using the MATLAB Stereo Calibration App.)

You should now be able to perform multiview calibrated labeling! See for instance the "Multiview Calibrated 2" section in /README.md.

Notes: MATLAB Stereo Calibration App

When APT first loads a stereo calibration file generated by the MATLAB Stereo Calibration App, it creates a wrapper object for use within APT. At creation-time, this object (of class 'CalRigMLStro') performs various self-calibrations and diagnostic tests. When these procedures are complete, you have the opportunity to save the self-calibrated CalRigMLStro object. We highly recommend this so that you can re-use this object again in other projects, or for other movies within a project!

Once you have saved the wrapper object to a MAT-file, the next time you use the Label>Load Calibration File menu option, you can specify this saved file. Otherwise, you can specify the original calibration file from the MATLAB Stereo Calibration App; the downside of this is that you will have to wait again for the self-calibration to be performed.

Clone this wiki locally