Skip to content

hailanyi/3D-Multi-Object-Tracker

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

48 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

3D Multi-Object Tracker

This project is developed for tracking multiple objects in 3D scene. The visualization code is from here. This is a re-produced and simplified version of paper. We have made some changes to support more detectors and adapt the HOTA metric. We also provided a simple guide to submit results to the KITTI test set.

Features

  • Fast: currently, the codes can achieve 700 FPS using only CPU (not include detection and data op), can perform tracking on all kitti val sequence in several seconds.
  • Support online, near online and global implementation. The overall framework of design is shown below:

Kitti Results

Car/Pedestrian tracking results on the Kitti tracking val seq [1,6,8,10,12,13,14,15,16,18,19] using second-iou, point-rcnn and pv-rcnn detections. We also followed the HOTA metric, and tuned the parameters by first considering the HOTA performance.

detector online near online (latency=2s) global
point-rcnn 76.653 77.576 78.853
second-iou 77.29/48.011 78.17/50.362 78.799/51.20
pv-rcnn 78.289/49.662 79.48/50.824 80.075/51.753

Online: CA-based KF + greedy matching. Near online: online + rescoring tracks in a temporal window. Global: online + rescoring tracks globally.

  • 2022/9/23 Update CasTrack. The detections are obtained by CasA detector trained on KITTI 3D detection trainval set. Currently rank first on the KITTI tracking dataset 🔥!
Detector online near online (latency=2s) global detections (training & testing set)
CasA 82.59 83.24 83.34 detections download link(40M)
  • Update VirConvTrack (2023/3/7), the detections are obtained by VirConv-T detector trained on KITTI 3D detection trainval set.
Detector online near online (latency=2s) global detections (training & testing set)
VirConv 85.40 85.98 86.17 detections download link(8M)

Waymo Results

Vehicle/Pedestrian/Cyclist online tracking results, evaluated by MOTA/L2 metrics of Waymo benchmark.

  • 2022/9/23 Update CasTrack-waymo, currently ranks third on Waymo tracking benchmark 🔥! You can download the CasA detections from here (Please register with waymo before downloading, as the Waymo open dataset is under strict non-commercial license).
detector set Vehicle_L2 Pedestrian_L2 Cyclist_L2 ALL_NS_L2
CasA val set 59.30 62.78 61.72 61.27
CasA test set 63.66 64.79 59.34 62.60

Prepare data

You can download the Kitti tracking pose data from here, and you can download the point-rcnn, second-iou and pv-rcnn detections from here. You can download the CasA detections(including training & testing set) from here.

To run this code, you should organize Kitti tracking dataset as below:

# Kitti Tracking Dataset       
└── kitti_tracking
       ├── testing 
       |      ├──calib
       |      |    ├──0000.txt
       |      |    ├──....txt
       |      |    └──0028.txt
       |      ├──image_02
       |      |    ├──0000
       |      |    ├──....
       |      |    └──0028
       |      ├──pose
       |      |    ├──0000
       |      |    |    └──pose.txt
       |      |    ├──....
       |      |    └──0028
       |      |         └──pose.txt
       |      ├──label_02
       |      |    ├──0000.txt
       |      |    ├──....txt
       |      |    └──0028.txt
       |      └──velodyne
       |           ├──0000
       |           ├──....
       |           └──0028      
       └── training # the structure is same as testing set
              ├──calib
              ├──image_02
              ├──pose
              ├──label_02
              └──velodyne 

Detections

└── point-rcnn
       ├── training
       |      ├──0000
       |      |    ├──000001.txt
       |      |    ├──....txt
       |      |    └──000153.txt
       |      ├──...
       |      └──0020
       └──testing 

Requirements

python3
numpy
opencv
yaml

Quick start

  • Please modify the dataset path and detections path in the yaml file to your own path.
  • Then run python3 kitti_3DMOT.py config/online/pvrcnn_mot.yaml
  • The results are automatically saved to evaluation/results/sha_key/data, and evaluated by HOTA metrics.

Notes

The evaluation codes are copied from Kitti.

Releases

No releases published

Packages

No packages published

Languages