This project is developed for tracking multiple objects in 3D scene. The visualization code is from here. This is a re-produced and simplified version of paper. We have made some changes to support more detectors and adapt the HOTA metric. We also provided a simple guide to submit results to the KITTI test set.
- Fast: currently, the codes can achieve 700 FPS using only CPU (not include detection and data op), can perform tracking on all kitti val sequence in several seconds.
- Support online, near online and global implementation. The overall framework of design is shown below:
Car/Pedestrian tracking results on the Kitti tracking val seq [1,6,8,10,12,13,14,15,16,18,19] using second-iou, point-rcnn and pv-rcnn detections. We also followed the HOTA metric, and tuned the parameters by first considering the HOTA performance.
detector | online | near online (latency=2s) | global |
---|---|---|---|
point-rcnn | 76.653 | 77.576 | 78.853 |
second-iou | 77.29/48.011 | 78.17/50.362 | 78.799/51.20 |
pv-rcnn | 78.289/49.662 | 79.48/50.824 | 80.075/51.753 |
Online: CA-based KF + greedy matching. Near online: online + rescoring tracks in a temporal window. Global: online + rescoring tracks globally.
- 2022/9/23 Update CasTrack. The detections are obtained by CasA detector trained on KITTI 3D detection trainval set. Currently rank first on the KITTI tracking dataset 🔥!
Detector | online | near online (latency=2s) | global | detections (training & testing set) |
---|---|---|---|---|
CasA | 82.59 | 83.24 | 83.34 | detections download link(40M) |
- Update VirConvTrack (2023/3/7), the detections are obtained by VirConv-T detector trained on KITTI 3D detection trainval set.
Detector | online | near online (latency=2s) | global | detections (training & testing set) |
---|---|---|---|---|
VirConv | 85.40 | 85.98 | 86.17 | detections download link(8M) |
Vehicle/Pedestrian/Cyclist online tracking results, evaluated by MOTA/L2 metrics of Waymo benchmark.
- 2022/9/23 Update CasTrack-waymo, currently ranks third on Waymo tracking benchmark 🔥! You can download the CasA detections from here (Please register with waymo before downloading, as the Waymo open dataset is under strict non-commercial license).
detector | set | Vehicle_L2 | Pedestrian_L2 | Cyclist_L2 | ALL_NS_L2 |
---|---|---|---|---|---|
CasA | val set | 59.30 | 62.78 | 61.72 | 61.27 |
CasA | test set | 63.66 | 64.79 | 59.34 | 62.60 |
You can download the Kitti tracking pose data from here, and you can download the point-rcnn, second-iou and pv-rcnn detections from here. You can download the CasA detections(including training & testing set) from here.
To run this code, you should organize Kitti tracking dataset as below:
# Kitti Tracking Dataset
└── kitti_tracking
├── testing
| ├──calib
| | ├──0000.txt
| | ├──....txt
| | └──0028.txt
| ├──image_02
| | ├──0000
| | ├──....
| | └──0028
| ├──pose
| | ├──0000
| | | └──pose.txt
| | ├──....
| | └──0028
| | └──pose.txt
| ├──label_02
| | ├──0000.txt
| | ├──....txt
| | └──0028.txt
| └──velodyne
| ├──0000
| ├──....
| └──0028
└── training # the structure is same as testing set
├──calib
├──image_02
├──pose
├──label_02
└──velodyne
Detections
└── point-rcnn
├── training
| ├──0000
| | ├──000001.txt
| | ├──....txt
| | └──000153.txt
| ├──...
| └──0020
└──testing
python3
numpy
opencv
yaml
- Please modify the dataset path and detections path in the yaml file to your own path.
- Then run
python3 kitti_3DMOT.py config/online/pvrcnn_mot.yaml
- The results are automatically saved to
evaluation/results/sha_key/data
, and evaluated by HOTA metrics.
The evaluation codes are copied from Kitti.