Huge translation error but reasonable pixel error and ACC measurement?

[F2Wang]

I have always been getting a huge translation error despite somewhat reasonable numbers on other metrics when training on the customized items.

2019-03-26 11:52:50 Mean corner error is 6.589319
2019-03-26 11:52:50 Acc using 5 px 2D Projection = 35.38%
2019-03-26 11:52:50 Acc using 0.0194 vx 3D Transformation = 34.59%
2019-03-26 11:52:50 Acc using 5 cm 5 degree metric = 44.50%
2019-03-26 11:52:50 Translation error: 35712796398.714020, angle error: 44.276071

I originally thought there is something wrong with my provided ground truth labels or the mesh used, but the estimated rotation and translation through the pnp algorithm for the ground truth labels appears to be right, while the R and T for the predicted points from the pnp algorithm is all over the place and obviously wrong.

What could be the possible cause of this huge translation error?

[btekin]

Hello, the first thing I would check would be if the 2D predictions (before PnP) are accurate by visualizing them. Actually the mean corner error that you report reflects that the 2D predictions are quite accurate. My guess is that the problem comes from the PnP stage. To fix that, I would make sure if the 3D point coordinates are in an appropriate unit (e.g. mm or m) and if the intrinsic matrix is correct. I would maybe suggest to follow the same units for the LINEMOD dataset. You could also find the discussion in Issue #49 and #68 useful to check if your .ply file is correctly formed.

[F2Wang]

Thank you for your reply! I think the occasional huge translation is caused by the PNP algorithm, I adjusted my distortion coefficient and use cv2.solvePnPRansac instead of solvePnP and now the problem is gone ...

[F2Wang]

After pnp fix, really nice result! Thank you for this great work!

2019-03-29 10:57:53 Results of sugar
2019-03-29 10:57:53 Acc using 5 px 2D Projection = 99.21%
2019-03-29 10:57:53 Acc using 10% threshold - 0.0194 vx 3D Transformation = 99.84%
2019-03-29 10:57:53 Acc using 5 cm 5 degree metric = 100.00%
2019-03-29 10:57:53 Mean 2D pixel error is 2.075887, Mean vertex error is 0.005785, mean corner error is 3.388159
2019-03-29 10:57:53 Translation error: 0.009067 m, angle error: 1.266151 degree, pixel error: 2.075887 pix