soft-nms cannot improve mAP used SSD detection(one-stage)

[ujsyehao]

I implement soft-nms in SSD(caffe), code link modify detection_output.cu, bbox_util.hpp, bbox_util.cpp, I ensure the custom implementation version is the same as author-provided python version(I have tested it use fake input)
Below is my test results(PASCAL VOC 2007 test dataset):

• SSD(VGG) nms(nms thresh: 0.45), mAP([email protected]): 75.1
• SSD(VGG) soft-nms-linear(nms thresh: 0.45), mAP([email protected]): 74.89
• SSD(VGG) soft-nms-gaussian(nms thresh: 0.45), mAP([email protected]): 74.3

Can you give me some advance?

[bharatsingh430]

we mention in the the paper why it is not good for ssd/yolo. you can still get some improvement at 0.75 iou though

[ujsyehao]

@bharatsingh430 , test results(PASCAL VOC 2007 test dataset) update:

• SSD(VGG) nms(nms thresh: 0.45), mAP([email protected]): 50.2
• SSD(VGG) nms(nms thresh: 0.7), mAP([email protected]): 51.8
• SSD(VGG) soft-nms-linear(nms thresh: 0.45), mAP([email protected]): 51.3
• SSD(VGG) soft-nms-linear(nms thresh: 0.65), mAP([email protected]): 51.8
• SSD(VGG) soft-nms-linear(nms thresh: 0.7), mAP([email protected]): 51.9
• SSD(VGG) soft-nms-linear(nms thresh: 0.75), mAP([email protected]): 51.8
• SSD(VGG) soft-nms-gaussian(nms thresh: 0.45), mAP([email protected]): 51.6

Thank you for your reply, soft-nms can get 1-1.4 mAP improvement at 0.75 IOU to use nms thresh 0.45.

I have some problems:

1. soft-nms paper say 'For detectors like SSD and YOLOv2 which are not proposal based, with the linear function, soft-nms only obtains an improvement of 0.5%.' How do you get the result? What is the IOU threshold used? What is nms threshold used? -> I check your answer under other issues, you use COCO metric
2. paper say 'proposal-based detectors have higher recall and hence Soft-NMS has more potential to improve recall at higher Ot', How to understand it?　Why soft-nms has more potential to improve recall at higher IOU thresh?

[bharatsingh430]

If you don't do NMS for 2 stage detectors like faster-rcnn and 1 stage detectors like yolo/ssd, and compare recall, you would observe a gap in recall (2 stage > 1 stage), which will increase at higher IoU. Soft-NMS retains detections instead of discarding them. If detections are not generated at the first place (i.e recall is low) even before NMS , it is not possible to reduce the miss rate.

[ujsyehao]

@bharatsingh430 Thank you!