How do I get evaluate() to work?

[Blankeos]

First off, thanks for the tutorial! You're a real lifesaver. I'm trying to get evaluate to work at the moment. I have num_keypoints=4. The model trains smoothly when I just comment out evaluate during training, but I need to evaluate the performance now so I'm trying to get that to work. I did what you instructed in your tutorial:

Update. It’s possible not to edit pycocotools/cocoeval.py file in pycocotools library to change kpt_oks_sigmas, but to edit coco_eval.py file, as Diogo Santiago suggested:

# self.coco_eval[iou_type] = COCOeval(coco_gt, iouType=iou_type)
coco_eval = COCOeval(coco_gt, iouType=iou_type)
coco_eval.params.kpt_oks_sigmas = np.array([.5, .5]) / 10.0
self.coco_eval[iou_type] = coco_eval

Since I have num_keypoints=4, I instead wrote np.array([.5, .5, .5, .5]) / 10.0 for kpt_oks_sigmas.

But I'm still getting the same error: ValueError: operands could not be broadcast together with shapes (4,) (17,)

Hoping you can help me with this! 🤞

[Blankeos]

😃 Fixed!

Apparently, the solution on the article was right. But for some reason, when I edit on Colab and save, it doesn't actually get saved. So I had to already have changed coco_eval.py before I even cloned it on my Colab runtime.

I basically did the same thing, but this time I downloaded the training scripts from PyTorch and created my own repository to clone:

1. Download the scripts from pytorch/vision/references/detection and created my own repo storing it
2. Open coco_eval.py
3. Edit the lines somewhere under __init__ for iou_type in iou_types:

   - self.coco_eval[iou_type] = COCOeval(coco_gt, iouType=iou_type)
   + coco_eval = COCOeval(coco_gt, iouType=iou_type)
   + coco_eval.params.kpt_oks_sigmas = np.array([.5, .5, .5, .5]) / 10.0 # 4 keypoints
   + self.coco_eval[iou_type] = coco_eval

4. At the start of the Training Notebook:

   !git clone https://github.com/<username>/<repo-name>.git
   %cd <repo-name>

[bryan273]

Thanks, man. I've been struggling with this error. Apparently, making direct changes to the file in Colab doesn't update it. weird.