Evaluation code roomlayout.m of cooperative_scene_parsing don't work on the output

[chengzhag]

Hi Yinyu:

I followed your guide to evaluate the results using code from cooperative_scene_parsing. With GT downloaded and some essential changes to path settings, the roomlayout.m runs successfully on the results predicted by your work.

However, the roomlayout.m outputs the same IoU of 0 on every sample. Here is sample 1 as an example:

I tried to turn on the visualization option looking to find the reason:

The predicted layout bounding box showed in the visualization above is nothing like the visualization with your code utils/visualize.py:

Is there anything I missed? If some more changes need to be made on the evaluation code from cooperative_scene_parsing, Is it possible to open-source your version of the final evaluation code?

[chengzhag]

Sorry to bother you again at this inappropriate time. In case you missed my last email, I'm trying to do some work based on your paper. Eagerly to compare with your work on the same matrics. May I ask to get the evaluation code you implemented based on cooperative_scene_parsing? If the code is not ready to publish on github, may I get the code before it's published? Hope to not take up you much time.

[YanjunLIU-ac]

@pidan1231239 Hello there, I am also doing some work based on this project, and I also ran into the problem similar to you, where in my case the IoU of 3d bounding box is always 0 using the MATLAB evaluation code from "Cooperative scene parsing". Have you solved this problem? Any ideas on how to make it work properly and get the final evaluation results?

[chengzhag]

Hi, I've contacted Yinyu via email and got the following reply:

Sorry for the late reply as we are currently pushing for a deadline. I think we would update the evaluation code after these busy days.
But I think the reason you get the IoUs with 0s is that you used the GT data from their work. Actually, we used a different world coordinate system to build our system. You can visualize (or save by yourself) our gt data from the visualization code by changing 'prediction' to 'gt' or ‘both’ in the arguments below:

Hope this can help you.