CUSTOM_BOX_OBJECTS support

[osu-uwrt-bot]

Preliminary Checks

• This issue is not a duplicate. Before opening a new issue, please search existing issues.
• This issue is not a question, bug report, or anything other than a feature request directly related to this project.

Proposal

Support for the CUSTOM_BOX_OBJECTS from the ZED object detection module. The ZED SDK has a parameter which allows users to submit custom detections to it and it is able to track the objects as well as output the position and 3d bounding boxes for the objects.

Currently the wrapper doesn't expose that part of the SDK and it isn't possible to access that part of the SDK through another ROS node while running the wrapper due to the wrapper having control of the camera.

Would it be possible to enable that as an option within the launch file and then accept the objects as a message to which the wrapper outputs a Detections3Darray (or even a custom message with the info from the SDK).

Use-Case

I would be able to use a custom detector in ROS with 3D bounding boxes

Anything else?

No response

[Myzhar]

We are working on this feature. The only issue is the latency:

1. The zed_node publishes the rgb image (dT_1)
2. The detector_node receives the rgb image (dT_2)
3. The detector_node extracts the 2D bounding box and publishes it (dT_3)
4. The zed_node receives the 2D bbox and starts the tracking

If dT_1+dT_2+dT_3<T_grab then there is only one frame of latency and the tracking can work correctly, otherwise it can be not so precise.

Furthermore, ROS introduces latency in the communication process... to know if this engine works we can only put all the gears together and test it.

[MrOCW]

Hi @Myzhar, how is the progress of this feature? Will it be implemented anytime soon?

[Marketos-Damigos]

Hi @Myzhar any progress on that? Would it be possible to change the underlying model of the base functions and theoretically have our own custom detection model run?

[Myzhar]

I'm sorry but this is not currently planned.
We suggest you add custom detection to the main code.

[Marketos-Damigos]

I'm sorry but this is not currently planned. We suggest you add custom detection to the main code.

Could you elaborate on how to approach it?

[shaoyifei96]

He commented here https://community.stereolabs.com/t/custom-object-detection-with-ros1/1527

[Mat198]

Any news about this resource? It would be very helpful in my project.

Meanwhile, I'll be working on a way to do this following the approach suggested above.

[creminem94]

Hi, did anyone actually implemented a custom detector in ROS2?