mask for detector

[pak-da]

Hi,
I am using find_object_2d for a single object detection and manipulation task with a robot arm.The object is well known so I can detect it even with a single RGB camera. I need precious and robust result that seems to be reached with SIFT detector and description however I'm sure that I can do something better with a color based mask in order to filter out all the keypoints that can be similar but belong to a region with different color. So how can I impose a mask for detector?

Thanks

[matlabbe]

The function here:

find-object/src/MainWindow.cpp

Line 1318 in fd9270a

if(findObject_->detect(sceneImage_, info))

can use a mask as parameter. You may create a mask from the input depth image topic like this.
You would have to create a fake depth image from the color mask, then feed that depth image to find_object_node.

[pak-da]

It seems that findObject::detect doesn't accept directly a mask.

find-object/src/FindObject.cpp

Line 1423 in fd9270a

bool FindObject::detect(const cv::Mat & image, find_object::DetectionInfo & info) const

However it uses the class ExtractFeaturesThread

find-object/src/FindObject.cpp

Line 1447 in fd9270a

ExtractFeaturesThread extractThread(detector_, extractor_, -1, grayscaleImg);

in which the mask seems just initialized

find-object/src/FindObject.cpp

Lines 749 to 756 in fd9270a

	computeFeatures(
	detector_,
	extractor_,
	image_,
	cv::Mat(),
	keypoints_,
	descriptors_,
	timeDetection_,

If I've understood fine you are suggesting to replace the initialization of the mask with a variable mask that is the 4th channel. Of course I have to give as input a 4 channel image. Is it right?

I'm not very experienced in programming in C++ so if you could give me some examples i would be grateful

[matlabbe]

If you are using ROS, we can make rtabmap subscribe to a depth image (single channel 16bits unsigned short, or CV_16UC1). The depth image is passed here before FindObject::detect() gets called:

find-object/src/MainWindow.cpp

Line 1296 in fd9270a

void MainWindow::update(const cv::Mat & image, const QString & frameId, double stamp, const cv::Mat & depth, float depthConstant)

As I said, you can add a new parameter to the function FindObject::detect(..., const cv::Mat & mask), and you can convert the depth to a mask matrix (CV_8UC1) using the conversion code I linked to pass the mask to that function. The mask can be then redirected to ExtractFeaturesThread.

Regards,
Mathieu

[pak-da]

I've tried what you suggested and it works fine. Thanks
Is there any way in order to avoid the tf publishing? (it is wrong because the depth channel is fake)

[matlabbe]

You could set if(0) here:

find-object/src/ros/FindObjectROS.cpp

Line 61 in fd9270a

if(info.objDetected_.size() && !depth.empty() && depthConstant != 0.0f)