Inference with customised MMPose.

[korneliaWatson]

Hi all,

I trained top-down heatmap-based MMPose on customised keypoint detection problem.

I'm seeing two ways in which a trained model is used for inference.

1. With the use of mmdeploy, after model conversion. Example described here.
2. With the use of MMPoseInferencer. Example described here

Could someone clarify what the recommended way is, and why?

I believe # 2 requires person detection model, while # 1 doesn't? If the detection model is needed, can a pretrained human model be used as in the example above? I recall a post somewhere that mentioned it has to be a custom trained model too, and another post, which said that in the absence of object detection model then a whole image is treated as a bounding box.

Any insights and clarifications are greatly appreciated, thank you.

[Ben-Louis]

For the first scenario, the image input mmdeploy/demo/resources/human-pose.jpg is a cropped picture with only one person.

MMPoseInferencer will automatically use a human detector if you're using top-down pose estimators. Thus, if you want to conduct pose estimation in a multi-person scenario, using the second method is recommended.

[korneliaWatson]

Thank you for the answer.

In my use-case I only need to detect keypoints on images of individual people. Going by your response, I should use method 1, which utilises mmdeploy. Just to double check, when using that method, I don't need to train my own human detector, do I? A whole image is treated as a bounding box, right?

[Ben-Louis]

Yes. If you want to treat the whole image as a bounding box, you don't need a detector.

BTW, MMPoseInferencer also supports inference with top-down pose estimator without human detector by setting det_model='whole_image'. You can find more details at https://mmpose.readthedocs.io/en/latest/user_guides/inference.html#custom-pose-estimation-models.

[korneliaWatson]

Excellent. Thank you for clarifying. :-)

[korneliaWatson]

Ah, one more related question... :-)

When utilising mmdeploy, there are functionalities that support input preparation, e.g.

# process input image
input_shape = get_input_shape(deploy_cfg)
model_inputs, _ = task_processor.create_input(image, input_shape)

If I use MMPoseInferencer instead, are there any input preparations that I have to do on my side, or is it all handled by the inferencer? E.g. at bare minimum, shall I rescale the image to match what the model was trained on, or will it be done internally by MMPoseInferencer?

[Ben-Louis]

The Inferencer will transform input images to fit model requirements based on the test pipeline defined in the config file. Preprocessing of images is not necessary.

[korneliaWatson]

Hello, me again. :-)

I tried to carry out inference with MMPoseInferencer using my customised keypoint detection model. I hoped that I would be able to have mmpose installed as a pip package, instead of from source, and execute the following, to make the inference:

    # Instantiate the inferencer.
    inferencer = MMPoseInferencer(
        pose2d=POSE2D,                    # my custom config file
        pose2d_weights=POSE2D_WEIGHTS,    # my custom weights
        det_model="whole_image", 
    )

    # The MMPoseInferencer API employs a lazy inference approach, creating a prediction generator when given input.
    result_generator = inferencer(img_path, show=True)
    result = next(result_generator)

However, I get:

ValueError: Cannot find model: /home/projects/mmpose/artifacts/front/384x288_94x64_pck_20230904_084728/384x288_94x64_pck_20230904_084728.py in mmpose

Does it mean that if I want to make an inference with a custom model, I have to have mmpose installed from source and included in any deployment package?