[Bug]: Pass RemoveMultiSubGraphOpDanglingParamsResults has bug

[nlskyfree]

OpenVINO Version

2024.1.0

Operating System

Ubuntu 18.04 (LTS)

Device used for inference

CPU

Framework

TensorFlow 1

Model used

No response

Issue description

Problem

ovc frozen_model.pb --compress_to_fp16=False
[ ERROR ] -------------------------------------------------
[ ERROR ] ----------------- INTERNAL ERROR ----------------
[ ERROR ] Unexpected exception happened.
[ ERROR ] Please verify parameters and environment.
[ ERROR ] If you think this is a bug, please create new ticket here:
[ ERROR ] https://github.com/openvinotoolkit/openvino/issues.
[ ERROR ] -------------- DETAILED INFORMATION -------------
[ ERROR ] Exception from src/core/src/node.cpp:590:
node index is out of range

[ ERROR ] Traceback (most recent call last):
File "/home/anaconda3/envs/py38/lib/python3.8/site-packages/openvino/tools/ovc/convert_impl.py", line 489, in _convert
ov_model = driver(argv, {"conversion_parameters": non_default_params})
File "/home/naconda3/envs/py38/lib/python3.8/site-packages/openvino/tools/ovc/convert_impl.py", line 240, in driver
ov_model = moc_emit_ir(prepare_ir(argv), argv)
File "/home/anaconda3/envs/py38/lib/python3.8/site-packages/openvino/tools/ovc/moc_frontend/moc_emit_ir.py", line 20, in moc_emit_ir
apply_moc_transformations(ngraph_function, cf=False, smart_reshape=True)
RuntimeError: Exception from src/core/src/node.cpp:590:
node index is out of range

[ ERROR ] ----------------- END OF REPORT -----------------
[ ERROR ] -------------------------------------------------

Debug Code

I located this problem by debuging the following code.

int main() {
    // Read the TensorFlow frozen model file
    std::string model_content = read_binary_file(frozen_pb_path);
    ov::frontend::tensorflow::FrontEnd frontend;

    // Load the model using the frontend
    std::vector<ov::Any> inputs = {ov::Any(frozen_pb_path)};
    auto input_model = frontend.load(inputs);
    if (!input_model) {
        throw std::runtime_error("Failed to load input model.");
    }

    // Convert the entire model
    auto converted_model = frontend.convert(input_model);
    if (!converted_model) {
        throw std::runtime_error("Failed to convert model.");
    }

    // Save the converted model
    ov::serialize(converted_model, output_model_path, bin_model_path);
    std::cout << "Model successfully converted and saved to " << output_model_path << std::endl;

    ov::pass::Manager manager;
    // manager.register_pass<ov::pass::SmartReshape>();
    // manager.register_pass<ov::pass::MOCTransformations>(false);
    manager.register_pass<ov::pass::RemoveMultiSubGraphOpDanglingParamsResults>();
    // manager.register_pass<ov::pass::FlushFP32SubnormalsToZero>();
    manager.run_passes(converted_model);
}

Problem Model Structure

The model before pass transformations has been saved.
unfortunately. I can't provide complete model for some reason.

<layer id="3034" name="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/Sign" type="Sign" version="opset1">
    <input>
        <port id="0" precision="FP32">
            <dim>1</dim>
        </port>
    </input>
    <output>
        <port id="1" precision="FP32" names="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/Sign,opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/Sign:0">
            <dim>1</dim>
        </port>
    </output>
</layer>
<layer id="3038" name="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/Any" type="ReduceLogicalOr" version="opset1">
    <data keep_dims="false" />
    <input>
        <port id="0" precision="BOOL">
            <dim>1</dim>
        </port>
        <port id="1" precision="I32">
            <dim>1</dim>
        </port>
    </input>
    <output>
        <port id="2" precision="BOOL" names="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/Any,opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/Any:0,opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/pred_id,opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/pred_id:0" />
    </output>
</layer> 
..........
<layer id="3039" name="If_4685" type="If" version="opset8">
    <else_port_map>
        <input external_port_id="1" internal_layer_id="0" />
        <output external_port_id="3" internal_layer_id="5" />
    </else_port_map>
    <then_port_map>
        <input external_port_id="2" internal_layer_id="0" />
        <output external_port_id="3" internal_layer_id="1" />
    </then_port_map>
    <input>
        <port id="0" precision="BOOL" />
        <port id="1" precision="FP32">
            <dim>1</dim>
        </port>
        <port id="2" precision="FP32">
            <dim>1</dim>
        </port>
    </input>
    <output>
        <port id="3" precision="FP32" names="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/Merge:0">
            <dim>1</dim>
        </port>
    </output>
    <then_body>
        <layers>
            <layer id="0" name="Parameter_4676" type="Parameter" version="opset1">
                <data shape="1" element_type="f32" />
                <output>
                    <port id="0" precision="FP32" names="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/Switch_1:1">
                        <dim>1</dim>
                    </port>
                </output>
            </layer>
            <layer id="1" name="Result_4683" type="Result" version="opset1">
                <input>
                    <port id="0" precision="FP32">
                        <dim>1</dim>
                    </port>
                </input>
            </layer>
        </layers>
        <edges>
            <edge from-layer="0" from-port="0" to-layer="1" to-port="0" />
        </edges>
        <rt_info />
    </then_body>
    <else_body>
        <layers>
            <layer id="0" name="Parameter_4673" type="Parameter" version="opset1">
                <data shape="1" element_type="f32" />
                <output>
                    <port id="0" precision="FP32" names="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/mul/Switch:0">
                        <dim>1</dim>
                    </port>
                </output>
            </layer>
            <layer id="1" name="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/mul/y" type="Const" version="opset1">
                <data element_type="f32" shape="" offset="364" size="4" />
                <output>
                    <port id="0" precision="FP32" names="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/mul/y,opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/mul/y:0" />
                </output>
            </layer>
            <layer id="2" name="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/mul" type="Multiply" version="opset1">
                <data auto_broadcast="numpy" />
                <input>
                    <port id="0" precision="FP32">
                        <dim>1</dim>
                    </port>
                    <port id="1" precision="FP32" />
                </input>
                <output>
                    <port id="2" precision="FP32" names="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/mul,opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/mul:0">
                        <dim>1</dim>
                    </port>
                </output>
            </layer>
            <layer id="3" name="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/add/y" type="Const" version="opset1">
                <data element_type="f32" shape="" offset="1600" size="4" />
                <output>
                    <port id="0" precision="FP32" names="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/add/y,opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/add/y:0" />
                </output>
            </layer>
            <layer id="4" name="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/add" type="Add" version="opset1">
                <data auto_broadcast="numpy" />
                <input>
                    <port id="0" precision="FP32">
                        <dim>1</dim>
                    </port>
                    <port id="1" precision="FP32" />
                </input>
                <output>
                    <port id="2" precision="FP32" names="opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/add,opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/cond/add:0">
                        <dim>1</dim>
                    </port>
                </output>
            </layer>
            <layer id="5" name="Result_4680" type="Result" version="opset1">
                <input>
                    <port id="0" precision="FP32">
                        <dim>1</dim>
                    </port>
                </input>
            </layer>
        </layers>
        <edges>
            <edge from-layer="0" from-port="0" to-layer="2" to-port="0" />
            <edge from-layer="1" from-port="0" to-layer="2" to-port="1" />
            <edge from-layer="2" from-port="2" to-layer="4" to-port="0" />
            <edge from-layer="3" from-port="0" to-layer="4" to-port="1" />
            <edge from-layer="4" from-port="2" to-layer="5" to-port="0" />
        </edges>
        <rt_info />
    </else_body>
</layer>
...........
<edge from-layer="3034" from-port="1" to-layer="3039" to-port="2" />
<edge from-layer="3034" from-port="1" to-layer="3039" to-port="1" />
<edge from-layer="3038" from-port="2" to-layer="3039" to-port="0" />
<edge from-layer="3038" from-port="2" to-layer="3039" to-port="3" />

Fix

The reason this issue occurs is that both subgraphs of If_4685 have dangling Parameters, and they need to delete the same Input "opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/Any" . Coincidentally, this Input is connected to the If_4685 node twice, which causes the check std::count(std::begin(op_inputs), std::end(op_inputs), current_input) failed, and op_inputs delete "opt_op_all/merge_all_info_mlp_3/merge_all_info_mlp_3_batch_norm/Any" twice, which cause the problem.
I fix this problem by adding the following code.

Step-by-step reproduction

No response

Relevant log output

No response

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[rkazants]

Hi @nlskyfree, thanks for reporting this.

Did you try the latest release 2024.3? Still reproduced?

@itikhono, @dorloff, please take a look.

Best regards,
Roman

[nlskyfree]

@rkazants yeah, still reproduce.

[rkazants]

@nlskyfree, can you please create PR with your proposed fix? We will take a look closely.

Best regards,
Roman

[avitial]

@nlskyfree do you have an update for us?