Fix multiple failures in KillProcess test on KVM.

[hdwhdw]

Description of PR

Summary:

• Gracefully skip the test for killprocess if the process does not exists, instead of failing it. Some process such as telemetry is not always there on KVM testbed and the validation of killprocess should not depend on it.
• Avoid killing swss and snmp process as it leads to errors on KVM testbed:isAutoNegEnabled: Failed to get port AutoNeg status for port pid:1000000000021

Fixes #16238

Type of change

• [x ] Bug fix
• Testbed and Framework(new/improvement)
• Test case(new/improvement)

Back port request

• 202012
• 202205
• 202305
• 202311
• 202405

Approach

The test tries to kill a process then restart it but the API throws an exception if the process does not exist.
Here we gracefully catch the exception and skip the test.

What is the motivation for this PR?

Fix the test for KVM.

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[yutongzhang-microsoft]

Have you tested locally?

[hdwhdw]

@yutongzhang-microsoft Yes.

============================= 11 passed, 3 skipped, 1 warning in 219.49s (0:03:39) ======================

[yutongzhang-microsoft]

Can we consider the issue #16238 has been fixed? Is so, you can close it and let's monitor the result in PR test.

[hdwhdw]

Thanks I will close the bug once this is merged.

[qiluo-msft]

snmp

It should be a valid reqirement to kill snmp cotainer. why remove?

[hdwhdw]

It is indeed valid. Restarting snmp container will produce an error log due to this line:
https://github.com/sonic-net/sonic-buildimage/blob/39e2131a7b76f6c3d5257b7e02c540dd33a24d5b/files/build_templates/docker_image_ctl.j2#L114

{%- elif docker_container_name == "snmp" %}
    $SONIC_DB_CLI STATE_DB HSET 'DEVICE_METADATA|localhost' chassis_serial_number $(decode-syseeprom -s)

Because

sudo decode-syseeprom -s
Failed to read system EEPROM info

I think this is known:

sonic-mgmt/tests/platform_tests/cli/test_show_platform.py

Line 146 in 5054d0a

# For kvm testbed, command `show platform syseeprom` will return the expected Error

This will also cause a similar issue when killing pmon (I think this is due to "missing sonic_platform module".)

So for now let's just skip these two for vs platform. I don't think this affect our ability to quality the KillProcess implementation.

[qiluo-msft]

swss

what is wrong to kill swss?

[hdwhdw]

Turns out it is because the test wasn't written correctly: We need to explicitly wait for critical processes to start after killing swss. Looks like killing and restarting swss will make a lot of other processes restart, and if we don't wait and immediately start the next testcase, it will generate some swss error (in the next testcase).

[hdwhdw]

Fixed after adding a wait for critical process back.

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).