Improvements to robustness tests to prevent issues like --experimental-wait-cluster-ready-timeout

[serathius]

What would you like to be added?

--experimental-wait-cluster-ready-timeout showed in #16672 that adding features to etcd is hard, especially if those are not well tested as they are optional. So testing just the default configuration is not enough.

Some ideas on preventing similar issues:

• Robustness tests should validate all possible configurations. To get good coverage we should look through etcd optional flags and pick a random subset of them.
• Testing etcd bootstrap is as important. We might extend failpoint injection from being HTTP only, to also environment variable based.
• Sleep failpoints! We should implement them already
• Test adding and removing members. Especially in context of migration to v3 store.

Why is this needed?

Correctnes