Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add MemberDowngradeUpgrade failpoint #19125

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

siyuanfoundation
Copy link
Contributor

@siyuanfoundation siyuanfoundation commented Jan 3, 2025

Please read https://github.com/etcd-io/etcd/blob/main/CONTRIBUTING.md#contribution-flow.

#17118

The MemberDowngradeUpgrade failpoint first downgrades all members, and upgrades some members. This is to test no robustness issue during the whole downgrade-upgrade process, which can be stopped at any point.

Tested locally with

go test -run TestRobustnessExploratory -v --count 100 --failfast --timeout 5h

There is a little flakiness due to reasons unrelated to what the test wants to do, like a server fails to start. But that's mainly because we are stopping and restarting the servers many times. I think <5% is probably acceptable.

@k8s-ci-robot
Copy link

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Copy link

codecov bot commented Jan 3, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.80%. Comparing base (9db8dcb) to head (6f8d37f).

Additional details and impacted files

see 25 files with indirect coverage changes

@@            Coverage Diff             @@
##             main   #19125      +/-   ##
==========================================
- Coverage   68.87%   68.80%   -0.08%     
==========================================
  Files         420      420              
  Lines       35649    35649              
==========================================
- Hits        24554    24527      -27     
- Misses       9674     9699      +25     
- Partials     1421     1423       +2     

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9db8dcb...6f8d37f. Read the comment docs.

@siyuanfoundation
Copy link
Contributor Author

/cc @ahrtr @serathius

@ahrtr
Copy link
Member

ahrtr commented Jan 11, 2025

like a server fails to start. But that's mainly because we are stopping and restarting the servers many times.

5% flakly on github or prow might be acceptable, because the test environment is out of our control, but in your local environment, it is still a little concerning.

Overall the PR looks good. Thanks for the nice work.

@k8s-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahrtr, siyuanfoundation

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

return nil
}

func verifyVersion(t *testing.T, clus *e2e.EtcdProcessCluster, member e2e.EtcdProcess, expectedServerVersion, expectedClusterVersion *semver.Version) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably we can move this function into test/framework/e2e/utils.go, so that it can be reused by other downgrade or migrate related e2e test cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

3 participants