Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GODRIVER-2960 Improve Evergreen Runtime and Usability #1472

Merged
merged 9 commits into from
Nov 28, 2023

Conversation

blink1073
Copy link
Member

@blink1073 blink1073 commented Nov 14, 2023

GODRIVER-2960

Summary

  • Make Evergreen output less verbose.
  • Attach test_suite.tgz file even in failure so we see the reason.
  • Only start CSE servers when we need to.
  • Clean up CSE startup code.

Background & Motivation

Improve Evergreen Runtime and Usability.

Copy link
Contributor

API Change Report

No changes found!

@blink1073 blink1073 marked this pull request as ready for review November 16, 2023 18:22
@blink1073 blink1073 requested a review from a team as a code owner November 16, 2023 18:22
@blink1073 blink1073 requested review from prestonvasquez and removed request for a team November 16, 2023 18:22
@@ -166,7 +166,7 @@ functions:
. ${DRIVERS_TOOLS}/.evergreen/venv-utils.sh
. ${DRIVERS_TOOLS}/.evergreen/find-python3.sh

export PYTHON3_BINARY="$(find_python3)"
export PYTHON3_BINARY="$(find_python3 2>/dev/null)"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for discarding error messages when running find_python3?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the recommended usage pattern, it prints debug info to stderr.

@@ -203,6 +203,19 @@ functions:
content_type: application/x-gzip
display_name: "fuzz.tgz"

upload-raw-test-suite:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC the reason this function is not in the upload-mo-artifacts's command array is that this needs to be used in multiple tasks. Is it possible that those cases result in workflows that do not call the post functions?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I grouped them together and renamed it to handle-test-artifacts, since we're already uploading the fuzz file there.

aws_secret: ${aws_secret}
local_file: src/go.mongodb.org/mongo-driver/test.suite
optional: true
remote_file: ${UPLOAD_BUCKET}/${build_variant}/${revision}/${version_id}/${build_id}/logs/${task_id}-${execution}-test.suite.txt
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tar'ing the test.suite.txt file before PUT would result in a faster upload.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

mongo-orchestration stop
cd -
rm -rf $DRIVERS_TOOLS || true
bash ${DRIVERS_TOOLS}/.evergreen/stop-orchestration.sh
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tools script for stop-orchestration.sh doesn't include the logic that removes the drivers tools clone:

cd -
rm -rf $DRIVERS_TOOLS || true

Is this worth retaining as part of the cleanup?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-added

@blink1073
Copy link
Member Author

Note that we need to wait for https://jira.mongodb.org/browse/DEVPROD-2629 to verify uploads are working.

prestonvasquez
prestonvasquez previously approved these changes Nov 17, 2023
@blink1073 blink1073 marked this pull request as draft November 20, 2023 20:37
@blink1073 blink1073 closed this Nov 27, 2023
@blink1073 blink1073 reopened this Nov 27, 2023
@blink1073 blink1073 marked this pull request as ready for review November 27, 2023 19:23
@@ -196,12 +200,31 @@ functions:
params:
aws_key: ${aws_key}
aws_secret: ${aws_secret}
optional: true
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there cases where the test suite does not occur?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lambda test and the ECS test don't.

@blink1073 blink1073 merged commit 8705829 into mongodb:v1 Nov 28, 2023
32 of 36 checks passed
@blink1073 blink1073 deleted the GODRIVER-2960 branch November 28, 2023 01:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants