Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E Tests parallel execution #2833

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

E2E Tests parallel execution #2833

wants to merge 2 commits into from

Conversation

CDimonaco
Copy link
Member

@CDimonaco CDimonaco commented Jul 29, 2024

Description

This pr splits the cypress tests execution across multiple CI jobs, we use the cypress-split plugin.
This PR does not fix any issue on flakiness or test logic, it just speeds up the execution of test pipeline.

@CDimonaco CDimonaco force-pushed the e2e_cypress_split branch 3 times, most recently from cabbeac to 70aa966 Compare July 30, 2024 07:59
@CDimonaco CDimonaco changed the title test e2e cypress parallel split E2E Tests parallel execution Jul 30, 2024
@CDimonaco CDimonaco self-assigned this Jul 30, 2024
@CDimonaco CDimonaco added enhancement New feature or request test labels Jul 30, 2024
@CDimonaco CDimonaco marked this pull request as ready for review July 30, 2024 08:01
@CDimonaco CDimonaco requested a review from balanza July 30, 2024 08:01
Copy link
Contributor

@gagandeepb gagandeepb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! I wonder what is the expected/observed speed-up with these changes ?

@CDimonaco
Copy link
Member Author

Thanks for this! I wonder what is the expected/observed speed-up with these changes ?

That's a good question! I observed an increase of speed of about 5 to 7 minutes, but this is just the first step, when we refactor the e2e tests, in order to make them less flaky and without some useless costly operations (mostly photofinish dump when not needed), this pr should make a lot more sense

Copy link
Contributor

@arbulu89 arbulu89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect! Really really appreciated!
Just a comment about putting the 4 as a variable

PD: I don't know if we would need some change in the artifact upload in failure cases, otherwise they might overwrite. Maybe we can add some sort of index to the folder name

id: prepare
uses: bahmutov/gh-build-matrix@main
with:
n: 4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have this 4 in some variable on top?
So we can change if needed, as it is used in 2 places

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also: where does this number come from? Should it be the number of cores?

steps:
- name: Create matrix ⊹
id: prepare
uses: bahmutov/gh-build-matrix@main
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: ‏As far as I get it, this action just provides a range for enumeration, right? Like you say 4 and it spits [1,2,3,4]

Comment on lines +437 to +447
SPLIT: ${{ strategy.job-total }}
SPLIT_INDEX: ${{ strategy.job-index }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: ‏Where do those variables come out? Shouldn't be something like strategy.matrix... or prepare.output.matrix...?

Copy link
Member

@balanza balanza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @CDimonaco, this PR is very valuable for me as it barely halves the CI execution time.

There are some unanswered questions in comments, besides that I think we should merge.

@balanza balanza force-pushed the e2e_cypress_split branch 4 times, most recently from 7ef888b to 8dbec90 Compare November 6, 2024 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request test
Development

Successfully merging this pull request may close these issues.

4 participants