Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eden tests will be subject to Docker Hub pull rate limits #1002

Open
deitch opened this issue Jul 15, 2024 · 0 comments
Open

eden tests will be subject to Docker Hub pull rate limits #1002

deitch opened this issue Jul 15, 2024 · 0 comments

Comments

@deitch
Copy link
Contributor

deitch commented Jul 15, 2024

eve tests were being slowed down, and at times completely blocked, due to Docker Hub pull rate limits; see this eve issue. Mostly, that has been fixed, due to a variety of removing unnecessary pulls, ensuring dependencies are part of docker OSS and having the Docker Hub ID used to perform the pulls run under an LFEdge paid account that allows up to 5,000 pulls per day.

eden tests still pull many images, as part of:

  • starting eden, e.g. redis or s3
  • running eve, eve images, although this is not an issue for 2 reasons: eve images are part of Docker-sponsored OSS and exempt from rate limits, and eden tests mostly take images from earlier GitHub Actions artifacts, rather than pulling from an OCI registry
  • running actual tests

The Docker Hub ID used for eden tests, especially those imported into eve as part of the eve CI pipeline, should be running under the paid ID provided by LFEdge. However, these will hit limits eventually.

This issue is to track finding where images are pulled as part of the eden suite, and find ways to avoid hitting rate limits via:

  • eliminating unnecessary pulls
  • restricting pulls to Docker sponsored OSS images, e.g. anything published under lfedge/*
  • sharing pulls among runs

Because eve runs a matrix of different eden tests, if each eden test run pulls "only" 5 images, and the matrix itself runs 5 tests, that is 25 tests for each such PR, not counting any others. Since most of the runs duplicate images, this scales exponentially.

The goal is to get to the point, where each PR needs just a few images pulls other than sponsored (rate limit-exempt) images.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant