Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: testing common wheels #2031

Merged
merged 37 commits into from
Sep 17, 2024

Conversation

tdejager
Copy link
Contributor

@tdejager tdejager commented Sep 10, 2024

Adds a new testings suite that uses pytest and pixi to test common wheels, thanks: @liblaf for the inspiration! See: https://github.com/liblaf/pixi-test

This PR add two pixi tasks:

  • test-common-wheels requires a build and tests the installation of some common wheels
  • *-ci task does the same but does not have the depends-on

There is a wheels.toml file that contains a small configuration structure for defining the wheels that should be tested in CI. The tests contain the necessary files to actually parse and execute upon these functions. The tests are run in parallel using python-xdist.

Running tests locally

Running the tests locally should give you something like:
image

Running in CI

In CI a github markdown summary should be generated for somewhat easier parsing, of the wheels that were tested.

Adding extra wheels

To add extra wheels the wheels.toml file can be modified and a PR can be made.

Further improvements

  • Because we collect the test results during a run, we could use this to test the wheel with uv should a failure occur.
  • We might need to pin versions of wheels (I didn't want to do this yet, so we test with the latests) but this might prove annoying
  • We might needs skipping of certain wheels for platforms
  • We might need to test more python versions.

@tdejager tdejager changed the title wip: testing common wheels test: testing common wheels Sep 12, 2024
@ruben-arts
Copy link
Contributor

Nice this is great!

Copy link
Contributor

@Hofer-Julian Hofer-Julian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small, preliminary review

pixi.toml Outdated Show resolved Hide resolved
tests/wheel_tests/wheels.toml Show resolved Hide resolved
tdejager and others added 5 commits September 16, 2024 14:03
Co-authored-by: Hofer-Julian <[email protected]>
1. Use file locking, I saw file corruption in CI
2. make ci tests red again
3. Remove 2 failing wheels for windows
.github/workflows/test_common_wheels.yml Outdated Show resolved Hide resolved
pixi.toml Show resolved Hide resolved
@ruben-arts
Copy link
Contributor

Are we now sure it fails if a wheel fails to install?

@tdejager
Copy link
Contributor Author

Are we now sure it fails if a wheel fails to install?

Yes.

@ruben-arts ruben-arts merged commit 9b6ed85 into prefix-dev:main Sep 17, 2024
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants