CI setuptools run fails on macos but not on linux

[aprsa]

Switching our project from distutils to setuptools, we ran into an unusual issue on a macos runner: it fails to build the package from sources with

No eggs found in /var/folders/3s/vfzpb5r51gs6y328rmlgzm7c0000gn/T/easy_install-77zddpri/phoebe-2.4.11/egg-dist-tmp-wg9g973n (setup script problem?)
error: The 'phoebe==2.4.11' distribution was not found and is required by the application
Error: Process completed with exit code 1.

The strange thing is that phoebe 2.4.11 is precisely what setuptools is trying to build. So somehow it's trying to install itself as a prerequisite? Again, this only happens on a macos runner; linux runner runs fine. The full CI output is available here. I don't have a mac lying around to try and reproduce this locally, which makes things more difficult.

[abravalheri]

Hi @aprsa, I noticed 2 things in your logs:

1. There is a compilation error on macos, which is probably the root cause of the problem, you probably need to have a look on that. It might be related to numpy.distutils and compiler + OS-level dependencies (clang: error: unknown argument: '-mavx5124fmaps', clang: error: unknown argument: '-mavx5124vnniw', etc...)
2. The workflow is using a deprecated build and installation method (any attempt to run python setup.py as a CLI tool is now deprecated, setup.py changed to be only a configuration file that happens to be written in Python, similarly to noxfile.py or conftest.py). Please have a look on how to run python -m build to build/install projects and pip install ./dist/path-to-your-wheel for installing (more info in https://packaging.python.org/en/latest/guides/modernize-setup-py-project/#what-is-the-build-isolation-feature).

[aprsa]

Thanks for the quick reply, @abravalheri!

I don't think that (1) is the root cause; if I'm interpreting the logs correctly, this is the sequence of events that I'm seeing:

1. the build system runs^(*) python setup.py build && python setup.py install --user;
2. the build system correctly identifies the OS, the OS version, and the compiler to be used;
3. the build system creates an egg and installs it properly;
4. the build system then downloads the official phoebe 2.4.11 release tarball from pypi as a dependency and tries to build it. This is the old version that still relies on distutils and everything starts to break left and right. I believe that this is the crux of the issue: why would CI/setuptools want to grab the tarball from pypi in the first place? There doesn't seem to be any build failure before that point.

^(*) I am well aware that this is deprecated and that's the next thing to update in the queue, but currently I don't think that's where the issue is.

[abravalheri]

Umm, tricky one.

I don't know the answers for these questions myself, also it is hard to reason about the setup.py in the project because it is heavily customised.

But setuptools (itself) only tries to download stuff from PyPI when a deprecated flow is employed¹ (e.g. installing eggs). So if you use the new recommended build steps this problem might go away (considering that there is no compilation error as you pointed out).

Please note that there is no priority on support for deprecated blocks of setuptools, so if something deprecated is causing problems, the best is to not use the deprecated functionality.

Footnotes

1. In theory is also possible to customise the build process via setup.py so that something is downloaded during the build, but setuptools does not do that on its own in non-deprecated workflows. ↩