Pip install error #450
Comments
|
I'm not able to reproduce, but I'm using windows, so that's a significant difference. Is there a specific reason you are installing pyogrio via pip instead of via conda? |
|
I can't reproduce this with virtualenv with the same macos version and pip version
|
|
The issue is that it is trying to build pyogrio from source, which requires that GDAL be installed, with either the I'm not sure why pip is forcing an install from source, since @kylebarron just confirmed that the available MacOS 12 wheels install without issues. If you indeed cannot install pyogrio via conda and need to build from source. To do this, you'll need to have additional things installed from conda. Specifically, you'll need to install at least Cython and libgdal, and possibly a couple other packages. See the conda-forge recipe for building pyogrio for more info. |
|
I can't reproduce the issue by trying to pip install pyogrio into a fresh conda environment (successfully installs existing wheels, does not build from source), but I'm on MacOS 12, which matches the wheels, so maybe conda is letting me off easier than trying to do this on MacOS 14. |
|
@stdavis Are you running on Apple Silicon or Intel? It seems that it works on Apple Silicon but might be tied to wheels being not recognised on intel chips and forcing build from source? |
I need to install via pip because this is a dependency of a local project defined via a
Intel Are there missing wheels for intel-based macs? I'm getting a new Apple Silicon machine next week so I'm not too worried about it. But I'm happy to help test our new builds on my old machine. |
I don't think so as we build them and they are available https://pypi.org/project/pyogrio/#files. Maybe you can try directly pulling the wheel and installing it? I am not sure why your resolver falls back to building from source. |
|
|
|
Ugh. I think we found the reason. Thanks! |
|
I'm confused by why this is happening - are wheels built on MacOS 12 not forward compatible with MacOS 14? Or maybe just the intel wheels are not forward compatible? It seems per the results from Kyle Barron above that the Arm64 wheels built on MacOS 12 work properly on MacOS 14. We're building x86_64 and arm64 wheels using Github's MacOS 12 runner (using VCPKG). While it looks like we are testing wheels in our release action against the macos-latest runner (MacOS 14 Arm64 ), we actually are incorrectly testing against the latest available Arm64 version of pyogrio on PyPI see logs here. Will raise an issue for that. |
|
@stdavis could you run a few other things to try to understand why pip thinks that wheel is not compatible? Looking at https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/, maybe show the result of import sys
import sysconfig
print(sys.implementation.name)
print(sysconfig.get_platform())And can you also try with |
conda activate test
(test)
~ via 🅒 test
❯ python
Python 3.11.9 (main, Apr 19 2024, 11:44:45) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import sysconfig
>>> print(sys.implementation.name)
cpython
>>> print(sysconfig.get_platform())
macosx-10.9-x86_64python -m pip install pyogrio
Collecting pyogrio
Using cached pyogrio-0.9.0.tar.gz (352 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [31 lines of output]
Traceback (most recent call last):
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/var/folders/sg/2h57mpl95yv6tqtwsvhzp0340000gn/T/pip-build-env-m0isihth/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 368, in prepare_metadata_for_build_wheel
self.run_setup()
File "/private/var/folders/sg/2h57mpl95yv6tqtwsvhzp0340000gn/T/pip-build-env-m0isihth/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 497, in run_setup
super().run_setup(setup_script=setup_script)
File "/private/var/folders/sg/2h57mpl95yv6tqtwsvhzp0340000gn/T/pip-build-env-m0isihth/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 313, in run_setup
exec(code, locals())
File "<string>", line 154, in <module>
File "<string>", line 129, in get_gdal_config
File "<string>", line 85, in get_gdal_config
File "<string>", line 85, in <dictcomp>
File "<string>", line 39, in read_response
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.11/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.11/subprocess.py", line 548, in run
with Popen(*popenargs, **kwargs) as process:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.11/subprocess.py", line 1026, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.11/subprocess.py", line 1955, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'gdal-config'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
(test) |
|
Unless I'm misreading this, this is the issue right here (version of MacOS that Python is aware of is too old): >>> print(sysconfig.get_platform())
macosx-10.9-x86_64Above, you say this is on MacOS 14.5, so I'm entirely confused now as to why Python thinks it is on a MacOS 10.9 machine. Is this in a Docker container or some other virtualization between the Python environment and the host? |
|
(and this is related to #417 and the release note "MacOS wheels are now only available for macOS 12+. For older unsupported macOS versions, pyogrio can still be built from source (requires GDAL to be installed)") |
|
No docker container or other virtualization. These are the specs on my machine: |
|
Can you please try import platform
print(platform.platform())I'm finding locally that |
❯ python
Python 3.11.9 (main, Apr 19 2024, 11:44:45) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.platform()
'macOS-10.16-x86_64-i386-64bit' |
|
I'm not sure exactly what pip checks when determining what wheels to search for, but it seems that your install of Python is convinced that it is on MacOS 10.x, which no longer has pyogrio wheels on PyPI for >= 0.9.0. Is it possible that Conda is using an old, cached version of a Python 3.11 installation, if you installed that on MacOS 10.x before upgrading to MacOS 14? What happens if you create a brand new python 3.12 environment in conda (requiring fresh installation of 3.12). |
|
I install python via HomeBrew which I run updates on regularly. ❯ which python3
/usr/local/Caskroom/miniconda/base/bin/python3
(base)
~
❯ python3
Python 3.12.4 | packaged by Anaconda, Inc. | (main, Jun 18 2024, 10:14:12) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
(base)
~ took 3sI'm using a python 3.11 environment because the project that I'm working on does not yet support 3.12. However, I tried a fresh 3.12 conda environment this morning and it had the same result: ❯ conda create --name test python=3.12
Retrieving notices: ...working... done
Channels:
- defaults
Platform: osx-64
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /usr/local/Caskroom/miniconda/base/envs/test
added / updated specs:
- python=3.12
The following NEW packages will be INSTALLED:
bzip2 pkgs/main/osx-64::bzip2-1.0.8-h6c40b1e_6
ca-certificates pkgs/main/osx-64::ca-certificates-2024.7.2-hecd8cb5_0
expat pkgs/main/osx-64::expat-2.6.2-hcec6c5f_0
libcxx pkgs/main/osx-64::libcxx-14.0.6-h9765a3e_0
libffi pkgs/main/osx-64::libffi-3.4.4-hecd8cb5_1
ncurses pkgs/main/osx-64::ncurses-6.4-hcec6c5f_0
openssl pkgs/main/osx-64::openssl-3.0.14-h46256e1_0
pip pkgs/main/osx-64::pip-24.0-py312hecd8cb5_0
python pkgs/main/osx-64::python-3.12.4-hcd54a6c_1
readline pkgs/main/osx-64::readline-8.2-hca72f7f_0
setuptools pkgs/main/osx-64::setuptools-69.5.1-py312hecd8cb5_0
sqlite pkgs/main/osx-64::sqlite-3.45.3-h6c40b1e_0
tk pkgs/main/osx-64::tk-8.6.14-h4d00af3_0
tzdata pkgs/main/noarch::tzdata-2024a-h04d1e81_0
wheel pkgs/main/osx-64::wheel-0.43.0-py312hecd8cb5_0
xz pkgs/main/osx-64::xz-5.4.6-h6c40b1e_1
zlib pkgs/main/osx-64::zlib-1.2.13-h4b97444_1
Proceed ([y]/n)?
Downloading and Extracting Packages:
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate test
#
# To deactivate an active environment, use
#
# $ conda deactivate
(base)
~ took 56s
❯ conda activate test
(test)
~ via 🅒 test
❯ pip install pyogrio
Collecting pyogrio
Using cached pyogrio-0.9.0.tar.gz (352 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [30 lines of output]
Traceback (most recent call last):
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/var/folders/sg/2h57mpl95yv6tqtwsvhzp0340000gn/T/pip-build-env-phtc7z9c/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 368, in prepare_metadata_for_build_wheel
self.run_setup()
File "/private/var/folders/sg/2h57mpl95yv6tqtwsvhzp0340000gn/T/pip-build-env-phtc7z9c/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 497, in run_setup
super().run_setup(setup_script=setup_script)
File "/private/var/folders/sg/2h57mpl95yv6tqtwsvhzp0340000gn/T/pip-build-env-phtc7z9c/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 313, in run_setup
exec(code, locals())
File "<string>", line 154, in <module>
File "<string>", line 129, in get_gdal_config
File "<string>", line 85, in get_gdal_config
File "<string>", line 39, in read_response
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.12/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.12/subprocess.py", line 548, in run
with Popen(*popenargs, **kwargs) as process:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.12/subprocess.py", line 1026, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/local/Caskroom/miniconda/base/envs/test/lib/python3.12/subprocess.py", line 1955, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'gdal-config'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
(test)
~ via 🅒 test took 36s
❯ python
Python 3.12.4 | packaged by Anaconda, Inc. | (main, Jun 18 2024, 10:14:12) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.platform()
'macOS-10.16-x86_64-i386-64bit'
>>> exit()
(test)
~ via 🅒 test took 11s
❯I've found a suitable workaround for this issue (running |
|
Thanks for further testing. I'm glad you've got a workaround. I'm wondering if this is sensitive to setting |
|
@brendan-ward That is another interesting workaround: (base)
~
❯ echo $SYSTEM_VERSION_COMPAT
(base)
~
❯ conda activate test
(test)
~ via 🅒 test
❯ echo $SYSTEM_VERSION_COMPAT
(test)
~ via 🅒 test
❯ export SYSTEM_VERSION_COMPAT=0
(test)
~ via 🅒 test
❯ python
Python 3.12.4 | packaged by Anaconda, Inc. | (main, Jun 18 2024, 10:14:12) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.platform()
'macOS-14.5-x86_64-i386-64bit'
>>> exit()
(test)
~ via 🅒 test took 15s
❯ pip install pyogrio
Collecting pyogrio
Downloading pyogrio-0.9.0-cp312-cp312-macosx_12_0_x86_64.whl.metadata (3.8 kB)
Collecting certifi (from pyogrio)
Using cached certifi-2024.7.4-py3-none-any.whl.metadata (2.2 kB)
Collecting numpy (from pyogrio)
Downloading numpy-2.0.1-cp312-cp312-macosx_14_0_x86_64.whl.metadata (60 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.9/60.9 kB 1.0 MB/s eta 0:00:00
Collecting packaging (from pyogrio)
Using cached packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Downloading pyogrio-0.9.0-cp312-cp312-macosx_12_0_x86_64.whl (16.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.1/16.1 MB 14.3 MB/s eta 0:00:00
Using cached certifi-2024.7.4-py3-none-any.whl (162 kB)
Downloading numpy-2.0.1-cp312-cp312-macosx_14_0_x86_64.whl (6.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.6/6.6 MB 47.4 MB/s eta 0:00:00
Using cached packaging-24.1-py3-none-any.whl (53 kB)
Installing collected packages: packaging, numpy, certifi, pyogrio
Successfully installed certifi-2024.7.4 numpy-2.0.1 packaging-24.1 pyogrio-0.9.0
(test)
~ via 🅒 test took 25s
❯ |
|
I've run into what looks like the same error when testing python https://github.com/vega/altair/actions/runs/10861487260/job/30143440337?pr=3591 |
|
@dangotbanned given that you mention Python 3.13, you probably get an error because we do not yet have 3.13 wheels uploaded for the latest release. Therefore it also tries to install from source, giving a similar error (but the reason it is installing from source is different). We do have Python 3.13 wheels added on main, but have to make a release to make those available. I opened #470 to track doing a release shortly. |
|
Thanks @jorisvandenbossche, I still have a lot to learn regarding packaging so your explanation was really helpful. Appreciate you opening #470 I'll link that over on vega/altair#3587 |
When I attempt to pip install this package into a conda environment, I get the following error message:
Is there some prerequisite that I'm missing?
To reproduce
conda create --name test python=3.11conda activate testpip install pyogrioEnvironment
MacOS:
Sonoma 14.5Conda:
24.5.0Pip:
24.0The text was updated successfully, but these errors were encountered: