Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sbpy in conda environment causes pytest collection to fail in Python<3.11 #368

Open
astronomerritt opened this issue Jan 20, 2023 · 11 comments
Open

Sbpy in conda environment causes pytest collection to fail in Python<3.11 #368

astronomerritt opened this issue Jan 20, 2023 · 11 comments
Labels
potential bug automated label to indicate issue reports

Comments

@astronomerritt
Copy link

astronomerritt commented Jan 20, 2023 •
edited
Loading

Creating a conda environment with sbpy causes pytest to fail to collect every single test due to a RuntimeWarning in Python<3.11.

The environments are created via:

conda create -n sspp_test -c conda-forge -c mjuric python=3.10 spiceypy=4.0.1 openorb numpy pandas matplotlib spice-utils pip pytest astropy scipy sbpy pytables

Then running pytest on this repo produces the following error during test collection for every single test:

____________________________________________________ ERROR collecting surveySimPP/modules/tests/test_PPVignetting.py ____________________________________________________
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:992: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:883: in exec_module
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
surveySimPP/__init__.py:1: in <module>
    from . import modules
surveySimPP/modules/__init__.py:3: in <module>
    from . import PPCalculateApparentMagnitude
surveySimPP/modules/PPCalculateApparentMagnitude.py:3: in <module>
    from .PPCalculateApparentMagnitudeInFilter import PPCalculateApparentMagnitudeInFilter
surveySimPP/modules/PPCalculateApparentMagnitudeInFilter.py:6: in <module>
    from sbpy.photometry import HG, HG1G2, HG12_Pen16, LinearPhaseFunc
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/sbpy/photometry/__init__.py:5: in <module>
    from .core import *
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/sbpy/photometry/core.py:16: in <module>
    from astropy.modeling import (Fittable1DModel, Parameter)
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/modeling/__init__.py:10: in <module>
    from . import fitting, models
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/modeling/fitting.py:39: in <module>
    from .spline import (  # noqa: F401
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/modeling/spline.py:15: in <module>
    from .core import FittableModel, ModelDefinitionError
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/modeling/core.py:29: in <module>
    from astropy.nddata.utils import add_array, extract_array
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/nddata/__init__.py:14: in <module>
    from .blocks import *
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/nddata/blocks.py:8: in <module>
    from .decorators import support_nddata
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/nddata/decorators.py:12: in <module>
    from .nddata import NDData
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/nddata/nddata.py:12: in <module>
    from astropy.wcs.wcsapi import (
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/wcs/__init__.py:26: in <module>
    from . import utils
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/wcs/utils.py:8: in <module>
    from astropy.coordinates import ITRS, CartesianRepresentation, SphericalRepresentation
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/coordinates/__init__.py:13: in <module>
    from .builtin_frames import *
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/coordinates/builtin_frames/__init__.py:30: in <module>
    from .cirs import CIRS
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/coordinates/builtin_frames/cirs.py:8: in <module>
    from .utils import DEFAULT_OBSTIME, EARTH_CENTER
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/coordinates/builtin_frames/utils.py:15: in <module>
    from astropy.utils import iers
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/utils/iers/__init__.py:1: in <module>
    from .iers import *
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/utils/iers/iers.py:23: in <module>
    from astropy.table import MaskedColumn, QTable
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/table/__init__.py:6: in <module>
    from .column import Column, ColumnInfo, MaskedColumn, StringTruncateWarning
/opt/miniconda3/envs/sspp_test_310/lib/python3.10/site-packages/astropy/table/column.py:20: in <module>
    from ._column_mixins import _ColumnGetitemShim, _MaskedColumnGetitemShim
astropy/table/_column_mixins.pyx:1: in init astropy.table._column_mixins
    ???
E   RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 96 from PyObject

As can be seen, the traceback leads to sbpy and then to astropy.

Tests run as expected in Python 3.11, no warnings. There's a full writeup in this issue.

For both Python 3.10 and Python 3.11:
Operating system and version: MacOS Monterey 12.6.2
sbpy version: 0.3.0
astropy version: 5.2.1
numpy version: 1.24.1

Thanks in advance for any help!
@astronomerritt astronomerritt added the potential bug automated label to indicate issue reports label Jan 20, 2023
@astronomerritt astronomerritt changed the title Sbpy and pytables in same conda environment causes pytest collection to fail in Python<3.11 Sbpy in conda environment causes pytest collection to fail in Python<3.11 Jan 20, 2023
@astronomerritt
Copy link
Author

Additional important note: pip-installing sbpy does not cause this problem to occur.

@mkelley
Copy link
Member

mkelley commented Jan 23, 2023

When I search on this error, I see a lot of references to upgrading or reinstalling numpy. Does pip installing sbpy affect the numpy version or does it pip install another numpy?

@astronomerritt
Copy link
Author

No, the numpy version remains the same: 1.24.1.

The pip install of sbpy only installs the following:

Successfully installed MarkupSafe-2.1.2 ads-0.12.3 astroquery-0.4.6 beautifulsoup4-4.11.1 html5lib-1.1 httpretty-0.8.10 jaraco.classes-3.2.3 keyring-23.13.1 mock-5.0.1 more-itertools-9.0.0 pyvo-1.4 sbpy-0.3.0 soupsieve-2.3.2.post1 synphot-1.1.1 webencodings-0.5.1 werkzeug-2.2.2

@astronomerritt
Copy link
Author

astronomerritt commented Jan 25, 2023 •
edited
Loading

If it's any help, the conda install does work with no test collection failure for Python 3.6 and sbpy 0.2.2: this installs numpy 1.19.5 and astropy 3.2.3. However the issue resumes with Python 3.7 (sbpy 0.3.0, numpy 1.21.6, astropy 4.3.1).

@astronomerritt
Copy link
Author

I'm still having this issue - any advice? We're looking to release soon, but we're unwilling to do so unless we can trust that a full conda environment will work without issues.

@mkelley
Copy link
Member

mkelley commented May 11, 2023

@astronomerritt Apologies for missing this message! I do have a PR that updates a few things that might help. We'll get that merged ASAP.

@mkelley
Copy link
Member

mkelley commented May 11, 2023

Although it may be we need to do a release, since you are installing via conda. That will still take a couple weeks.

But maybe re-building the conda package could address this? @mwcraig , is this something that can be done without a release? And if there is something that I can do to make it happen, let me know.

@mkelley
Copy link
Member

mkelley commented May 11, 2023

Ah, got it. I'm seeing now that astropy ignores this warning. If we do the same, then the tests should pass. We will need to make a new release.

@mkelley
Copy link
Member

mkelley commented May 11, 2023

@astronomerritt In the meantime, are you able to add command line arguments to filter the warning out? Something like -W ignore:numpy.ndarray size changed:RuntimeWarning The documentation on the warnings filter is at https://docs.python.org/3/library/warnings.html#warning-filter

@astronomerritt
Copy link
Author

This is terrific news, thanks for getting back to this! We're absolutely fine to wait a couple weeks for the next conda release on this, our own release has been delayed, as these things often are.

@mwcraig
Copy link
Contributor

mwcraig commented May 12, 2023

@mkelley -- we could patch the source in the conda recipe as a temporary workaround but since a new release is coming soon it seems better to wait.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
potential bug automated label to indicate issue reports
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mkelley @mwcraig @astronomerritt