Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move swift and dxpy into extras #150

Draft
wants to merge 18 commits into
base: master
Choose a base branch
from

Conversation

jtratner
Copy link
Contributor

@jtratner jtratner commented Apr 19, 2024

Follow-on to #143 . Makes swift and dxpy actually optional so people can opt into complex dependencies rather than being forced to use 'em, yay!

Tested this out in a codespace:

@jtratner ➜ /workspaces/stor (move-swift-and-dxpy-into-extras) $ poetry build
@jtratner ➜ /workspaces/stor (move-swift-and-dxpy-into-extras) $ for extratype in dxpy dxpy,swift swift; do rm -rf example-env/ && virtualenv example-env && source example-env/bin/activate && pip install 'dist/stor-4.2.0-py3-none-any.whl['$extratype']' && pip list > $extratype.txt && deactivate; done

New traceback:

Traceback (most recent call last):
  File "/workspaces/stor/stor/dx.py", line 21, in <module>
    import dxpy
ModuleNotFoundError: No module named 'dxpy'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/workspaces/stor/stor/base.py", line 47, in __new__
    cls = utils.find_dx_class(path)
  File "/workspaces/stor/stor/utils.py", line 273, in find_dx_class
    from stor.dx import DXPath, DXCanonicalPath, DXVirtualPath
  File "/workspaces/stor/stor/dx.py", line 25, in <module>
    raise utils.missing_storage_library_exception("dx", e) from e
ImportError: ModuleNotFoundError: No module named 'dxpy'
To use a 'dx' path, stor needs an additional python library. Please specify it as an extra in the installation.
i.e.,: `pip install stor[dx]` or `stor[dx] >= 5` in requirements.txt or `poetry add stor[dx]`.
Alternatively, change an existing pyproject.toml file to specify `stor = {version="5.0", extras = dx}`

And then showed results

`cat dxpy.txt`
@jtratner ➜ /workspaces/stor (move-swift-and-dxpy-into-extras) $ cat dxpy.txt 
Package            Version
------------------ -----------
argcomplete        3.3.0
boto3              1.34.90
botocore           1.34.90
cached-property    1.5.2
certifi            2024.2.2
charset-normalizer 3.3.2
dxpy               0.373.0
idna               3.7
jmespath           1.0.1
pip                24.0
psutil             5.9.8
python-dateutil    2.9.0.post0
requests           2.31.0
s3transfer         0.10.1
setuptools         69.5.1
six                1.16.0
stor               4.2.0
urllib3            2.1.0
websocket-client   0.54.0
wheel              0.43.0
`cat swift.txt`
@jtratner ➜ /workspaces/stor (move-swift-and-dxpy-into-extras) $ cat swift.txt 
Package               Version
--------------------- -----------
boto3                 1.34.90
botocore              1.34.90
cached-property       1.5.2
certifi               2024.2.2
charset-normalizer    3.3.2
debtcollector         3.0.0
idna                  3.7
iso8601               2.1.0
jmespath              1.0.1
keystoneauth1         5.6.0
msgpack               1.0.8
netaddr               1.2.1
netifaces             0.11.0
os-service-types      1.7.0
oslo.config           9.4.0
oslo.i18n             6.3.0
oslo.serialization    5.4.0
oslo.utils            7.1.0
packaging             24.0
pbr                   6.0.0
pip                   24.0
pyparsing             3.1.2
python-dateutil       2.9.0.post0
python-keystoneclient 5.4.0
python-swiftclient    4.5.0
PyYAML                6.0.1
requests              2.31.0
rfc3986               2.0.0
s3transfer            0.10.1
setuptools            69.5.1
six                   1.16.0
stevedore             5.2.0
stor                  4.2.0
tzdata                2024.1
urllib3               2.2.1
wheel                 0.43.0
wrapt                 1.16.0
`cat dxpy,swift.txt`
@jtratner ➜ /workspaces/stor (move-swift-and-dxpy-into-extras) $ cat dxpy,swift.txt 
Package               Version
--------------------- -----------
argcomplete           3.3.0
boto3                 1.34.90
botocore              1.34.90
cached-property       1.5.2
certifi               2024.2.2
charset-normalizer    3.3.2
debtcollector         3.0.0
dxpy                  0.373.0
idna                  3.7
iso8601               2.1.0
jmespath              1.0.1
keystoneauth1         5.6.0
msgpack               1.0.8
netaddr               1.2.1
netifaces             0.11.0
os-service-types      1.7.0
oslo.config           9.4.0
oslo.i18n             6.3.0
oslo.serialization    5.4.0
oslo.utils            7.1.0
packaging             24.0
pbr                   6.0.0
pip                   24.0
psutil                5.9.8
pyparsing             3.1.2
python-dateutil       2.9.0.post0
python-keystoneclient 5.4.0
python-swiftclient    4.5.0
PyYAML                6.0.1
requests              2.31.0
rfc3986               2.0.0
s3transfer            0.10.1
setuptools            69.5.1
six                   1.16.0
stevedore             5.2.0
stor                  4.2.0
tzdata                2024.1
urllib3               2.1.0
websocket-client      0.54.0
wheel                 0.43.0
wrapt                 1.16.0

dxpy pulls in a wide array of (often legacy) packages, skipping the import means
we only need to pull it in as necessary which hopefully makes library
more useful to others.
make dxpy into an extra as well
include with and without dxpy
Rather than leave the user hanging, show something helpful to the user
so they can resolve the situation by installing an additional package.

Left this as a separate function so it's easy to test the exception
generation fucntion doesn't break without needing to do painful testing
contortions. (which should be caught later on by test matrix)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant