Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: pyqtdeploy-sysroot sip-module didn't create a sdist. #43

Open
stone-cloud opened this issue May 14, 2024 · 13 comments
Open

Bug: pyqtdeploy-sysroot sip-module didn't create a sdist. #43

stone-cloud opened this issue May 14, 2024 · 13 comments
Labels
Priority: Critical Issue is critical to the working of the system Target: pyqtdeploy The target of the issue is pyqtdeploy Type: Bug Issue related to the unexpected behaviour of an existing feature

Comments

@stone-cloud
Copy link

stone-cloud commented May 14, 2024

Hi,@achille-martin
thank you for your amazing meticulous job and selfless open source.
according to tutorial, I have an issue while implementing step 1.7. I couldn't find a solution to the relevant problem online, if you know how to solve it, please let me know and Thank you so much!
The error message is as follows:

$ python3 build_app.py --pdt $PYQT_CROM_DIR/examples/demo/demo_project/config.pdt --jobs 1 --target android-64 --qmake $QT_DIR/android/bin/qmake --verbose

----- REVIEWING COMMAND-LINE ARGS -----

[INFO] The .pdt path received is: /home/gw/LYH/androidDepoly/pyqt-crom/examples/demo/demo_project/config.pdt
[INFO] The number of jobs received is: 1
[INFO] The qmake path received is: /home/gw/Qt/5.15.2/android/bin/qmake
[INFO] The request to reload the sysroot is: False
[INFO] The request to disable progress messages is: False
[INFO] The request to enable verbose progress messages is: True

----- INITIALISING AND COLLECTING VARIABLES -----

[INFO] Pdt directory location is: /home/gw/LYH/androidDepoly/pyqt-crom/examples/demo/demo_project. This is the reference directory.
[INFO] The sysroot path received is: /home/gw/LYH/androidDepoly/pyqt-crom/examples/demo/demo_project/sysroot.toml
[INFO] Setting application name to: DemoCrossPlatformApp
[INFO] The app entrypoint name received is: demo_app
[INFO] The app package path received is: /home/gw/LYH/androidDepoly/pyqt-crom/examples/demo/demo_project/demo_pkg
[INFO] The app release dir is set to: /home/gw/LYH/androidDepoly/pyqt-crom/examples/demo/demo_project/demo_pkg/../releases/2024_05_14-08_32_08

----- BUILDING TARGET SYSROOT -----

Verifying host architecture 'linux-64'...
Verifying target architecture 'android-64'...
zlib: verifying...
zlib: determining installed version from '/home/gw/Android/Sdk/ndk/21.4.7075529/sysroot/usr/include/zlib.h'.
zlib: verified v1.2.7.
Qt: verifying...
Running '/home/gw/Qt/5.15.2/android/bin/qmake -query'.
Qt: verified v5.15.2.
Python: verifying...
WARNING: Python: v3.8.19 is untested.
Python: verified v3.8.19.
SIP: verifying...
Running 'sip-module --version'.
SIP: verified v6.7.12.
PyQt: verifying...
PyQt: verified v5.15.10.
Deleting /home/gw/LYH/androidDepoly/pyqt-crom/examples/demo/demo_project/sysroot-android-64/build.
Creating /home/gw/LYH/androidDepoly/pyqt-crom/examples/demo/demo_project/sysroot-android-64/build.
SIP: installing component...
Running 'sip-module --sdist PyQt5.sip --abi-version 12'.
pyqtdeploy-sysroot: SIP: sip-module didn't create an sdist.

system information:

Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.6 LTS
Release:	20.04
Codename:	focal

Linux gw-System-Product-Name 5.15.0-105-generic #115~20.04.1-Ubuntu SMP Mon Apr 15 17:33:04 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

@achille-martin
Copy link
Owner

achille-martin commented May 14, 2024

Hi @stone-cloud,

Thank you for your interest!
And thanks for the debug prints, that is useful.

I am working on bug templates and a debugging tool for pyqt-crom, but it will be released in the next version, so for now we have to debug "manually".

First of all, I can see that you are using:

  • Ubuntu 20.04
  • Python 3.8.19

Now for your bug that might not be a problem, but for full support, I would recommend you to use:

  • Ubuntu 22.04
  • Python 3.10.12

That's quite restrictive for now because I can't keep track of thousands of specs!

Anyway, regarding your bug:

Observations

  • sip-module didn't create an sdist. tells me that sip-module encountered a problem

Research

  • In fact, the command sip-module --sdist PyQt5.sip --abi-version 12 didn't create the source archive required. Information about this can be found in pyqtdeploy-3.3.0/pyqtdeploy/sysroot/plugins/SIP.py.
  • Furthermore, pyqtdeploy-3.3.0/pyqtdeploy/sysroot/plugins/SIP.py tells us that SIP requires ['Python', 'Qt'] to be pre-installed. Which does not seem to be the case in your log.

Potential solutions

Here are some solutions to try out:

  • There might be something odd with PyQt5-sip module -> Can you run the command pip3 show PyQt5-sip (so I can know which version you are using -> it should be 12.13.0 according to requirements.txt).
  • Try to run the failed command in verbose mode: sip-module --sdist PyQt5.sip --abi-version 12 (but I am not sure that the flag --verbose will work) -> so observe the output in case errors are showing up
  • Move the [SIP] section to the bottom in your sysroot.toml and remove all the sysroot-android-64 generated folders to avoid caching issues and then rebuild the app

@achille-martin achille-martin changed the title pyqtdeploy-sysroot: SIP: sip-module didn't create an sdist. Bug: pyqtdeploy-sysroot sip-module didn't create an sdist. May 14, 2024
@stone-cloud
Copy link
Author

Hi@achille-martin,
First of all, I test Python-3.8.19 and Python-3.10.12 in the same Ubuntu20.04 system, and they all have this Bug.
But I still can't be sure if it's unconcerned.
I tried all the solutions, but none of them worked.
The detailed information is as follows:

Try solutions

There might be something odd with PyQt5-sip module -> Can you run the command pip3 show PyQt5-sip (so I can know which version you are using -> it should be 12.13.0 according to requirements.txt).

Name: PyQt5-sip
Version: 12.13.0
Summary: The sip module support for PyQt5
Home-page: https://www.riverbankcomputing.com/software/sip/
Author: Riverbank Computing Limited
Author-email: [email protected]
License: SIP
Location: /home/gw/anaconda3/envs/torch19/lib/python3.8/site-packages
Requires: 
Required-by: PyQt5
Name: PyQt5
Version: 5.15.10
Summary: Python bindings for the Qt cross platform application toolkit
Home-page: https://www.riverbankcomputing.com/software/pyqt/
Author: Riverbank Computing Limited
Author-email: [email protected]
License: GPL v3
Location: /home/gw/anaconda3/envs/torch19/lib/python3.8/site-packages
Requires: PyQt5-Qt5, PyQt5-sip
Required-by: pyqtdeploy
Package                Version
---------------------- ------------
coloredlogs            15.0.1
dill                   0.3.8
flatbuffers            24.3.25
humanfriendly          10.0
mkl-fft                1.3.8
mkl-random             1.2.4
mkl-service            2.4.0
mpmath                 1.3.0
numpy                  1.24.3
onnx                   1.16.0
onnxconverter-common   1.13.0
onnxruntime-gpu        1.17.1
onnxruntime-tools      1.7.0
opencv-python-headless 4.9.0.80
packaging              23.2
pillow                 10.3.0
pip                    24.0
pipdeptree             2.18.1
ply                    3.11
protobuf               3.20.3
psutil                 5.9.8
py-cpuinfo             9.0.0
py3nvml                0.2.7
PyQt-builder           1.15.3
PyQt5                  5.15.10
PyQt5-Qt5              5.15.2
PyQt5-sip              12.13.0
pyqtdeploy             3.3.0
setuptools             69.5.1
sip                    6.7.12
sympy                  1.12
toml                   0.10.2
tomli                  2.0.1
torch                  1.9.0+cu111
torchaudio             0.9.0
torchvision            0.10.0+cu111
typing_extensions      4.11.0
wheel                  0.43.0
xmltodict              0.13.0

Try to run the failed command in verbose mode: sip-module --sdist PyQt5.sip --abi-version 12 (but I am not sure that the flag --verbose will work) -> so observe the output in case errors are showing up

I tried, but no information was printed. 😫

Move the [SIP] section to the bottom in your sysroot.toml and remove all the sysroot-android-64 generated folders to avoid caching issues and then rebuild the app

I tried, but it is not worked.

@achille-martin
Copy link
Owner

Ok, let's take it a step at a time then, since there are no debug prints associated to sip-module.

  1. Run sip-module --sdist PyQt5.sip --abi-version 12 anywhere

It should download in the current directory the archive for PyQt5-sip matching with your installed version.
So you should see a folder called PyQt5_sip-12.13.0.tar.gz in your current directory.

If you don't see it -> there is an issue with your sip or PyQt5-sip and I would recommend you pip3 uninstall and pip3 cache purge && pip3 install them back again.

If you see it -> then there might be an issue with your pyqtdeploy and we will need to add debug prints to $PYQT_CROM_DIR/venv/pyqt-crom-venv/lib/python3.10/site-packages/pyqtdeploy/sysroot/plugins/SIP.py.

@stone-cloud
Copy link
Author

Amazing!Base on your guidance, I found pyqt5_sip-12.13.0.tar.gz in my current directory. So I debug $PYQT_CROM_DIR/venv/pyqt-crom-venv/lib/python3.10/site-packages/pyqtdeploy/sysroot/plugins/SIP.py.

Debug

I found that glob can't recognize the pyqt5_sip-12.13.0.tar.gz due to a mismatch in the capitalization of the instructions archives = glob.glob(pattern)(https://www.riverbankcomputing.com/hg/pyqtdeploy/file/df680b956780/pyqtdeploy/sysroot/plugins/SIP.py). The file name is lowercase and the instructions contain uppercase.

Solutions

  1. First, What comes to my mind is to modify SIP module_name in pyqt-crom/examples/demo/demo_project/sysroot.toml. But it is not worked and print error.
  2. So I modify the code to archives = glob.glob(pattern.lower()) and it's working!

I'm not sure if this is a special case, such as the reason for the version number.

@achille-martin achille-martin changed the title Bug: pyqtdeploy-sysroot sip-module didn't create an sdist. Bug: pyqtdeploy-sysroot sip-module didn't create a sdist. May 17, 2024
@achille-martin
Copy link
Owner

achille-martin commented May 17, 2024

Glad you figured out the issue.

I suspect that the "lowercase" action is performed at some point in the sip package for some reason.

To investigate a bit further, can you:

  • Add a debug print print(args.sip_modules) (at the beginning of the try block) in $PYQT_CROM_DIR/venv/pyqt-crom-venv/lib/python3.10/site-packages/sipbuild/module/main.py
  • Run the command sip-module --sdist PyQt5.sip --abi-version 12
  • Share the output of the command, which should be ['PyQt5.sip']

If you get ['PyQt5.sip'], then the problem is likely later on in the code. Therefore, add debug prints in $PYQT_CROM_DIR/venv/pyqt-crom-venv/lib/python3.10/site-packages/sipbuild/module/module.pyto figure out when the lowercase operation happens (because it should not according tosip==6.7.12`).

If you get ['pyqt5.sip'], then the problem likely comes from the ArgumentParser (so argparse itself, which is built-in python3).

WARNINGS

  1. It is not recommended to modify the site-packages directory directly

This is because changes cannot be tracked and it creates confusion when you want to change versions.

The recommended way is to use pip in "edit" mode -> https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs

  1. It is not recommended to modify the pyqtdeploy package directly

This is because of the licence which forces you to disclose all the changes you made to the package before a release of your app. If you are happy doing that, then you are fine.

@stone-cloud
Copy link
Author

Hi @achille-martin ,

It's been a long time.

I tested the detailed output of '$PYQT_CROM_DIR/venv/pyqt-crom-venv/lib/python3.10/site-packages/sipbuild/module/module.py', and I found that when running the subprocess.run( [sys.executable, 'setup.py', '--quiet', 'sdist', '--dist-dir', '..'])command, the python package name, 'pyqt5_sip-12.13.0.tar.gz' is changed to lowercase letters.

I asked ChatGPT, and it told me that it might be because some systems are case-insensitive. And it suggested using all lowercase letters. Maybe it needs to consider the system version for testing in the next version.

@achille-martin
Copy link
Owner

achille-martin commented May 21, 2024

@stone-cloud thanks for your debugging efforts!

I am a bit surprised by the fact that subprocess.run( [sys.executable, 'setup.py', '--quiet', 'sdist', '--dist-dir', '..']) modifies the module_name, I thought it would not affect it as it is installing from setup.py instructions already populated.

If you are happy to proceed, can you replace your $PYQT_CROM_DIR/venv/pyqt-crom-venv/lib/python3.10/site-packages/sipbuild/module/module.py file with the one I have attached:

module.txt

can you replace your $PYQT_CROM_DIR/venv/pyqt-crom-venv/lib/python3.10/site-packages/sipbuild/module/main.py file with the one I have attached:

main.txt.txt

I have added a lot of debug prints and I have added verbose tag to suspicious commands - make sure to save your original files (in site-packages) before replacing their content (so you can revert back to your version, alternatively you can reinstall sip via pip).

Then run sip-module --sdist PyQt5.sip --abi-version 12.

Then please send me the terminal outputs so we can have a look.

@stone-cloud
Copy link
Author

stone-cloud commented May 22, 2024

The terminal outputs:

(androidDeploy) gw@gw-System-Product-Name:~/LYH/androidDepoly/QTgui$ sip-module --sdist PyQt5.sip --abi-version 12
main - ARGS: Namespace(abi_version='12', project=None, sdist=True, setup_cfg=None, sip_h=False, sip_rst=False, target_dir=None, sip_modules=['PyQt5.sip'])
module - SIP MODULE: PyQt5.sip
module - PROJECT: None
module - PROJECT 2: PyQt5_sip
_create_patches - SIP MODULE PARTS: ['PyQt5', 'sip']
_create_patches - SIP MODULE NAME: sip
_create_patches - SIP MODULE: PyQt5.sip
module - PATCHES: {'@SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@SIP_MODULE_PROJECT_NAME@': 'PyQt5_sip', '@SIP_MODULE_PACKAGE_NAME@': 'PyQt5', '@SIP_MODULE_VERSION@': '12.13.0', '@_SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@_SIP_MODULE_NAME@': 'sip', '@_SIP_MODULE_SHARED@': '1', '@_SIP_MODULE_ENTRY@': 'PyInit_sip', '@_SIP_MODULE_LEGACY@': '1', '@_SIP_VERSION@': '0x6070c', '@_SIP_VERSION_STR@': '6.7.12'}
module - SDIST DIR: PyQt5_sip-12.13.0
module - SDIST DIR 2: PyQt5_sip-12.13.0
_create_sdist - SDIST DIR: PyQt5_sip-12.13.0
_create_sdist - SDIST DIR 2: PyQt5_sip-12.13.0
_create_sdist - MODULE SOURCE DIR: /home/gw/anaconda3/envs/androidDeploy/lib/python3.10/site-packages/sipbuild/module/source/12
_create_sdist - SETUP CONFIG: None
_create_sdist - NAME FROM SOURCE DIR: MANIFEST.in
_create_sdist - NAME FROM SOURCE DIR: objmap.c
_create_sdist - NAME FROM SOURCE DIR: sip_array.h
_create_sdist - NAME FROM SOURCE DIR: siplib.c
_create_sdist - NAME FROM SOURCE DIR: sipint.h
_create_sdist - NAME FROM SOURCE DIR: LICENSE
_create_sdist - NAME FROM SOURCE DIR: threads.c
_create_sdist - NAME FROM SOURCE DIR: sip.rst.in
_create_sdist - NAME FROM SOURCE DIR: sip_array.c
_create_sdist - NAME FROM SOURCE DIR: setup.py.in
_install_source_file - module_source_dir: /home/gw/anaconda3/envs/androidDeploy/lib/python3.10/site-packages/sipbuild/module/source/12
name: setup.py
target_dir: PyQt5_sip-12.13.0
_install_file - NAME OUT: PyQt5_sip-12.13.0/setup.py
PATCHES: {'@SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@SIP_MODULE_PROJECT_NAME@': 'PyQt5_sip', '@SIP_MODULE_PACKAGE_NAME@': 'PyQt5', '@SIP_MODULE_VERSION@': '12.13.0', '@_SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@_SIP_MODULE_NAME@': 'sip', '@_SIP_MODULE_SHARED@': '1', '@_SIP_MODULE_ENTRY@': 'PyInit_sip', '@_SIP_MODULE_LEGACY@': '1', '@_SIP_VERSION@': '0x6070c', '@_SIP_VERSION_STR@': '6.7.12'}
_create_sdist - NAME FROM SOURCE DIR: bool.cpp
_create_sdist - NAME FROM SOURCE DIR: setup.cfg.in
_create_sdist - NAME FROM SOURCE DIR: LICENSE-GPL3
_create_sdist - NAME FROM SOURCE DIR: int_convertors.c
_create_sdist - NAME FROM SOURCE DIR: voidptr.c
_create_sdist - NAME FROM SOURCE DIR: apiversions.c
_create_sdist - NAME FROM SOURCE DIR: LICENSE-GPL2
_create_sdist - NAME FROM SOURCE DIR: sip.pyi
_create_sdist - NAME FROM SOURCE DIR: pyproject.toml
_create_sdist - NAME FROM SOURCE DIR: sip.h.in
_install_source_file - module_source_dir: /home/gw/anaconda3/envs/androidDeploy/lib/python3.10/site-packages/sipbuild/module/source/12
name: sip.h
target_dir: PyQt5_sip-12.13.0
_install_file - NAME OUT: PyQt5_sip-12.13.0/sip.h
PATCHES: {'@SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@SIP_MODULE_PROJECT_NAME@': 'PyQt5_sip', '@SIP_MODULE_PACKAGE_NAME@': 'PyQt5', '@SIP_MODULE_VERSION@': '12.13.0', '@_SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@_SIP_MODULE_NAME@': 'sip', '@_SIP_MODULE_SHARED@': '1', '@_SIP_MODULE_ENTRY@': 'PyInit_sip', '@_SIP_MODULE_LEGACY@': '1', '@_SIP_VERSION@': '0x6070c', '@_SIP_VERSION_STR@': '6.7.12'}
_create_sdist - NAME FROM SOURCE DIR: qtlib.c
_create_sdist - NAME FROM SOURCE DIR: descriptors.c
_create_sdist - NAME FROM SOURCE DIR: README.in
_install_source_file - module_source_dir: /home/gw/anaconda3/envs/androidDeploy/lib/python3.10/site-packages/sipbuild/module/source/12
name: README
target_dir: PyQt5_sip-12.13.0
_install_file - NAME OUT: PyQt5_sip-12.13.0/README
PATCHES: {'@SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@SIP_MODULE_PROJECT_NAME@': 'PyQt5_sip', '@SIP_MODULE_PACKAGE_NAME@': 'PyQt5', '@SIP_MODULE_VERSION@': '12.13.0', '@_SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@_SIP_MODULE_NAME@': 'sip', '@_SIP_MODULE_SHARED@': '1', '@_SIP_MODULE_ENTRY@': 'PyInit_sip', '@_SIP_MODULE_LEGACY@': '1', '@_SIP_VERSION@': '0x6070c', '@_SIP_VERSION_STR@': '6.7.12'}
_create_sdist - SETUP CONFIG 2: /home/gw/anaconda3/envs/androidDeploy/lib/python3.10/site-packages/sipbuild/module/source/12/setup.cfg.in
_install_file - NAME OUT: PyQt5_sip-12.13.0/setup.cfg
PATCHES: {'@SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@SIP_MODULE_PROJECT_NAME@': 'PyQt5_sip', '@SIP_MODULE_PACKAGE_NAME@': 'PyQt5', '@SIP_MODULE_VERSION@': '12.13.0', '@_SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@_SIP_MODULE_NAME@': 'sip', '@_SIP_MODULE_SHARED@': '1', '@_SIP_MODULE_ENTRY@': 'PyInit_sip', '@_SIP_MODULE_LEGACY@': '1', '@_SIP_VERSION@': '0x6070c', '@_SIP_VERSION_STR@': '6.7.12'}
_create_sdist - SAVED CWD: /home/gw/LYH/androidDepoly/QTgui
_create_sdist - SDIST DIR 2: PyQt5_sip-12.13.0
_create_sdist - SYS EXEC: /home/gw/anaconda3/envs/androidDeploy/bin/python
running sdist
running egg_info
creating PyQt5_sip.egg-info
writing PyQt5_sip.egg-info/PKG-INFO
writing dependency_links to PyQt5_sip.egg-info/dependency_links.txt
writing top-level names to PyQt5_sip.egg-info/top_level.txt
writing manifest file 'PyQt5_sip.egg-info/SOURCES.txt'
reading manifest file 'PyQt5_sip.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
adding license file 'LICENSE-GPL2'
adding license file 'LICENSE-GPL3'
writing manifest file 'PyQt5_sip.egg-info/SOURCES.txt'
running check
creating pyqt5_sip-12.13.0
creating pyqt5_sip-12.13.0/PyQt5_sip.egg-info
copying files to pyqt5_sip-12.13.0...
copying LICENSE -> pyqt5_sip-12.13.0
copying LICENSE-GPL2 -> pyqt5_sip-12.13.0
copying LICENSE-GPL3 -> pyqt5_sip-12.13.0
copying MANIFEST.in -> pyqt5_sip-12.13.0
copying README -> pyqt5_sip-12.13.0
copying apiversions.c -> pyqt5_sip-12.13.0
copying bool.cpp -> pyqt5_sip-12.13.0
copying descriptors.c -> pyqt5_sip-12.13.0
copying int_convertors.c -> pyqt5_sip-12.13.0
copying objmap.c -> pyqt5_sip-12.13.0
copying pyproject.toml -> pyqt5_sip-12.13.0
copying qtlib.c -> pyqt5_sip-12.13.0
copying setup.cfg -> pyqt5_sip-12.13.0
copying setup.py -> pyqt5_sip-12.13.0
copying sip.h -> pyqt5_sip-12.13.0
copying sip_array.c -> pyqt5_sip-12.13.0
copying sip_array.h -> pyqt5_sip-12.13.0
copying sipint.h -> pyqt5_sip-12.13.0
copying siplib.c -> pyqt5_sip-12.13.0
copying threads.c -> pyqt5_sip-12.13.0
copying voidptr.c -> pyqt5_sip-12.13.0
copying PyQt5_sip.egg-info/PKG-INFO -> pyqt5_sip-12.13.0/PyQt5_sip.egg-info
copying PyQt5_sip.egg-info/SOURCES.txt -> pyqt5_sip-12.13.0/PyQt5_sip.egg-info
copying PyQt5_sip.egg-info/dependency_links.txt -> pyqt5_sip-12.13.0/PyQt5_sip.egg-info
copying PyQt5_sip.egg-info/top_level.txt -> pyqt5_sip-12.13.0/PyQt5_sip.egg-info
copying PyQt5_sip.egg-info/SOURCES.txt -> pyqt5_sip-12.13.0/PyQt5_sip.egg-info
Writing pyqt5_sip-12.13.0/setup.cfg
Creating tar archive
removing 'pyqt5_sip-12.13.0' (and everything under it)
_create_sdist - SAVED CWD 2: /home/gw/LYH/androidDepoly/QTgui
_create_sdist - SDIST DIR 3: PyQt5_sip-12.13.0

and in my root, I get PyQt5_sip-12.13.0 fold and 'pyqt5_sip-12.13.0.tar.gz' file.

@achille-martin achille-martin added Priority: High Issue is not critical but needs to be addressed whenever possible Target: pyqtdeploy The target of the issue is pyqtdeploy Type: Bug Issue related to the unexpected behaviour of an existing feature labels May 22, 2024
@achille-martin
Copy link
Owner

Ok I can confirm that you are right about the command setup.py messing things up.

I dug a bit more into the setup template and I have added a few more debug prints to module.py:

module.txt

Again, if you have time, replace module.py with my extra verbose code and run sip-module --sdist PyQt5.sip --abi-version 12 and we'll inspect the output.

This time, the output will tell us from which python function the problem emerges (it might be the replace).

If this round comes back inconclusive, I'll close the issue and mark it as "temporarily fixed" with your previous fix.

@stone-cloud
Copy link
Author

All right! The print log information is in log file:

log.txt

If you locate where the problem occurred, please let me know.

@achille-martin achille-martin added Priority: Critical Issue is critical to the working of the system and removed Priority: High Issue is not critical but needs to be addressed whenever possible labels May 23, 2024
@achille-martin
Copy link
Owner

achille-martin commented May 23, 2024

Ah! I found it... I was looking at the wrong place...

Details of the issue

The "issue" comes from setuptools v69.3.0 Changelog infos here:

  • sip-module --sdist PyQt5.sip --abi-version 12 calls the command subprocess.run( [sys.executable, 'setup.py', '--quiet', 'sdist', '--dist-dir', '..'])
  • The command subprocess.run( [sys.executable, 'setup.py', '--quiet', 'sdist', '--dist-dir', '..']) calls the make_distribution() from command/sdist.py which calls the one from setuptools/_distutils/command/sdist.py
  • make_distribution() calls get_fullname() from setuptools/_core_metadata.py
  • get_fullname() calls canonicalize_name() from setuptools/_vendor/packaging/utils.py which is effectively distutils (https://github.com/pypa/distutils)

The changes have been made with this chronology:

Context of the issue

  • When the sip component is checked by the sysroot, the command sip-module --sdist PyQt5.sip --abi-version 12 is run to get the source distribution.
  • However, pyqtdeploy struggles to find the archive PyQt5_sip-12.13.0.tar.gz because the name has been canonicalised to pyqt5_sip-12.13.0.tar.gz by distutils.

Solutions

  1. Constrain setuptools to be < v69.3.0 in requirements.txt -> quite restrictive but works for now
  2. Modify self.module_name to be canonical in pyqtdeploy/pyqtdeploy/sysroot/plugins/SIP.py so that it can find the archive downloaded (the tar.gz) -> need to check with Phil
  3. Modify the value of module_name to pyqt5.sip in sysroot.toml under [SIP]

Thanks for your debugging support!

@achille-martin
Copy link
Owner

TL;DR: for now, use pip install setuptools==69.2.0

@stone-cloud
Copy link
Author

That's great! We can continue doing this great work. 😉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority: Critical Issue is critical to the working of the system Target: pyqtdeploy The target of the issue is pyqtdeploy Type: Bug Issue related to the unexpected behaviour of an existing feature
Projects
None yet
Development

No branches or pull requests

2 participants