Cannot npm install if system Python version is newer than 3.10

[audiodude]

Steps to reproduce:

1. Clone a fresh copy of mwoffliner
2. npm install

Expected result:

The dependencies of the library build and install

Actual result:

Building @openzim/libzim fails with the following stack trace:

npm ERR! gyp verb gyp gyp format was not specified; forcing "make"
npm ERR! gyp info spawn /home/tmoney/.asdf/installs/python/3.12.0/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   '/home/tmoney/code/mwoffliner/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'make',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/home/tmoney/code/mwoffliner/node_modules/@openzim/libzim/build/config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/home/tmoney/code/mwoffliner/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/home/tmoney/.cache/node-gyp/18.18.0/include/node/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/home/tmoney/.cache/node-gyp/18.18.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/home/tmoney/code/mwoffliner/node_modules/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/home/tmoney/.cache/node-gyp/18.18.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/home/tmoney/code/mwoffliner/node_modules/@openzim/libzim',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! Traceback (most recent call last):
npm ERR!   File "/home/tmoney/code/mwoffliner/node_modules/node-gyp/gyp/gyp_main.py", line 50, in <module>
npm ERR!     sys.exit(gyp.script_main())
npm ERR!              ^^^^^^^^^^^^^^^^^
npm ERR!   File "/home/tmoney/code/mwoffliner/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 554, in script_main
npm ERR!     return main(sys.argv[1:])
npm ERR!            ^^^^^^^^^^^^^^^^^^
npm ERR!   File "/home/tmoney/code/mwoffliner/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 547, in main
npm ERR!     return gyp_main(args)
npm ERR!            ^^^^^^^^^^^^^^
npm ERR!   File "/home/tmoney/code/mwoffliner/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 520, in gyp_main
npm ERR!     [generator, flat_list, targets, data] = Load(
npm ERR!                                             ^^^^^
npm ERR!   File "/home/tmoney/code/mwoffliner/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 136, in Load
npm ERR!     result = gyp.input.Load(build_files, default_variables, includes[:],
npm ERR!              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
npm ERR!   File "/home/tmoney/code/mwoffliner/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 2782, in Load
npm ERR!     LoadTargetBuildFile(build_file, data, aux_data,
npm ERR!   File "/home/tmoney/code/mwoffliner/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 391, in LoadTargetBuildFile
npm ERR!     build_file_data = LoadOneBuildFile(build_file_path, data, aux_data,
npm ERR!                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
npm ERR!   File "/home/tmoney/code/mwoffliner/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 234, in LoadOneBuildFile
npm ERR!     build_file_contents = open(build_file_path, 'rU').read()
npm ERR!                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
npm ERR! ValueError: invalid mode: 'rU' while trying to load binding.gyp
npm ERR! gyp ERR! configure error 

Looks like it's this issue:

https://stackoverflow.com/questions/74715990/node-gyp-err-invalid-mode-ru-while-trying-to-load-binding-gyp

Workaround is to use a local Python version of 3.10

[kelson42]

I suspect because we rely on older version of gyp. We should just move to latest version of node-js and this will solve this issue as well IMHO.

Blocked by #1576

[audiodude]

Using latest node-libzim, which has updated gyp in openzim/node-libzim#139 fixes this problem. Indeed blocked.

[THEBOSS0369]

Hey @audiodude !
Thanks for sharing the stackoverflow link but it is like half works and half doesn't and took me few hours to solve this, so i am sharing the steps that worked for me, so that in future no Dev will face the issues.

I did these steps on Github Codespace because i didn't want to change my system's default setting.

First Have to install python 3.10.0 using pyenv this will solve the Problem.

pyenv install 3.10.0
pyenv global 3.10.0

Else
If it shows pyenv is not installed or something like that pyenv command not found then

curl https://pyenv.run | bash

then open .bashrc file using this command

nano ~/.bashrc

add this code in the file

export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init --path)"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

and then save & exit, then run this command to apply changes

source ~/.bashrc

Ensure node-gyp is installed if not run this

npm install -g node-gyp

Last set the Python for the Npm

npm config set python $(pyenv which python)
export PYTHON=$(pyenv which python)

To check run echo $PYTHON it will show /home/codespace/.pyenv/versions/3.10.0/bin/python it means you are on the right track.

That's it now run npm ci and you are done!

Thanks!