Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to install PyNaCl on Windows on Arm64 #775

Open
prayaas-a opened this issue Oct 19, 2022 · 8 comments
Open

Unable to install PyNaCl on Windows on Arm64 #775

prayaas-a opened this issue Oct 19, 2022 · 8 comments

Comments

@prayaas-a
Copy link

prayaas-a commented Oct 19, 2022

When you try to install PyNaCl using pip (with Python 3.11 which is native to Arm64 on Windows), pip tries to build PyNaCl from scratch which doesn't seem possible on most Windows systems given the dependency on "make". This is because the Windows on Arm64 version of pip has only "win_arm64" and "any" as compatible tags, so the prebuilt wheels for x86 and x64 don't work.

Could you consider publishing a prebuilt wheel for the win_arm64 target as well? I believe Visual Studio 2019 includes support for cross-compiling for Arm64, but you should be able to use VS2022 if not. It looks like libsodium isn't publishing Arm64 binaries for Windows yet, but it should be a fairly trivial task for them considering they're using MSVC 2019 and 2022. I can also open an issue with them if that's going to be a hard dependency. Thanks.

@reaperhulk
Copy link
Member

Please do file the upstream issue. We'd like to support this, although lack of arm64 windows CI with GHA makes this a bit worrisome to ship since we can't test it automatically.

@prayaas-a
Copy link
Author

Please do file the upstream issue. We'd like to support this, although lack of arm64 windows CI with GHA makes this a bit worrisome to ship since we can't test it automatically.

Thanks for the response. I have created an upstream issue here. libsodium already supports building for an Arm64 target, but they haven't published binaries yet.

As far as GitHub Actions, yeah, that's a bit problematic. I'm not sure how well it integrates into GitHub Actions but you should be able to use Azure VMs and/or pipelines as they've begun offering Windows on Arm64 images.

@reaperhulk
Copy link
Member

The reliability of CI drops off pretty rapidly as we add multiple CI services (and we already use both GHA and CircleCI), but could you point me at where pipelines offers a microsoft-hosted arm64 windows runner? I haven't been able to find anything about native arm64 windows there.

@prayaas-a
Copy link
Author

prayaas-a commented Oct 21, 2022

Yes, that's a real concern. Ideally you don't want to have to deal with complex processes with moving parts. I see that Microsoft does not currently offer Microsoft-hosted Arm64 agents; however, you can create a Windows Arm64 VM on Azure, connect to it, and install the agent software on it manually. It can then be used by Azure Pipelines like any other build agent/pool.

When you create a VM on Azure, choose one of the Arm options for size (I selected Standard D2pdsv5). For the image, the "Windows 11 Enterprise - Gen 2" options appears to support Arm64. You can also click "See all images" and search for Arm to find one. Once the VM is set up, you can install the x64 version of the Azure Pipelines agent on it as usual. I can confirm that it works fine.

@reaperhulk
Copy link
Member

Self-hosted isn’t a viable option for us since it doesn’t provide the security boundary of ephemerality nor the maintenance properties we need (it’s not practical for us to monitor and patch a persistent instance). So while that would hypothetically work it’s not a path we’re willing to pursue. Supporting this really does need some support from a CI provider (and hey, Azure pipelines and GitHub just happen to be owned by the company that also produces Windows! 😄)

@echoix
Copy link

echoix commented Jan 22, 2023

I happened to stumble upon this when looking why PyNaCl had to be compiled in our docker images since no adequate wheel was available, and maybe this could help both of us. Have you ever taken a look at https://cibuildwheel.readthedocs.io/en/stable/? It seems to be a tool that is there to help specifically with the problem of building for multiple platforms. In the case of Windows arm64, it is indeed not easily available for CI. We have to keep in mind that Windows server 2019 and 2022 don't have any ARM releases at all. And we have to remember that we learned (as a public) in Nov 2021 that Qualcomm had an exclusivity for Windows on ARM, explaining the limited devices available. All that to say, that the approach cibuildwheel has in that case to help out with cross compiling is pretty appropriate, and probably way faster anyway. Because of this cross-compiling, and the fact explained above that a legal Windows on ARM version doesn't exist to be emulated, it is not really possible to "test" the Windows arm64 wheels that are built.

@alex
Copy link
Member

alex commented Jan 22, 2023

We're aware of cibuildwheel, but it doesn't solve the problem of the fact that we will not distribute wheels that we cannot test.

If MS has made it impossible to actually have arm64 in CI, then that problem needs to be directed at them, we're not going ship untested things.

@mihir1739
Copy link

I tried building paramiko that has a dependency on pynacl. Although the build for cryptography was successful, and I have even installed OpenSSL. Yet the build for pynacl fails with a cryptic error message. Can anyone please help.
Error messages:

Building wheels for collected packages: pynacl
  Building wheel for pynacl (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for pynacl (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [41 lines of output]
      C:\Users\mihir\AppData\Local\Temp\pip-build-env-coskp_n9\overlay\Lib\site-packages\setuptools\_distutils\dist.py:261: UserWarning: Unknown distribution option: 'tests_require'
        warnings.warn(msg)
      running bdist_wheel
      running build
      running build_py
      creating build\lib.win-arm64-cpython-313\nacl
      copying src\nacl\encoding.py -> build\lib.win-arm64-cpython-313\nacl
      copying src\nacl\exceptions.py -> build\lib.win-arm64-cpython-313\nacl
      copying src\nacl\hash.py -> build\lib.win-arm64-cpython-313\nacl
      copying src\nacl\hashlib.py -> build\lib.win-arm64-cpython-313\nacl
      copying src\nacl\public.py -> build\lib.win-arm64-cpython-313\nacl
      copying src\nacl\secret.py -> build\lib.win-arm64-cpython-313\nacl
      copying src\nacl\signing.py -> build\lib.win-arm64-cpython-313\nacl
      copying src\nacl\utils.py -> build\lib.win-arm64-cpython-313\nacl
      copying src\nacl\__init__.py -> build\lib.win-arm64-cpython-313\nacl
      creating build\lib.win-arm64-cpython-313\nacl\pwhash
      copying src\nacl\pwhash\argon2i.py -> build\lib.win-arm64-cpython-313\nacl\pwhash
      copying src\nacl\pwhash\argon2id.py -> build\lib.win-arm64-cpython-313\nacl\pwhash
      copying src\nacl\pwhash\scrypt.py -> build\lib.win-arm64-cpython-313\nacl\pwhash
      copying src\nacl\pwhash\_argon2.py -> build\lib.win-arm64-cpython-313\nacl\pwhash
      copying src\nacl\pwhash\__init__.py -> build\lib.win-arm64-cpython-313\nacl\pwhash
      creating build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\crypto_aead.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\crypto_box.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\crypto_core.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\crypto_generichash.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\crypto_hash.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\crypto_kx.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\crypto_pwhash.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\crypto_scalarmult.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\crypto_secretbox.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\crypto_secretstream.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\crypto_shorthash.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\crypto_sign.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\randombytes.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\sodium_core.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\utils.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\bindings\__init__.py -> build\lib.win-arm64-cpython-313\nacl\bindings
      copying src\nacl\py.typed -> build\lib.win-arm64-cpython-313\nacl
      running build_clib
      error: [WinError 193] %1 is not a valid Win32 application
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pynacl
Failed to build pynacl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

5 participants