Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable creation of venv with existing python version #217

Open
xmatthias opened this issue Dec 29, 2024 · 3 comments
Open

Enable creation of venv with existing python version #217

xmatthias opened this issue Dec 29, 2024 · 3 comments
Labels
documentation Improvements or additions to documentation

Comments

@xmatthias
Copy link

xmatthias commented Dec 29, 2024 •
edited
Loading

the FAQ specifically mentions:

Using actions/setup-python can be faster, because GitHub caches the Python versions alongside the runner.

So as a consequence - i'm trying to use that - but use the astral-sh/setup-uv function to install UV.
Unfortunately, there seems to be no way to have uv setup a virtual environment - all while reusing the existing python installation.

The only way i found was through passing in python-version: ${{ matrix.python-version }} - but that installs a 2nd copy of python alongside the one installed via actions/setup-python - so contradicts the above FAQ entry (i could remove the setup-python action as it's python ain't used).

    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: ${{ matrix.python-version }}

    - name: Install uv
      uses: astral-sh/setup-uv@v5
      with:
        python-version: ${{ matrix.python-version }}
current workaround

(note: the below is for the windows variant - others will need slightly different workarounds)

    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: ${{ matrix.python-version }}

    - name: Install uv
      uses: astral-sh/setup-uv@v5

    - name: Installation
      run: |
        uv venv
        .venv\Scripts\activate
        # persist the venv path for future steps
        "$(pwd)/.venv/Scripts" >> $env:GITHUB_PATH
 ...

Desired solution:
A flag/parameter to allow UV to use the existing python installed via setup-python - but still do the venv magic (create venv, aktivate for the rest of the job, caching, ...)

    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: ${{ matrix.python-version }}

    - name: Install uv
      uses: astral-sh/setup-uv@v5
      with:
        create-venv: true
@xmatthias xmatthias changed the title Enable creation of venv with installed python version Enable creation of venv with existing python version Dec 29, 2024
@danielhollas
Copy link

danielhollas commented Jan 7, 2025 •
edited
Loading

The only way i found was through passing in python-version: ${{ matrix.python-version }} - but that installs a 2nd copy of python alongside the one installed via actions/setup-python

@xmatthias How did you determine that a 2nd copy of python is installed in this case? I want to start using this setup with v5 version of the action and I don't see anything in the logs that would suggest that an extra python version is installed. Here's what I see:

Run astral-sh/setup-uv@v5
 with:
    version: 0.5.x
    python-version: 3.9
    github-token: ***
    enable-cache: auto
    cache-dependency-glob: **/uv.lock
  **/requirements*.txt
  
    prune-cache: true
    ignore-nothing-to-cache: false
  env:
    FORCE_COLOR: 1
    pythonLocation: /opt/hostedtoolcache/Python/3.9.20/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.9.20/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.20/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.20/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.20/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.20/x64/lib
Downloading uv from "https://github.com/astral-sh/uv/releases/download/0.5.15/uv-x86_64-unknown-linux-gnu.tar.gz" ...
/usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/1ef64fdc-6ad9-4843-8111-ad76ac81f02b -f /home/runner/work/_temp/845516ea-631c-4ee4-8788-580661a28878
Added /opt/hostedtoolcache/uv/0.5.15/x86_64 to the path
Added /home/runner/.local/bin to the path
Set UV_PYTHON to 3.9
Activating python venv...
Set UV_CACHE_DIR to /home/runner/work/_temp/setup-uv-cache
Successfully installed uv version 0.5.15
Searching files using cache dependency glob: **/uv.lock,**/requirements*.txt
/home/runner/work/aiida-core/aiida-core/.docker/requirements.txt
/home/runner/work/aiida-core/aiida-core/utils/requirements.txt
/home/runner/work/aiida-core/aiida-core/uv.lock
Found 3 files to hash.
Trying to restore uv cache from GitHub Actions cache with key: setup-uv-1-x86_64-unknown-linux-gnu-3.9-26a4bf15a8c248eb29df58f5f6bb814cbbe6de47014bb6cc38985b07095394d9
Cache Size: ~8 MB (8535039 B)
/usr/bin/tar -xf /home/runner/work/_temp/b111819d-5d7d-4852-93db-37c5973549af/cache.tzst -P -C /home/runner/work/aiida-core/aiida-core --use-compress-program unzstd
Cache restored successfully
uv cache restored from GitHub Actions cache with key: setup-uv-1-x86_64-unknown-linux-gnu-3.9-26a4bf15a8c248eb29df58f5f6bb814cbbe6de47014bb6cc38985b07095394d9

And this is the YAML:

    - name: Set Up Python    
      uses: actions/setup-python@v5    
      with:    
        python-version: ${{ inputs.python-version }}    
       
    - name: Set up uv                                                                                                                      
      uses: astral-sh/setup-uv@v5
      with:
        version: 0.5.x
        python-version: ${{ inputs.python-version }}

@xmatthias
Copy link
Author

seems like you're right - though logs don't actually show this

3.10 setup (no actions/setup-python@v5)

Run astral-sh/setup-uv@v5
  with:
    python-version: 3.10
    enable-cache: false
    version: latest
    github-token: ***
    cache-dependency-glob: **/uv.lock
  **/requirements*.txt
  
    prune-cache: true
    ignore-nothing-to-cache: false
Downloading uv from "https://github.com/astral-sh/uv/releases/download/0.5.15/uv-x86_64-unknown-linux-gnu.tar.gz" ...
/usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/716dd560-5309-4cea-814a-d[2](https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190189#step:4:2)b3187daa79 -f /home/runner/work/_temp/04d98144-afb2-4b72-9651-be[3](https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190189#step:4:3)2e3644a45
Added /opt/hostedtoolcache/uv/0.5.15/x86_6[4](https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190189#step:4:4) to the path
Added /home/runner/.local/bin to the path
Set UV_PYTHON to 3.10
Activating python venv...
Set UV_CACHE_DIR to /home/runner/work/_temp/setup-uv-cache
Successfully installed uv version 0.[5](https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190189#step:4:5).15

vs. 3.11 setup (this one is with the setup python action upfront).

Run astral-sh/setup-uv@v5
  with:
    python-version: 3.11
    enable-cache: false
    version: latest
    github-token: ***
    cache-dependency-glob: **/uv.lock
  **/requirements*.txt
  
    prune-cache: true
    ignore-nothing-to-cache: false
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.11.11/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.11.11/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.11/x64
    Python[2](https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190471#step:4:2)_ROOT_DIR: /opt/hostedtoolcache/Python/[3](https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190471#step:4:3).11.11/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.11/x6[4](https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190471#step:4:4)
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.11.11/x64/lib
Downloading uv from "https://github.com/astral-sh/uv/releases/download/0.[5](https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190471#step:4:5).15/uv-x86_64-unknown-linux-gnu.tar.gz" ...
/usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/ce37[6](https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190471#step:4:6)b53-fdf0-41a1-93a6-b5c8442c99de -f /home/runner/work/_temp/9[7](https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190471#step:4:7)cd8965-25fd-466b-8471-a6f2a3a7b0d1
Added /opt/hostedtoolcache/uv/0.5.15/x[8](https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190471#step:4:8)6_64 to the path
Added /home/runner/.local/bin to the path
Set UV_PYTHON to 3.[11](https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190471#step:4:11)
Activating python venv...
Set UV_CACHE_DIR to /home/runner/work/_temp/setup-uv-cache
Successfully installed uv version 0.5.15

In the subsequent step - i then run uv python list - which shows for 3.11 (from actions)

Python 3.11.11 (main, Dec  4 2024, 12:58:35) [GCC 13.2.0]
cpython-3.14.0a3+freethreaded-linux-x86_64-gnu    <download available>
cpython-3.14.0a3-linux-x86_64-gnu                 <download available>
cpython-3.13.1+freethreaded-linux-x86_64-gnu      <download available>
cpython-3.13.1-linux-x86_64-gnu                   <download available>
cpython-3.12.8-linux-x86_64-gnu                   <download available>
cpython-3.12.3-linux-x86_64-gnu                   /usr/bin/python3.12
cpython-3.12.3-linux-x86_64-gnu                   /usr/bin/python3 -> python3.12
cpython-3.12.3-linux-x86_64-gnu                   /usr/bin/python -> python3
cpython-3.12.3-linux-x86_64-gnu                   /bin/python3.12
cpython-3.12.3-linux-x86_64-gnu                   /bin/python3 -> python3.12
cpython-3.12.3-linux-x86_64-gnu                   /bin/python -> python3
cpython-3.11.11-linux-x86_64-gnu                  /opt/hostedtoolcache/Python/3.11.11/x64/python -> ./bin/python3.11
cpython-3.11.11-linux-x86_64-gnu                  /opt/hostedtoolcache/Python/3.11.11/x64/bin/python3.11
cpython-3.11.11-linux-x86_64-gnu                  /opt/hostedtoolcache/Python/3.11.11/x64/bin/python3 -> python3.11
cpython-3.11.11-linux-x86_64-gnu                  /opt/hostedtoolcache/Python/3.11.11/x64/bin/python -> python3.11
cpython-3.11.11-linux-x86_64-gnu                  <download available>
cpython-3.10.16-linux-x86_64-gnu                  <download available>
cpython-3.9.21-linux-x86_64-gnu                   <download available>
cpython-3.8.20-linux-x86_64-gnu                   <download available>
cpython-3.7.9-linux-x86_64-gnu                    <download available>
pypy-3.10.14-linux-x86_64-gnu                     <download available>
pypy-3.9.19-linux-x86_64-gnu                      <download available>
pypy-3.8.16-linux-x86_64-gnu                      <download available>
pypy-3.7.13-linux-x86_64-gnu                      <download available>

vs. 3.10 (installed via UV):

cpython-3.14.0a3+freethreaded-linux-x86_64-gnu    <download available>
cpython-3.14.0a3-linux-x86_64-gnu                 <download available>
cpython-3.13.1+freethreaded-linux-x86_64-gnu      <download available>
cpython-3.13.1-linux-x86_64-gnu                   <download available>
cpython-3.12.8-linux-x86_64-gnu                   <download available>
cpython-3.12.3-linux-x86_64-gnu                   /usr/bin/python3.12
cpython-3.12.3-linux-x86_64-gnu                   /usr/bin/python3 -> python3.12
cpython-3.12.3-linux-x86_64-gnu                   /usr/bin/python -> python3
cpython-3.12.3-linux-x86_64-gnu                   /bin/python3.12
cpython-3.12.3-linux-x86_64-gnu                   /bin/python3 -> python3.12
cpython-3.12.3-linux-x86_64-gnu                   /bin/python -> python3
cpython-3.11.11-linux-x86_64-gnu                  <download available>
cpython-3.10.16-linux-x86_64-gnu                  /home/runner/.local/share/uv/python/cpython-3.10.16-linux-x86_64-gnu/bin/python3.10
cpython-3.9.21-linux-x86_64-gnu                   <download available>
cpython-3.8.20-linux-x86_64-gnu                   <download available>
cpython-3.7.9-linux-x86_64-gnu                    <download available>
pypy-3.10.14-linux-x86_64-gnu                     <download available>
pypy-3.9.[19](https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190189#step:5:20)-linux-x86_64-gnu                      <download available>
pypy-3.8.16-linux-x86_64-gnu                      <download available>
pypy-3.7.13-linux-x86_64-gnu                      <download available>

The only place you can tell you're now using a "uv" python is by having the path point to /home/runner/.local/share/uv/python/....

This is good - as it means behavior is already the way i expect it
However - log messages could/should be improved to highlight that it installs a python version - vs. it's using one that's already present.

source run:
3.10: https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190189
3.11: https://github.com/xmatthias/actionstest/actions/runs/12656191391/job/35268190471

@danielhollas
Copy link

danielhollas commented Jan 7, 2025 •
edited
Loading

I agree it would be great to have better logs.
For sure if this action ends up downloading PBS python it should say so.
And ideally even if it doesn't it should note that it found an existing interpreter.

Perhaps you could modify the title of this issue and the original message accordingly?

@eifinger eifinger added the documentation Improvements or additions to documentation label Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@eifinger @xmatthias @danielhollas