platform specific tasks and system-requirements same solve group

[geoHeil]

How can I run platform-specific tasks?

pixi run create_environment

✨ Pixi task (create_environment in research-cpu): pixi install -e mlx
  × The platform you are running on should at least have the virtual package __cuda on version 12, build_string: 0

[environments]
mlx = {features = ["mlx"], solve-group = "default"}
cuda = {features = ["cuda"], solve-group = "default"}


[feature.cuda]
platforms = ["linux-64"]
system-requirements = { cuda = "12" }


[project]
name = "foo"
platforms = ["linux-64",  "osx-arm64"]
channels = ["conda-forge", "pytorch"]


[feature.mlx.tasks]
create_environment = {cmd = "pixi install -e mlx"}

However, a pixi install -e research-cpu will install fine. This works in this particular case, but fails in the generic task runner case due to not being able to execute the dependencies outside of the pixi enabled shell.

When looking at https://github.com/prefix-dev/pixi/blob/main/examples/multi-machine/pixi.toml and modifying it slightly and choosing separate solve groups - it works

[environments]
mlx = {features = ["mlx"], solve-group = "default"}
cuda = {features = ["cuda"], solve-group = "cuda"}


[feature.cuda]
platforms = ["linux-64"]
system-requirements = { cuda = "12" }


[project]
name = "foo"
platforms = ["linux-64",  "osx-arm64"]
channels = ["conda-forge", "pytorch"]


[feature.mlx.tasks]
create_environment = {cmd = "pixi install -e mlx"}

But if I want to ensure that (except for the cuda stuff) the same dependencies are used is there any chance I can get this to work in the same groups?

[geoHeil]

My real project is a bit different:

pixi install
  × Unable to solve pypi dependencies for the research-cuda environment because no compatible python interpreter can be installed for the current platform
     ╭─[174:13]
 173 │ [feature.research-cuda]
 174 │ platforms = ["linux-64"]
     ·             ──────┬─────
     ·                   ╰── feature 'research-cuda' does not support 'osx-arm64'
platforms = ["linux-64", "win-64", "osx-arm64"]
     ·             ─────────────────┬─────────────────
     ·                              ╰── even though the projects does include support for 'osx-arm64'

I indeed do not WANT to support osx for this feature. How can I disable osx support for good for that feature? Why is the platforms directive limiting to linux (see below) not working as expected?

This is why

[feature.research-cuda]
platforms = ["linux-64"]

are set.

• When choosing the same solve group --> install works, but task runs fail
• When choosing different solve groups installation fails with the aforementioned error -

[ruben-arts]

You can't use solve-groups and pypi-dependencies when the different environments can't be installed on the machine your solving on. This is because we need to install the conda part of the environment before we can solve the pypi-dependencies but with the platforms you specify that the environment can not be installed on that platform. So cannot solve it.

[geoHeil]

What do you think the suggested way forward is? Use different solve groups. No pinning of system dependencies (seems to work so far)

Separate pixi file for each env? (I really would want to avoid it.

Also I didn`t specify any pip pipy-dependencies sections - where are these coming from (in the more complex example yes perhaps) but in the one above not - and still, I was running into the same error

[ruben-arts]

[project]
name = "foo"
# Default feature/environment work on both systems
platforms = ["linux-64",  "osx-arm64"]
channels = ["conda-forge"]

[feature.linux]
# Overwrite platforms
platforms = ["linux-64"]
# Overwrite channels
channels = ["conda-forge", "pytorch"]
# Linux feature requires cuda, so it can not be installed on osx.
system-requirements.cuda = "12"

[feature.macos]
# Overwrite platforms
platforms = ["osx-arm64"]

[environments]
linux = ["linux"]
macos = ["macos"]

This works as the two environments don't need to be installed for solving and don't interfere with solve groups.

• Adding the same solve-group to both environments will fail as you then try to solve the complete dependency set for both.
• Adding pypi-dependencies will not work as you can't install all environments on all systems. Which doesn't allow you to solve.

This is the issue related to solving your issues:

• Unsolvable pypi dependencies if platform excluded by feature #1051
• pixi install installs all environments in multi env setting when pypi-dependency is used #1046

And It would be good to check out: #1295

[geoHeil]

But why is my:

[feature.cuda]
platforms = ["linux-64"]

override not working?

it looks to me exactly the same as yours:

[feature.linux]
# Overwrite platforms
platforms = ["linux-64"]

[geoHeil]

Thanks for the links - at least for me it looks like setting:

[tool.pixi.feature.research-cuda.system-requirements]
cuda = "12"

can be a suitable workaround