docs: Added LLM example

[ytjhai]

I noticed there had been a few issues like #206 and #261 for demoing Pixi with an ML stack. This example shows it's possible with the current feature set to run Llama-index with llama.cpp on a local machine with a Mistral LLM, also running locally with GPU support on an ARM M1 Mac. Hopefully this helps someone else get started with LLM inference.

[tdejager]

Hi @ytjhai, thanks a lot for the PR.

Would you be open to adding

# linux-64 to the platforms
platforms = ["osx-arm64", "linux-64"]

# setting this avoids building scikit_learn from source (on arm)
[system-requirements]
macos = "12.0"

[tdejager]

Hi @ytjhai, thanks a lot for the PR.

Would you be open to adding

# linux-64 to the platforms
platforms = ["osx-arm64", "linux-64"]

# setting this avoids building scikit_learn from source (on arm)
[system-requirements]
macos = "12.0"

Even better would be having separate features and environments. Where the system-requirement for osx-arm64 and osx are different. This would allow older non-arm osx versions.

[ytjhai]

I don't currently have access to a linux machine, so can't 100% verify if this lock file would work on linux-64. But I could put a test environment together when I have some time? It would have an AMD GPU, which is a bit off the beaten path (similar to the idea behind this build).

Regarding older OSX environments, GPU strength was very patchy and my personal use case was running some LLMs locally since M1 has good GPU support in even base models (like the Mac Mini). Even llama.cpp doesn't support that architecture on Macs AFAIK.

[Hofer-Julian]

I don't currently have access to a linux machine, so can't 100% verify if this lock file would work on linux-64. But I could put a test environment together when I have some time? It would have an AMD GPU, which is a bit off the beaten path (similar to the idea behind this build).

I tested it with an AMD GPU (I assume it is using the CPU though) and it seems to work fine :)

[ytjhai]

Great! I made the above requested changes, adding linux-64 as a supported environment

[ruben-arts]

Awesome thanks!

[tdejager]

Last, tiny small thing, could you maybe add a pixi run start task that runs the python command :)?

[tdejager]

Last, tiny small thing, could you maybe add a pixi run start task that runs the python command :)?

Don't worry I've added it :)

[Hofer-Julian]

Thanks!