Add Falcon LLM

[humaidq-tii]

Description of changes

• Add ollama (AI backend) and alpaca (AI frontend).
  • Autoload falcon2 model whenever there is an internet connection
  • Added load-falcon command to load the model, which shows user-friendly notification.
  • This command should be integrated with UI later, such as in system bar or widgets.
• Increased system partitions size until a permanent fix is available. fixed by Fix postboot command for ZFS installation. #813

This PR carries some overlays and systemd options that should be removed by #769 #809

Checklist for things done

• Summary of the proposed changes in the PR description
• More detailed description in the commit message(s)
• Commits are squashed into relevant entities - avoid a lot of minimal dev time commits in the PR
• Contribution guidelines followed
• Ghaf documentation updated with the commit - https://tiiuae.github.io/ghaf/
• PR linked to architecture documentation and requirement(s) (ticket id)
• Test procedure described (or includes tests). Select one or more:
  • Tested on Lenovo X1 x86_64
  • Tested on Jetson Orin NX or AGX aarch64
  • Tested on Polarfire riscv64
• Author has run make-checks and it passes
• All automatic Github Action checks pass - see actions
• Author has added reviewers and removed PR draft status
• Change requires full re-installation: YES as per Fix postboot command for ZFS installation. #813
• Change can be updated with nixos-rebuild ... switch: NO full reinstallation needed as per Fix postboot command for ZFS installation. #813

Instructions for Testing

• List all targets that this applies to: x1 carbon only
• Is this a new feature
  • List the test steps to verify:
    • Fresh installation of Ghaf (rebuild is not enough)
    • Connect to Wi-Fi, open terminal in gui-vm, run load-falcon
      • On a 50Mb/s internet connection, it may take a bit over 20 minutes
      • You can see the progress by running systemctl status ollama-model-loader. If you see messages with "blob", it is downloading. If deactivated successfully, then the model should be available.
    • Open Falcon AI application, you should see "Falcon2" on the top, you should be able to ask a basic question and it would reply.
    • Please don't test the model's accuracy or behaviour as it is out of scope 😃
    • Do not untoggle the "Use Remote Connection" option in the Alpaca settings.
• If it is an improvement how does it impact existing functionality?

[leivos-unikie]

Is falcon2 model supposed to be available for use also without internet connection once the model has been loaded? While testing I somehow I managed to loose the model from the upper model menu and then had to manually start loading the last falcon model again (although df -h on gui-vm showed that there was already 6G used on storagevm by previously loaded model.

[humaidq-tii]

Is falcon2 model supposed to be available for use also without internet connection once the model has been loaded?

Yes, even after reboot. Also the download should be able to resume even between reboots, so even on very slow internet connections it would eventually be downloaded.

While testing I somehow I managed to loose the model from the upper model menu and then had to manually start loading the last falcon model again (although df -h on gui-vm showed that there was already 6G used on storagevm by previously loaded model.

There is a problem where it sometimes uses Alpaca's own ollama instance, which downloads to .local instead. I need to make sure it doesn't use that... As long as you don't untick the "Use Remote Connection" in settings, it shouldn't do this.

[leivos-unikie]

As a summary of our discussions: it would be better not to download the model automatically, instead have some manual trigger on UI to launch the download.

[leivos-unikie]

Otherwise I didn't notice any regression

• Menu icons work
• ci-test-automation passes

If falcon model download happens during performance tests it decreases ghaf-host fileio results and memory speed results on net-vm and gui-vm.