-
Notifications
You must be signed in to change notification settings - Fork 58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Falcon LLM #806
Add Falcon LLM #806
Conversation
Is falcon2 model supposed to be available for use also without internet connection once the model has been loaded? While testing I somehow I managed to loose the model from the upper model menu and then had to manually start loading the last falcon model again (although df -h on gui-vm showed that there was already 6G used on storagevm by previously loaded model. |
Yes, even after reboot. Also the download should be able to resume even between reboots, so even on very slow internet connections it would eventually be downloaded.
|
As a summary of our discussions: it would be better not to download the model automatically, instead have some manual trigger on UI to launch the download. |
Otherwise I didn't notice any regression
If falcon model download happens during performance tests it decreases ghaf-host fileio results and memory speed results on net-vm and gui-vm. |
Signed-off-by: Humaid Alqasimi <[email protected]>
Description of changes
Autoload falcon2 model whenever there is an internet connectionIncreased system partitions size until a permanent fix is available.fixed by Fix postboot command for ZFS installation. #813This PR carries some overlays and systemd options that should be removed by
#769#809Checklist for things done
x86_64
aarch64
riscv64
make-checks
and it passesnixos-rebuild ... switch
: NO full reinstallation needed as per Fix postboot command for ZFS installation. #813Instructions for Testing
load-falcon
systemctl status ollama-model-loader
. If you see messages with "blob", it is downloading. If deactivated successfully, then the model should be available.