Ollama doc update, with good news

[PieBru]

Is your feature request related to a problem? Please describe.
The doc refers to Ollama with the mixtral model.

Describe the solution you'd like
Update the doc.

Describe alternatives you've considered
I tested llama3.2 (FAST) and llama3.1(SMART) and I confirm they works for general needs.
Also, llama3.2 works reasonably well as SMART model, so the whole system can run pretty well locally on a 4GB VRAM GPU.

Additional context
Add any other context or screenshots about the feature request here.

[assafelovic]

Hey @PieBru how about helping us with contributing this updates here: https://docs.gptr.dev/docs/gpt-researcher/llms/llms

[assafelovic]

Or here: https://github.com/assafelovic/gpt-researcher/blob/master/docs/docs/gpt-researcher/llms/running-with-ollama.md

[assafelovic]

Also if you can help this issue: #904

[Davery92]

how are you running it with Ollama? with docker or what?

[PieBru]

@Davery92

how are you running it with Ollama? with docker or what?

I run Ollama in a LXC with GPU pass through and LLM beefy storage on a separate NVMe. It's all on a old notebook installed with Proxmox VE 8.2.
Thus Ollama appears as a standalone server with GPU and lot of storage.

I mostly followed this guide https://fileflows.com/docs/guides/linux/proxmox-lxc-nvidia
P.

[PieBru]

For who may be interested, here is my GPTR config that works fully local with SearXNG and Ollama.

export DOC_PATH=./my-docs

export RETRIEVER=searx
export SEARX_URL="http://10.4.0.101:32768"

export LLM_PROVIDER=ollama
export OLLAMA_BASE_URL="http://10.4.0.100:11434"
export FAST_LLM=ollama:llama3.2
export SMART_LLM=ollama:llama3.2
export TEMPERATURE="0.1"
export EMBEDDING_PROVIDER=ollama
export OLLAMA_EMBEDDING_MODEL=nomic-embed-text

[Davery92]

Interesting, I have a searxng server running, copied your config settings but with my own ip and port, and I still get 'no content for query'