Docspell in Docker uses too much ressources while importing

[andbez]

I setup Docspell as documented. It starting and useable. As soon as I import documents (dsc) the whole setup uses always all ressources (CPU) as available (and more. Docker runs in a Proxmox-VM. The VM is then not really usable). How can I limit the ressources Docspell uses?
I am really interested in running Docspell and want to migrate from paperless-ngx (> 6000 Documents). But I can't get it running smoothly since 2 weeks :(

Thank you.

[vakilando]

Hi,
I'm running docspell on unraid and had similar problems.
Runnig Joex with the following parameter/option worked for me:
-e JAVA_OPTS="-Xmx2500m"

[andbez]

Thanks!

1. Unfortunately I have no idea how to set the JAVA_OPTS inside the Container (via docker compose). I did not find any Environment-Setting in the Docspell-Documentation. Any hint?
2. XMX sets the maximum RAM-Size. Does this also affect the CPU-Usage?

[vakilando]

Yes, you are right. I had the problem that my Server was running out of RAM... sorry.
But i think it could help perhaps.
Do you know which container is using so much CPU or wich process?

[andbez]

Yes, it's the solr-container. As soon as the solr-container is started, CPU-Usage goes max.

[eikek]

high cpu usage could be caused by too little memory, in this case the garbage collector would run very often and occupies the cpu significantly. i don't know if this applies to your case, if course

[andbez]

Thanks for your Answer, eikek. VM is showing 8 GB available (free) memory.

[eikek]

Thanks for your Answer, eikek. VM is showing 8 GB available (free) memory.

That sounds good, you might then want to assign more heap to solr and see if that makes a difference.

[andbez]

Perfect. Seems to work. Thank you. Following set in Docker-Compose-File for SOLR:
environment:
- SOLR_HEAP="2g"

Now importing > 6000 Docs.

[andbez]

Ok. Did not help. Solr is always crashing even with lot of memory. Now I added more memory for joex (as mentioned in Docspell-Documentation) and reduced Solr again and now it works. See what will happen ... stay tuned.

[eikek]

Do you see any logs at solr (or the thing that crashes)?

[andbez]

still importing > 6000 Docs :D When finished, I will setup the old config and give you more information. What I can say is, that Solr crashes (and docker restarts the container always because of unless-stopped), when joex needs the whole CPU-Ressources.