Ways to reduce idle CPU usage #9988
-
Describe the bugHello,
I debugged rabbitmq, and there is literally nothing ongoing, no messages at all. This is bothering me, so I tried to fiddle with the configurations of rabbitmq, but it does not seem that any of the many settings has an impact on the idle CPU usage of rabbit. Is this expected? Anything I can do about it? I tried various container releases including latest and also the distributed debian package (without docker) all with the same result that the idle CPU usage is relatively high. Reproduction steps
Expected behaviorI would expect substantially less CPU resources used when being idle. Additional contextRunning rabbitmq on a debian server. |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 6 replies
-
|
@daniel86 CPU usage of 0.2% or 0.3% (and this is very likely 0.3% of just one core) is hardly excessive. A node that does not accept, route or delivery any messages still emits and collects metrics periodically. You do not mention what specific settings you have tried but there is a dedicated doc section Those running on CentOS Stream 9 and not expecting a lot of connections can also take a look at this blog post. |
Beta Was this translation helpful? Give feedback.
-
|
@michaelklishin I did not say the usage is excessive. but relatively high which is visible in the grafana screenshot. I gladly provide more details. I did follow the page you mentioned before, without success (I invested like two days going through all the material I could find). Here is the docker compose configuration I use: The content of so I disabled all the metrics plugin without any apparent change in idle CPU usage to my surprise. I played a lot with all the settings in the configuration file, the one currently mounted in the container has these entries: nothing of above appears to have influence on idle CPU usage. The log of rabbit suggests it adopts my configuration: |
Beta Was this translation helpful? Give feedback.
-
|
Increasing statistics emission interval (thus reducing the frequency) and busy waiting in the runtime are the recommendations, pinning a node to just one scheduler is not. RabbitMQ was not designed for single core environments. CLI tool can be pinned to one scheduler but they are not long running, so I don't know what difference it might make. |
Beta Was this translation helpful? Give feedback.
-
|
ok, thanks for clarification! I'll switch to "+S 4:4" then. So the settings I spotted might have an impact? Maybe then I'll give it another shot to adjust them a bit. Do you have other ideas of settings that I could have a look at? |
Beta Was this translation helpful? Give feedback.
-
|
@daniel86 on Kubernetes, setting the number of schedulers is rarely necessary because Kubernetes has its own tools. Only with some kernel and cgroups versions the Erlang runtime detects more cores than the K8S limit. I honestly fail to see the end goal of this exercise (with idle CPU usage being below 0.3% of one core). The two primary settings that you can configure are listed in the docs and explained below. In any practical environment where there are connections, queues, streams, and so on, idle nodes will spend at least some CPU cycles collecting metrics of those connections, etc, but also external metrics such as the amount of free disk space on the partition where the node stores its data. You cannot disable all metric collection, and I would recommend having solid monitoring (with Prometheus) in place over having an extra < 1% of the CPU core to spare. |
Beta Was this translation helpful? Give feedback.
-
|
There are certain things in the above settings that are unnecessary or counterproductive:
Using Monitoring tools scrape data from the Prometheus endpoint every so often. Some tools monitor with The quest to bringing down a RabbitMQ node's CPU usage down to zero will likely have consequences for monitoring, so beyond the two key recommendations above, you have to ask yourself if this is really worth the gains. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for this elaboration. To my defence :) ... this is not thought to be a production configuration, I tried everything to see if I can impact idle CPU time, and this is how the config looked when I decided to stop trying. But whatever I did it remains constant at the value shown in grafana. My setup is for me personally, so scaling is not needed because there is only one user and a couple of private task boards. I just want to keep the consumption of my server at a minimum. I know prometheus scraping can cause high CPU load. I experienced it with cadvisor. But it was tamable and is now running on my server with relatively low constant load after I disabled quiete a few metrics I was not using. I actually already accepted it to be like it is, but thought maybe someone can confirm that or has some additional tips what I could do. After your tips I will give it another shot and report back in case I could achieve any success this time |
Beta Was this translation helpful? Give feedback.
-
|
If you want to dig deeper - use a profiler: https://blog.rabbitmq.com/posts/2022/05/flame-graphs/ or some continues profiling tool like parca.dev. |
Beta Was this translation helpful? Give feedback.
-
|
As well as the metrics RabbitMQ nodes provide about where the schedulers spend their time. |
Beta Was this translation helpful? Give feedback.
@daniel86 CPU usage of 0.2% or 0.3% (and this is very likely 0.3% of just one core) is hardly excessive.
A node that does not accept, route or delivery any messages still emits and collects metrics periodically.
You do not mention what specific settings you have tried but there is a dedicated doc section
on reducing CPU usage on mostly idle systems.
Those running on CentOS Stream 9 and not expecting a lot of connections can also take a look at this blog post.