Replies: 4 comments 4 replies
-
These reproduction steps is not something our team can work with. "Publish notifications to classic queues and measure performance" is not specific at all. We have no idea whether you use CQv2 or not, what your message sizes are, or anything at all. "Performance" is not a constant, it is a function of the workload, the configuration and the environment used. We do not guess in this community. See these blog posts to learn about how our team Finally, the latest version of RabbitMQ is |
Beta Was this translation helpful? Give feedback.
-
You haven't provided any details on what your "simple AMQP client" does and measures but those numbers are about two orders of magnitude lower than what a modern QQ or a non-mirrored CQv2 can achieve with reasonable hardware, small messages (up to 1 kiB) and the most basic PerfTest flags. A Simplest Possible CQv2 BenchmarkFor example, a PerfTest run against a local PerfTest consumers use automatic acknowledgements by default.
Yields between 130K and 160K with a single classic queue on an M1 Mac from 2021-2022.
A Simplest Possible QQ BenchmarkBenchmarking quorum queues with a single replica is not really fair because the whole point Still, to demonstrate that 400-700 "reads" a second is absolutely nothing for a QQ leader replica,
which yields about 100K small messages a second on the same machine as above:
Mirrored Classic Queues are GoneIf mirrored classic queues are used, we will not investigate any claims of throughput regressions: CQ mirroring was removed entirely for 4.0 after about three years of deprecation. |
Beta Was this translation helpful? Give feedback.
-
In order to get a QQ to a ≈ 400-600 messages per second range I have to increase message size to 1-4 MiB:
This is expected: data volumes transferred are 2000 greater than what I used in the first test, For 4.0, there is ongoing work specifically for CQv2 but also QQs to reduce memory footprint with such large messages. There aren't that many ways to reduce disk I/O and in open source RabbitMQ, there aren't any ways to reduce network data flow volumes, so throughput is unlikely to significantly change even with those improvements. That's as much as I can say given that we do not know anything about the environment, a homegrown "simple" benchmarking tool is used instead of |
Beta Was this translation helpful? Give feedback.
-
Between 3.13.1 and 3.13.2 classic queues have only changed for large messages (> 4K) so what you are seeing likely isn't directly related. There wasn't a lot of changes between the two versions, so what you may want to do is a |
Beta Was this translation helpful? Give feedback.
-
Describe the bug
During the process of upgrading to using quorum queues, we've read that their performance should be substantially better than classic going forward.
As a result I created a simple AMQP client which published messages to a topic and then another one which creates 100 queues of either classic or quorum and then reads for 30 seconds printing the number per second we've managed to read.
On RabbitMQ version 3.13.1 classic queues are about 150% than quorum. With reads per second of about 660~ and 400~ respectively.
However on RabbitMQ version 3.13.2 and above, the performance of classic queues seems to go out the window and now averages about 100~ reads per second, whereas quorum remains unchanged.
Obviously my test is bottlenecked by my machines performance, however i'd expect similar results to be similar given the same environment being used.
Reproduction steps
Deploy RabbitMQ in docker using the command:
docker run -d --hostname my-rabbit --name some-rabbit -p 5672:5672 -p 15672:15672 --ulimit nofile=65536:65536 -e RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS="+P 1048576" rabbitmq:3.13.1-management
Publish notifications to classic queues and measure performance
Remove and redeploy rabbitmq on 3.13.2 or above
Publish notifications to classic queues and remeasure performance
Expected behavior
Firstly, i wouldnt expect the performance of classic queues to degrade massively between minors versions, or even at all.
Is quorum indeed faster for this use-case? This link suggests that 'classic queues offer a third of the performance'
https://www.rabbitmq.com/docs/migrate-mcq-to-qq
Additional context
NoAck = true
Durable = true
Beta Was this translation helpful? Give feedback.
All reactions