In this article I am going to present few rules I use when selecting prefetch count value for RabbitMQ. Those rules are based on the experience I gained when working with RabbitMQ. You should treat them as a guidance and starting point, remembering that each application, network and queue topology is different. Let’s start with quick explanation what prefetch count is.

Prefetchcount

RabbitMQ allows consumers to specify the size of the limit of unacknowledged messages on a queue basis. This value is used to specify how many messages is send to the consumer and cached by RabbitMQ client library. Once the buffer is full the RabbitMQ will wait with delivering new messages to that consumer until it sends ACKs / NACKs. The main purpose for pre fetching messages is to optimise message delivering. The ideal situation is where there is a message delivered and waiting for the processor to be consumed.
Of course, caching messages on the consumer side has repercussions. All pre-fetched messages are removed from the queue and become invisible to other processors. Hence setting prefetch count to a high value will skew message distribution between different consumers.

Consumer types

Below are rules for selecting prefetch count depending on the consumer type.

Fast, single consumer

If there is only one consumer processing messages quickly you definitely want all messages to be pre-fetched. The optimum value will probably be something higher than 20, but setting it to anything above 50 will rarely delivery more value. Remember that pre-fetched messages are cached on the client side consuming its resources. High value will also restrict your ability to peek messages.

Multiple fast consumers

When you have multiple consumers which are processing messages fast you need to find value which keeps messages waiting for consumers as well as allows for balanced message delivery between consumers. Low value will penalise consumers as they need to wait for messages. High value will damage load balancing. I usually use anything between 20 and 30.

Slow consumers

For slow consumers message delivery time is negligible. My advice is to set prefetch count to 1 and let RabbitMQ load balance message between consumers.

Application crash and buffered messages

Let’s see what happens to buffered messages when the application crashes. Because RabbitMQ automatically puts all not ACKed messages back to the queue, if the application crashes then all cached messages will simply go back to the top of the queue.

Setting prefetch count

To set ‘prefetch’ count in RabbitMQ you have to execute basic.qos command[1].

Setting prefetch count in EasyNetQ

You can set prefetch count value in EasyNetQ by including it in the bus’ connection string. Default value used by EasyNetQ is 50.
Below example sets prefetch count to 30:

var bus = RabbitHutch.CreateBus("host=localhost;prefetchcount=30");

Resources

You may find more information about prefetch count in following articles:

For example of usage check Work Queues tutorial. ↩︎