Tech and Tips

Here is a lesson learnt regarding using ActiveMQ with several consumers for one queue and transactions enabled.

What we wanted to achieve was to have the following setup for each queue:

• several consumers to spread the load and avoid that one message gets stuck if a route gets into trouble or takes a long time to complete
• each consumer should get only one message at a time so no message gets stuck in the local client buffer
• the camel route should be transacted so if something goes wrong the message is not lost but gets delivered to another consumer

We are using camel and the camel-activemq component.
So defining several consumers is rather easy.
We used prefetch=1 to make sure that only one message at a time is delivered to a route .
And we used a transaction manager for our JmsConfiguration bean.
So my config looked like something like that

<bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
        <property name="connectionFactory" ref="pooledConnectionFactory" />
        <property name="transacted" value="true" /> 
        <property name="transactionManager" ref="jmsTransactionManager" />
        <property name="cacheLevelName" value="CACHE_CONSUMER" />
</bean>

So far nothing special.

What I then noticed was that sometimes a consumer could get 2 messages assigned, which puzzled me since I specified prefetch=1!
One being processed and waiting for an ACK to finish the transaction, and the other one just waiting to be taken and not being taken by another consumer even if one is available.
And this was bad because sometimes my route took several minutes to completed, effectively preventing the 2nd message to get through right away.

The culprit was CACHE_CONSUMER. This pre-assigns the next message to the consumer to speed things up.

So I changed this to CACHE_NONE and now everything behaves as expected!