Tech and Tips

If you are working with OSGi, you are surely using the Maven Bundle Plugin to create your MANIFEST file and jar bundles.

After a while, it can happen that your maven build it getting slower than it should be.
In my case, the "Writing OBR metadata" part of the maven build was clearly to blame.
And this is because it is parsing and updating a file called repository.xml, which gets really big with time.

Solution:
mv ~/.m2/repository/repository.xml ~/.m2/repository/repository.xml.old

ActiveMQ being a messaging system based on queues (aka FIFOs), one would take for granted that if there is only one producer and one consumer for a given queue (and they are both single threaded), the order of the messages is preserved.
Well, not always!

Let's say I have the following configuration for my ActiveMQ client running Camel:

<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">

    <bean id="amqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
        <property name="brokerURL" value="${env.activemq.broker.url}"/>
        <property name="userName" value="${env.activemq.broker.username}"/>
        <property name="password" value="${env.activemq.broker.password}"/>
    </bean>

    <bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
        <property name="maxConnections" value="12"/>
        <property name="maximumActiveSessionPerConnection" value="200"/>
        <property name="connectionFactory" ref="amqConnectionFactory"/>
    </bean>

    <bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
        <property name="connectionFactory" ref="pooledConnectionFactory"/>
        <property name="transacted" value="true"/>
        <property name="cacheLevelName" value="CACHE_NONE"/>
    </bean>

    <bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
        <property name="configuration" ref="jmsConfig"/>
    </bean>

</blueprint>

With this configuration, even with a prefetch of 1 and only one consumer, you risk having messages being consumed out of order even if they were produced in the right order.
The culprit is CACHE_NONE, which you want to use if you are using XA transactions.
But in normal circumstances, with a local transaction manager or with the one built-in with the JmsConfiguration bean, it is recommended to use CACHE_CONSUMER not only to improve performance but also to ensure proper message ordering.

Side note regarding prefetch=1:
Even though one could expect having only one message sent to the consumer until it gets ack-ed (which is when you are done processing it when you have transacted=true), it is still possible to have a second message assigned to that consumer in the dispatch queue, which in effect get blocked until the first message is fully processed (which can be a problem for slow consumers).
The solution (if this is really a problem) would be to use prefetch=0 for that given consumer, but this is costly since the consumer is now polling the broker!

More info here


If message ordering is a big requirement for you, you might want to look at the Camel resequencer.


Update 20160624: And now there is a Jira for this!
[ENTMQ-1783] Combination of CACHE_NONE and Transacted Affects Message Ordering - JBoss Issue Tracker

Here is a lesson learnt regarding using ActiveMQ with several consumers for one queue and transactions enabled.

What we wanted to achieve was to have the following setup for each queue:

• several consumers to spread the load and avoid that one message gets stuck if a route gets into trouble or takes a long time to complete
• each consumer should get only one message at a time so no message gets stuck in the local client buffer
• the camel route should be transacted so if something goes wrong the message is not lost but gets delivered to another consumer

We are using camel and the camel-activemq component.
So defining several consumers is rather easy.
We used prefetch=1 to make sure that only one message at a time is delivered to a route .
And we used a transaction manager for our JmsConfiguration bean.
So my config looked like something like that

<bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
        <property name="connectionFactory" ref="pooledConnectionFactory" />
        <property name="transacted" value="true" /> 
        <property name="transactionManager" ref="jmsTransactionManager" />
        <property name="cacheLevelName" value="CACHE_CONSUMER" />
</bean>

So far nothing special.

What I then noticed was that sometimes a consumer could get 2 messages assigned, which puzzled me since I specified prefetch=1!
One being processed and waiting for an ACK to finish the transaction, and the other one just waiting to be taken and not being taken by another consumer even if one is available.
And this was bad because sometimes my route took several minutes to completed, effectively preventing the 2nd message to get through right away.

The culprit was CACHE_CONSUMER. This pre-assigns the next message to the consumer to speed things up.

So I changed this to CACHE_NONE and now everything behaves as expected!

The problem:
When using properties files in Fuse/ServiceMix (which you deploy in $FUSE_HOME/etc), you can define a camel endpoint like this

my.file.endpoint=file:/var/tmp/in?move=/var/tmp/sent/${date:now:yyyyMMdd}-${file:name}

This will work fine if you define it as a properties in your camel context XML file, like this:

<cm:property-placeholder persistent-id="my.properties">
    <cm:default-properties>
        <cm:property name="my.file.endpoint" value="file:/var/tmp/in?move=/var/tmp/sent/${date:now:yyyyMMdd}-${file:name}"/>
    </cm:default-properties>
</cm:property-placeholder>

But if you use the same definition in your properties file, because you want a more environment specific value for each of your servers, you are out of luck.
All you will achieve is having your files moved to a folder called "-" under sent.
This is because everything within a ${} is interpreted as a variable and will be resolved (unsuccessfully) before being passed along.


The solution:
Use the simple language!

my.file.endpoint=file:/var/tmp/in?move=/var/tmp/sent/$simple{date:now:yyyyMMdd}-$simple{file:name}

According to the specs for Fuse, if one uses blueprint, it is possible to create several files called *.xml under OSGI-INF/blueprint and they will all be used in the bundle.
This is true but I just found out that the alphabetical order of the file names can have its importance.
Indeed, I have a separate file for my properties (where my cm:property-placeholder is located) and I found out that the properties I wanna use as parameters to bean or outside the camel context are not taken into account if the file is read after the other blueprint files.
So name you properties file aaa_properties and not just properties, especially if your blueprint file is called blueprint.xml :)