22 June 2016

ActiveMQ: Message ordering

Bambitroll @ 15:25

ActiveMQ being a messaging system based on queues (aka FIFOs), one would take for granted that if there is only one producer and one consumer for a given queue (and they are both single threaded), the order of the messages is preserved.
Well, not always!

Let's say I have the following configuration for my ActiveMQ client running Camel:

<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">

    <bean id="amqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
        <property name="brokerURL" value="${env.activemq.broker.url}"/>
        <property name="userName" value="${env.activemq.broker.username}"/>
        <property name="password" value="${env.activemq.broker.password}"/>
    </bean>

    <bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
        <property name="maxConnections" value="12"/>
        <property name="maximumActiveSessionPerConnection" value="200"/>
        <property name="connectionFactory" ref="amqConnectionFactory"/>
    </bean>

    <bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
        <property name="connectionFactory" ref="pooledConnectionFactory"/>
        <property name="transacted" value="true"/>
        <property name="cacheLevelName" value="CACHE_NONE"/>
    </bean>

    <bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
        <property name="configuration" ref="jmsConfig"/>
    </bean>

</blueprint>

With this configuration, even with a prefetch of 1 and only one consumer, you risk having messages being consumed out of order even if they were produced in the right order.
The culprit is CACHE_NONE, which you want to use if you are using XA transactions.
But in normal circumstances, with a local transaction manager or with the one built-in with the JmsConfiguration bean, it is recommended to use CACHE_CONSUMER not only to improve performance but also to ensure proper message ordering.

Side note regarding prefetch=1:
Even though one could expect having only one message sent to the consumer until it gets ack-ed (which is when you are done processing it when you have transacted=true), it is still possible to have a second message assigned to that consumer in the dispatch queue, which in effect get blocked until the first message is fully processed (which can be a problem for slow consumers).
The solution (if this is really a problem) would be to use prefetch=0 for that given consumer, but this is costly since the consumer is now polling the broker!

More info here


If message ordering is a big requirement for you, you might want to look at the Camel resequencer.


Update 20160624: And now there is a Jira for this!
[ENTMQ-1783] Combination of CACHE_NONE and Transacted Affects Message Ordering - JBoss Issue Tracker


04 May 2016

ActiveMQ Command-line utility

Bambitroll @ 15:44

I have been looking for a nice little ActiveMQ CLI utility in order to push/consume messages to/from ActiveMQ via the standard OpenWire protocol for quite a while now and I finally found it!

Thanks for the glorious developer behind this github project :)

ActiveMQ Command-line utility
http://nordlander.co/activemq-command-line-utility-a/

Make sure you have a look at all the parameters on the main github page of the project (https://github.com/fmtn/a) and use the jar with dependencies if you don't have maven installed.

Here is a little bash script you can use so make things simpler:

#!/bin/sh
#
# blog: http://nordlander.co/activemq-command-line-utility-a/
# github: https://github.com/fmtn/a
#
# Parameters examples:
# -U admin -P admin -p "toto" q_fake
# -U admin -P admin -c 9 -p "toto" q_fake
# -U admin -P admin -o /tmp/msgs/fakemsg -c 9 -g q_fake
# -U admin -P admin -p @/tmp/msgs/fakemsg-no4 q_fake
#
java -jar ~/bin/a-1.3.0-jar-with-dependencies.jar "$@"

Now if you have saved the script above under the file called amq_client.sh, you can use it like this:
$ ./amq_client.sh -U admin -P admin -p "toto" q_fake


20 January 2016

Using asciidocs to write documentation/PDF in a maven project

Bambitroll @ 10:51

Writing documentation like release notes in Word is a pain, especially when you have it within  your project which is under version control.
Diffs between commits are impossible and you risk having 2 people editing the document at the same time and overwriting each others changes (no automatic merge possibility like you have with the code).

The idea is then to use asciidocs to write the documentation and then generate PDF documents from there with some kind of template in order to have a final document which is easy to pass around.

So comes asciidoctor to the rescue, which has a maven plugin allowing us to do everything via maven!

You need this in your pom.xml file:

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <asciidoctor.maven.plugin.version>1.5.3</asciidoctor.maven.plugin.version>
        <asciidoctorj.pdf.version>1.5.0-alpha.11</asciidoctorj.pdf.version>
        <asciidoctorj.version>1.5.4</asciidoctorj.version>
        <jruby.version>1.7.21</jruby.version>
    </properties>

    <build>
        <defaultGoal>process-resources</defaultGoal>
        <plugins>
            <plugin>
                <groupId>org.asciidoctor</groupId>
                <artifactId>asciidoctor-maven-plugin</artifactId>
                <version>${asciidoctor.maven.plugin.version}</version>
                <dependencies>
                    <dependency>
                        <groupId>org.asciidoctor</groupId>
                        <artifactId>asciidoctorj-pdf</artifactId>
                        <version>${asciidoctorj.pdf.version}</version>
                    </dependency>
                    <!-- Comment this section to use the default jruby artifact provided by the plugin -->
                    <dependency>
                        <groupId>org.jruby</groupId>
                        <artifactId>jruby-complete</artifactId>
                        <version>${jruby.version}</version>
                    </dependency>
                    <!-- Comment this section to use the default AsciidoctorJ artifact provided by the plugin -->
                    <dependency>
                        <groupId>org.asciidoctor</groupId>
                        <artifactId>asciidoctorj</artifactId>
                        <version>${asciidoctorj.version}</version>
                    </dependency>
                </dependencies>
                <configuration>
                    <sourceDirectory>src/main/resources/ReleaseNotes</sourceDirectory>
                    <!-- Attributes common to all output formats -->
                    <attributes>
                        <sourcedir>${project.build.sourceDirectory}</sourcedir>
                    </attributes>
                </configuration>
                <executions>
                    <execution>
                        <id>generate-pdf-doc</id>
                        <phase>generate-resources</phase>
                        <goals>
                            <goal>process-asciidoc</goal>
                        </goals>
                        <configuration>
                            <backend>pdf</backend>
                            <!-- Since 1.5.0-alpha.9 PDF back-end can use 'rouge' as well as 'coderay' source highlighting -->
                            <sourceHighlighter>rouge</sourceHighlighter>
                            <attributes>
                                <pdf-stylesdir>${project.basedir}/src/main/resources/ReleaseNotes/themes</pdf-stylesdir>
                                <pdf-style>my-theme</pdf-style>
                                <icons>font</icons>
                                <tabsize>4</tabsize>
                                <pagenums/>
                                <toc/>
                                <idprefix/>
                                <idseparator>-</idseparator>
                            </attributes>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>


If you have an image in src/main/resources/ReleaseNotes/images/image1.png and a theme in src/main/resources/ReleaseNotes/themes/my-theme.yml like this

title_page:
  align: right
  background_image: logo.jpg

page:
  layout: portrait
  margin: [2cm, 1cm, 2cm, 1cm] # top, right, bottom, left
  size: A4
base:
  font_color: #333333
  line_height_length: 17
  line_height: $base_line_height_length / $base_font_size
vertical_rhythm: $base_line_height_length
heading:
  font_color: #333333
  font_size: 17
  font_style: bold
  line_height: 1.2
  margin_bottom: $vertical_rhythm
link:
  font_color: #0e83a4
outline_list:
  indent: $base_font_size * 1.5
header:
  height: 1.5cm
  line_height: 1
  recto_content:
    center: '{document-title}'
  verso_content:
    center: '{document-title}'
footer:
  height: 1.5cm
  line_height: 1
  recto_content:
#    right: '{section-title} | *{page-number}/{page-count}*'
    right: '*{page-number}/{page-count}*'
  verso_content:
#    left: '*{page-number}/{page-count}* | {section-title}'
    left: '*{page-number}/{page-count}*'
image:
  align: center
caption:
  align: center
  font_color: #FF0000
  font_size: 10

You can then write your release notes in asciidocs format in src/main/resources/ReleaseNotes/Release_Notes.adoc, which could like like this:

= Release Notes 1.0.0
My Project
v1.0.0

// variables
:MyProj: My fancy project name


// Settings:
:sectnums:
:toc:
:toclevels: 3
:imagesdir[:imagesdir: images]




[.thumb]
image::image1.png[scaledwidth=20%]

[abstract]


== Introduction

This document contains release notes and change log for releases and patches to {MyProj}. +
 +
The document describes changes, known issues, deliverables and versions of sub components for a single version of the application. +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
,===
*Author*, Bugs Bunny
*Status*, Final
*Approved by*, Daffy Duck
*Version*, 1.0.0
*Created*,  January 14th 2016
*Last changed*, January 19th 2016
,===


<<<

== Revision History

*Only for this release* +
 +
[format="csv", options="header"]
|===
Date, Version, Description, Author
*15.01.2016*,0.1,First draft, Bugs Bunny
*15.01.2016*,0.2,Updates, Daffy Duck
|===

<<<

== Version Description


=== Deliverables (Sub components/Features)

bla bla bla...

<<<


=== Dependencies

+++<span style="color: #FF0000">Just some red text</span>+++ +

=== Comments

Comments comments +
 +
 +
*+++<span style="color: red;">WARNING: red and bold</span>+++* +



Then when you compile your project via maven, you will get a PDF document in target/generated-docs!
It should like like this.


In this example, we use asciidoctor to generate the PDF file so the syntax can be a bit different than plain asciidoc.


References:
AsciiDoc Syntax Quick Reference
http://asciidoctor.org/docs/asciidoc-syntax-quick-reference/

Chapter 10. Text Formatting
http://www.methods.co.nz/asciidoc/chunked/ch10.html

asciidoctor-maven-examples/asciidoctor-pdf-example at master · asciidoctor/asciidoctor-maven-examples
https://github.com/asciidoctor/asciidoctor-maven-examples/tree/master/asciidoctor-pdf-example

asciidoctor/asciidoctor-pdfhttps://github.com/asciidoctor/asciidoctor-pdf

asciidoctor-pdf/theming-guide.adoc
https://github.com/asciidoctor/asciidoctor-pdf/blob/master/docs/theming-guide.adoc

AsciiDoc Writer’s Guide
http://asciidoctor.org/docs/asciidoc-writers-guide/

Asciidoctor User Manual
http://asciidoctor.org/docs/user-manual/

Asciidoctor Documentation
http://asciidoctor.org/docs/

Differences between Asciidoctor and AsciiDoc
http://asciidoctor.org/docs/asciidoc-asciidoctor-diffs/

AsciiDoc Writer’s Guide
http://asciidoctor.org/docs/asciidoc-writers-guide/