04 January 2018

Maven Bundle Plugin slowing down the maven build

Bambitroll @ 10:04
If you are working with OSGi, you are surely using the Maven Bundle Plugin to create your MANIFEST file and jar bundles.

After a while, it can happen that your maven build it getting slower than it should be.
In my case, the "Writing OBR metadata" part of the maven build was clearly to blame.
And this is because it is parsing and updating a file called repository.xml, which gets really big with time.

Solution:
mv ~/.m2/repository/repository.xml ~/.m2/repository/repository.xml.old

22 June 2016

ActiveMQ: Message ordering

Bambitroll @ 15:25

ActiveMQ being a messaging system based on queues (aka FIFOs), one would take for granted that if there is only one producer and one consumer for a given queue (and they are both single threaded), the order of the messages is preserved.
Well, not always!

Let's say I have the following configuration for my ActiveMQ client running Camel:

<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">

    <bean id="amqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
        <property name="brokerURL" value="${env.activemq.broker.url}"/>
        <property name="userName" value="${env.activemq.broker.username}"/>
        <property name="password" value="${env.activemq.broker.password}"/>
    </bean>

    <bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
        <property name="maxConnections" value="12"/>
        <property name="maximumActiveSessionPerConnection" value="200"/>
        <property name="connectionFactory" ref="amqConnectionFactory"/>
    </bean>

    <bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
        <property name="connectionFactory" ref="pooledConnectionFactory"/>
        <property name="transacted" value="true"/>
        <property name="cacheLevelName" value="CACHE_NONE"/>
    </bean>

    <bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
        <property name="configuration" ref="jmsConfig"/>
    </bean>

</blueprint>

With this configuration, even with a prefetch of 1 and only one consumer, you risk having messages being consumed out of order even if they were produced in the right order.
The culprit is CACHE_NONE, which you want to use if you are using XA transactions.
But in normal circumstances, with a local transaction manager or with the one built-in with the JmsConfiguration bean, it is recommended to use CACHE_CONSUMER not only to improve performance but also to ensure proper message ordering.

Side note regarding prefetch=1:
Even though one could expect having only one message sent to the consumer until it gets ack-ed (which is when you are done processing it when you have transacted=true), it is still possible to have a second message assigned to that consumer in the dispatch queue, which in effect get blocked until the first message is fully processed (which can be a problem for slow consumers).
The solution (if this is really a problem) would be to use prefetch=0 for that given consumer, but this is costly since the consumer is now polling the broker!

More info here


If message ordering is a big requirement for you, you might want to look at the Camel resequencer.


Update 20160624: And now there is a Jira for this!
[ENTMQ-1783] Combination of CACHE_NONE and Transacted Affects Message Ordering - JBoss Issue Tracker


04 May 2016

ActiveMQ Command-line utility

Bambitroll @ 15:44

I have been looking for a nice little ActiveMQ CLI utility in order to push/consume messages to/from ActiveMQ via the standard OpenWire protocol for quite a while now and I finally found it!

Thanks for the glorious developer behind this github project :)

ActiveMQ Command-line utility
http://nordlander.co/activemq-command-line-utility-a/

Make sure you have a look at all the parameters on the main github page of the project (https://github.com/fmtn/a) and use the jar with dependencies if you don't have maven installed.

Here is a little bash script you can use so make things simpler:

#!/bin/sh
#
# blog: http://nordlander.co/activemq-command-line-utility-a/
# github: https://github.com/fmtn/a
#
# Parameters examples:
# -U admin -P admin -p "toto" q_fake
# -U admin -P admin -c 9 -p "toto" q_fake
# -U admin -P admin -o /tmp/msgs/fakemsg -c 9 -g q_fake
# -U admin -P admin -p @/tmp/msgs/fakemsg-no4 q_fake
#
java -jar ~/bin/a-1.3.0-jar-with-dependencies.jar "$@"

Now if you have saved the script above under the file called amq_client.sh, you can use it like this:
$ ./amq_client.sh -U admin -P admin -p "toto" q_fake


20 January 2016

Using asciidocs to write documentation/PDF in a maven project

Bambitroll @ 10:51

Writing documentation like release notes in Word is a pain, especially when you have it within  your project which is under version control.
Diffs between commits are impossible and you risk having 2 people editing the document at the same time and overwriting each others changes (no automatic merge possibility like you have with the code).

The idea is then to use asciidocs to write the documentation and then generate PDF documents from there with some kind of template in order to have a final document which is easy to pass around.

So comes asciidoctor to the rescue, which has a maven plugin allowing us to do everything via maven!

You need this in your pom.xml file:

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <asciidoctor.maven.plugin.version>1.5.3</asciidoctor.maven.plugin.version>
        <asciidoctorj.pdf.version>1.5.0-alpha.11</asciidoctorj.pdf.version>
        <asciidoctorj.version>1.5.4</asciidoctorj.version>
        <jruby.version>1.7.21</jruby.version>
    </properties>

    <build>
        <defaultGoal>process-resources</defaultGoal>
        <plugins>
            <plugin>
                <groupId>org.asciidoctor</groupId>
                <artifactId>asciidoctor-maven-plugin</artifactId>
                <version>${asciidoctor.maven.plugin.version}</version>
                <dependencies>
                    <dependency>
                        <groupId>org.asciidoctor</groupId>
                        <artifactId>asciidoctorj-pdf</artifactId>
                        <version>${asciidoctorj.pdf.version}</version>
                    </dependency>
                    <!-- Comment this section to use the default jruby artifact provided by the plugin -->
                    <dependency>
                        <groupId>org.jruby</groupId>
                        <artifactId>jruby-complete</artifactId>
                        <version>${jruby.version}</version>
                    </dependency>
                    <!-- Comment this section to use the default AsciidoctorJ artifact provided by the plugin -->
                    <dependency>
                        <groupId>org.asciidoctor</groupId>
                        <artifactId>asciidoctorj</artifactId>
                        <version>${asciidoctorj.version}</version>
                    </dependency>
                </dependencies>
                <configuration>
                    <sourceDirectory>src/main/resources/ReleaseNotes</sourceDirectory>
                    <!-- Attributes common to all output formats -->
                    <attributes>
                        <sourcedir>${project.build.sourceDirectory}</sourcedir>
                    </attributes>
                </configuration>
                <executions>
                    <execution>
                        <id>generate-pdf-doc</id>
                        <phase>generate-resources</phase>
                        <goals>
                            <goal>process-asciidoc</goal>
                        </goals>
                        <configuration>
                            <backend>pdf</backend>
                            <!-- Since 1.5.0-alpha.9 PDF back-end can use 'rouge' as well as 'coderay' source highlighting -->
                            <sourceHighlighter>rouge</sourceHighlighter>
                            <attributes>
                                <pdf-stylesdir>${project.basedir}/src/main/resources/ReleaseNotes/themes</pdf-stylesdir>
                                <pdf-style>my-theme</pdf-style>
                                <icons>font</icons>
                                <tabsize>4</tabsize>
                                <pagenums/>
                                <toc/>
                                <idprefix/>
                                <idseparator>-</idseparator>
                            </attributes>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>


If you have an image in src/main/resources/ReleaseNotes/images/image1.png and a theme in src/main/resources/ReleaseNotes/themes/my-theme.yml like this

title_page:
  align: right
  background_image: logo.jpg

page:
  layout: portrait
  margin: [2cm, 1cm, 2cm, 1cm] # top, right, bottom, left
  size: A4
base:
  font_color: #333333
  line_height_length: 17
  line_height: $base_line_height_length / $base_font_size
vertical_rhythm: $base_line_height_length
heading:
  font_color: #333333
  font_size: 17
  font_style: bold
  line_height: 1.2
  margin_bottom: $vertical_rhythm
link:
  font_color: #0e83a4
outline_list:
  indent: $base_font_size * 1.5
header:
  height: 1.5cm
  line_height: 1
  recto_content:
    center: '{document-title}'
  verso_content:
    center: '{document-title}'
footer:
  height: 1.5cm
  line_height: 1
  recto_content:
#    right: '{section-title} | *{page-number}/{page-count}*'
    right: '*{page-number}/{page-count}*'
  verso_content:
#    left: '*{page-number}/{page-count}* | {section-title}'
    left: '*{page-number}/{page-count}*'
image:
  align: center
caption:
  align: center
  font_color: #FF0000
  font_size: 10

You can then write your release notes in asciidocs format in src/main/resources/ReleaseNotes/Release_Notes.adoc, which could like like this:

= Release Notes 1.0.0
My Project
v1.0.0

// variables
:MyProj: My fancy project name


// Settings:
:sectnums:
:toc:
:toclevels: 3
:imagesdir[:imagesdir: images]




[.thumb]
image::image1.png[scaledwidth=20%]

[abstract]


== Introduction

This document contains release notes and change log for releases and patches to {MyProj}. +
 +
The document describes changes, known issues, deliverables and versions of sub components for a single version of the application. +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 +
,===
*Author*, Bugs Bunny
*Status*, Final
*Approved by*, Daffy Duck
*Version*, 1.0.0
*Created*,  January 14th 2016
*Last changed*, January 19th 2016
,===


<<<

== Revision History

*Only for this release* +
 +
[format="csv", options="header"]
|===
Date, Version, Description, Author
*15.01.2016*,0.1,First draft, Bugs Bunny
*15.01.2016*,0.2,Updates, Daffy Duck
|===

<<<

== Version Description


=== Deliverables (Sub components/Features)

bla bla bla...

<<<


=== Dependencies

+++<span style="color: #FF0000">Just some red text</span>+++ +

=== Comments

Comments comments +
 +
 +
*+++<span style="color: red;">WARNING: red and bold</span>+++* +



Then when you compile your project via maven, you will get a PDF document in target/generated-docs!
It should like like this.


In this example, we use asciidoctor to generate the PDF file so the syntax can be a bit different than plain asciidoc.


References:
AsciiDoc Syntax Quick Reference
http://asciidoctor.org/docs/asciidoc-syntax-quick-reference/

Chapter 10. Text Formatting
http://www.methods.co.nz/asciidoc/chunked/ch10.html

asciidoctor-maven-examples/asciidoctor-pdf-example at master · asciidoctor/asciidoctor-maven-examples
https://github.com/asciidoctor/asciidoctor-maven-examples/tree/master/asciidoctor-pdf-example

asciidoctor/asciidoctor-pdfhttps://github.com/asciidoctor/asciidoctor-pdf

asciidoctor-pdf/theming-guide.adoc
https://github.com/asciidoctor/asciidoctor-pdf/blob/master/docs/theming-guide.adoc

AsciiDoc Writer’s Guide
http://asciidoctor.org/docs/asciidoc-writers-guide/

Asciidoctor User Manual
http://asciidoctor.org/docs/user-manual/

Asciidoctor Documentation
http://asciidoctor.org/docs/

Differences between Asciidoctor and AsciiDoc
http://asciidoctor.org/docs/asciidoc-asciidoctor-diffs/

AsciiDoc Writer’s Guide
http://asciidoctor.org/docs/asciidoc-writers-guide/


09 March 2015

ActiveMQ, transactions and several consumers

Bambitroll @ 16:57
Here is a lesson learnt regarding using ActiveMQ with several consumers for one queue and transactions enabled.

What we wanted to achieve was to have the following setup for each queue:
  • several consumers to spread the load and avoid that one message gets stuck if a route gets into trouble or takes a long time to complete
  • each consumer should get only one message at a time so no message gets stuck in the local client buffer
  • the camel route should be transacted so if something goes wrong the message is not lost but gets delivered to another consumer
We are using camel and the camel-activemq component.
So defining several consumers is rather easy.
We used prefetch=1 to make sure that only one message at a time is delivered to a route .
And we used a transaction manager for our JmsConfiguration bean.
So my config looked like something like that

<bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
        <property name="connectionFactory" ref="pooledConnectionFactory" />
        <property name="transacted" value="true" /> 
        <property name="transactionManager" ref="jmsTransactionManager" />
        <property name="cacheLevelName" value="CACHE_CONSUMER" />
</bean>

So far nothing special.

What I then noticed was that sometimes a consumer could get 2 messages assigned, which puzzled me since I specified prefetch=1!
One being processed and waiting for an ACK to finish the transaction, and the other one just waiting to be taken and not being taken by another consumer even if one is available.
And this was bad because sometimes my route took several minutes to completed, effectively preventing the 2nd message to get through right away.

The culprit was CACHE_CONSUMER. This pre-assigns the next message to the consumer to speed things up.

So I changed this to CACHE_NONE and now everything behaves as expected!

27 February 2015

Run Hawtio locally and connect to Jolokia/ActiveMQ 5.9.1 and above remotely

Bambitroll @ 15:33

Get the Hawtio app from http://hawt.io
Start it like this:

java -jar -Dhawtio.offline=true -Dhawtio.proxyWhitelist=* hawtio-app-2.0.2.jar

Then go to http://localhost:8080/hawtio (actually the tab should open by itself in your default browser after you ran the command above)




Choose Connect->Remote, then Add Connection and use port 8161 and path /api/jolokia




And voila :)


Remember to adjust 2 settings by clicking on the little user icon in the top right corner (in Preferences):
- the ActiveMQ username/password in order to send/consume messages
- the max collection size for Jolokia in order to get all the objects back from JMX

From ActiveMQ 5.9.1 and up, the jolokia agent is installed by default so you should not need to do anything special with your ActiveMQ installation.

29 January 2015

Installing Hawtio in ActiveMQ 5.9.1

Bambitroll @ 11:07

Hawtio is not included by default in ActiveMQ 5.9.1 as it was in 5.9.0.

But by following the instructions from these 2 pages, it is fairly simple to add it manually:
http://sensatic.net/activemq/activemq-and-hawtio.html
http://activemq.2283324.n4.nabble.com/Hawto-log-in-td4673552.html

The main info from the second page is this missing info from the first one:
-Djava.security.auth.login.config=$ACTIVEMQ_CONF/login.config

Also make sure that you update the usernames/password in the following files in $ACTIVEMQ_HOME/conf:
- credentials.properties
- users.properties
- groups.properties
- login.config

Make sure you also add this to $ACTIVEMQ_HOME/bin/activemq, somewhere around line 160:
ACTIVEMQ_OPTS="$ACTIVEMQ_OPTS -Dhawtio.realm=activemq -Dhawtio.role=admins -Dhawtio.rolePrincipalClasses=org.apache.activemq.jaas.GroupPrincipal -Djava.security.auth.login.config=$ACTIVEMQ_CONF/login.config"
ACTIVEMQ_OPTS="$ACTIVEMQ_OPTS -Dhawtio.offline=true -Dhawtio.config.cloneOnStartup=false -Dhawtio.config.pullOnStartup=false"


The second set of sysetm properties is to prevent hawtio to try to fetch data from github which is does by default. If you are behind a firewall, this can cause hawtio to hang for over 2 minutes.

2 more things:
- if you have some special proxy settings, you might want to use 127.0.1.1 instead of 127.0.0.1 to reach your local hawtio
- the hawtio tree structure on the left side gets all screwed up each time a refresh occurs. I am not sure if this is a bug in hawtio but it is very annoying.

More links:
http://hawt.io/getstarted/
http://hawt.io/configuration/index.html#Configuration_Properties


Good luck!

26 June 2014

Eclipse Luna bug in Preferences for XML Editor

Bambitroll @ 09:59

If you are using the first release version of Eclipse 4.4 Luna, there is a bug in Windows->Preferences->XML->XML Files->Editor.

You can not change the settings for indentation size when using spaces. It is impossible to write a new number in the field and the -/+ buttons are disabled. The default value is 1! :(

The work around I found is to edit the following file in your workspace:
.metadata/.plugins/org.eclipse.core.runtime/.settings/org.eclipse.wst.xml.core.prefs

And add the following line (if you want 4 char for your indentation):
indentationSize=4


17 June 2014

ActiveMQ: memory management for queues

Bambitroll @ 11:05
With JMS persistent messages, all the message are persisted to the storage device by default, but also kept in memory. When a queue can potentially contain many messages, it is then necessary to set a max memory allowed, like this:
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="/var/activemq/data">
    <destinationPolicy>
        <policyMap>
            <policyEntries>
                <policyEntry queue=">" producerFlowControl="false" memoryLimit="100mb" queuePrefetch=”1” maxPageSize=”10” lazyDispatch=”true” />
            </policyEntries>
        </policyMap>
    </destinationPolicy>
</broker>

02 May 2014

Fuse and expressions in properties files

Bambitroll @ 11:23
The problem:
When using properties files in Fuse/ServiceMix (which you deploy in $FUSE_HOME/etc), you can define a camel endpoint like this
my.file.endpoint=file:/var/tmp/in?move=/var/tmp/sent/${date:now:yyyyMMdd}-${file:name}

This will work fine if you define it as a properties in your camel context XML file, like this:
<cm:property-placeholder persistent-id="my.properties">
    <cm:default-properties>
        <cm:property name="my.file.endpoint" value="file:/var/tmp/in?move=/var/tmp/sent/${date:now:yyyyMMdd}-${file:name}"/>
    </cm:default-properties>
</cm:property-placeholder>

But if you use the same definition in your properties file, because you want a more environment specific value for each of your servers, you are out of luck.
All you will achieve is having your files moved to a folder called "-" under sent.
This is because everything within a ${} is interpreted as a variable and will be resolved (unsuccessfully) before being passed along.


The solution:
Use the simple language!

my.file.endpoint=file:/var/tmp/in?move=/var/tmp/sent/$simple{date:now:yyyyMMdd}-$simple{file:name}

25 April 2014

Create fake TCP server to test a port is up and running

Bambitroll @ 16:41
There might be a way easier way to do this but here is how I solved it.

What I needed was to have a mock TCP server running on a given port on a given server to check that the firewall between my machine and the remote machine was properly open for a given port.
So I got a little  program running on the remote machine listening on the given port.
I also found a little client program, but using telnet works as well.

Under linux, just run "gcc server.c -o server_test" to compile the program and then run it with "server_test "
Also run "gcc client.c -o client_test" to compile and then run the client with "client_test "

Here is server.c:
/* A simple server in the internet domain using TCP
   The port number is passed as an argument */
#include 
#include 
#include 
#include 
#include  
#include 
#include 

void error(const char *msg)
{
    perror(msg);
    exit(1);
}

int main(int argc, char *argv[])
{
     int sockfd, newsockfd, portno;
     socklen_t clilen;
     char buffer[256];
     struct sockaddr_in serv_addr, cli_addr;
     int n;
     if (argc < 2) {
         fprintf(stderr,"ERROR, no port provided\n");
         exit(1);
     }
     sockfd = socket(AF_INET, SOCK_STREAM, 0);
     if (sockfd < 0) 
        error("ERROR opening socket");
     bzero((char *) &serv_addr, sizeof(serv_addr));
     portno = atoi(argv[1]);
     serv_addr.sin_family = AF_INET;
     serv_addr.sin_addr.s_addr = INADDR_ANY;
     serv_addr.sin_port = htons(portno);
     if (bind(sockfd, (struct sockaddr *) &serv_addr,
              sizeof(serv_addr)) < 0) 
              error("ERROR on binding");
     listen(sockfd,5);
     clilen = sizeof(cli_addr);
     newsockfd = accept(sockfd, 
                 (struct sockaddr *) &cli_addr, 
                 &clilen);
     if (newsockfd < 0) 
          error("ERROR on accept");
     bzero(buffer,256);
     n = read(newsockfd,buffer,255);
     if (n < 0) error("ERROR reading from socket");
     printf("Here is the message: %s\n",buffer);
     n = write(newsockfd,"I got your message",18);
     if (n < 0) error("ERROR writing to socket");
     close(newsockfd);
     close(sockfd);
     return 0; 
}

Here is client.c:
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include  

void error(const char *msg)
{
    perror(msg);
    exit(0);
}

int main(int argc, char *argv[])
{
    int sockfd, portno, n;
    struct sockaddr_in serv_addr;
    struct hostent *server;

    char buffer[256];
    if (argc < 3) {
       fprintf(stderr,"usage %s hostname port\n", argv[0]);
       exit(0);
    }
    portno = atoi(argv[2]);
    sockfd = socket(AF_INET, SOCK_STREAM, 0);
    if (sockfd < 0) 
        error("ERROR opening socket");
    server = gethostbyname(argv[1]);
    if (server == NULL) {
        fprintf(stderr,"ERROR, no such host\n");
        exit(0);
    }
    bzero((char *) &serv_addr, sizeof(serv_addr));
    serv_addr.sin_family = AF_INET;
    bcopy((char *)server->h_addr, 
         (char *)&serv_addr.sin_addr.s_addr,
         server->h_length);
    serv_addr.sin_port = htons(portno);
    if (connect(sockfd,(struct sockaddr *) &serv_addr,sizeof(serv_addr)) < 0) 
        error("ERROR connecting");
    printf("Please enter the message: ");
    bzero(buffer,256);
    fgets(buffer,255,stdin);
    n = write(sockfd,buffer,strlen(buffer));
    if (n < 0) 
         error("ERROR writing to socket");
    bzero(buffer,256);
    n = read(sockfd,buffer,255);
    if (n < 0) 
         error("ERROR reading from socket");
    printf("%s\n",buffer);
    close(sockfd);
    return 0;
}



Sources: Linux Howtos: C/C++ -> Sockets Tutorial

09 April 2014

Adding/Removing queue to/from ActiveMQ via command line

Bambitroll @ 10:09

There is a way to add/remove queues on ActiveMQ from the command line.
It uses the REST interface of Jolokia which comes with Hawtio on ActiveMQ 5.9.0.

The first 2 commands contain all you need to create and then to remove the queue called q_titi, using the user admin/admin:
curl -u admin:admin -d "{\"type\":\"exec\",\"mbean\":\"org.apache.activemq:type=Broker,brokerName=localhost\",\"operation\":\"addQueue(java.lang.String)\",\"arguments\":[\"q_titi\"]}" http://localhost:8161/hawtio/jolokia/

curl -u admin:admin -d "{\"type\":\"exec\",\"mbean\":\"org.apache.activemq:type=Broker,brokerName=localhost\",\"operation\":\"removeQueue(java.lang.String)\",\"arguments\":[\"q_titi\"]}" http://localhost:8161/hawtio/jolokia/


The next 2 command do the same for the queue q_OMG but use the attached files for the necessary JSON request (so no more need to escape the double quotes and way more readable!):

curl -u admin:admin -d "@/tmp/jmr/createQ_json.txt" http://localhost:8161/hawtio/jolokia/
curl -u admin:admin -d "@/tmp/jmr/removeQ_json.txt" http://localhost:8161/hawtio/jolokia/

createQ_json.txt looks like this:
{
 "type":"exec",
 "mbean":"org.apache.activemq:type=Broker,brokerName=localhost",
 "operation":"addQueue(java.lang.String)",
 "arguments":["q_OMG"]
}


removeQ_json.txt looks like this:
{
 "type":"exec",
 "mbean":"org.apache.activemq:type=Broker,brokerName=localhost",
 "operation":"removeQueue(java.lang.String)",
 "arguments":["q_OMG"]
}


More info here:
Monitoring ActiveMQ via HTTP http://www.jakubkorab.net/2013/11/monitoring-activemq-via-http.html

03 February 2014

Properties files, blueprint and Fuse

Bambitroll @ 16:38
According to the specs for Fuse, if one uses blueprint, it is possible to create several files called *.xml under OSGI-INF/blueprint and they will all be used in the bundle.
This is true but I just found out that the alphabetical order of the file names can have its importance.
Indeed, I have a separate file for my properties (where my cm:property-placeholder is located) and I found out that the properties I wanna use as parameters to bean or outside the camel context are not taken into account if the file is read after the other blueprint files.
So name you properties file aaa_properties and not just properties, especially if your blueprint file is called blueprint.xml :)

18 September 2013

Camel cxfEndpoint not only for localhost

Bambitroll @ 16:51
Here is how to define a camel cxfEndpoint, as described in this example page:

<cxf:cxfEndpoint id="soapMessageEndpoint"
                 serviceClass="org.apache.camel.example.cxf.provider.GreeterProvider"
                 address="http://localhost:{{port}}/GreeterContext/SOAPMessageService"
                 wsdlURL="wsdl/hello_world.wsdl"
                 endpointName="s:SoapOverHttpRouter"
                 serviceName="s:SOAPService"
                 xmlns:s="http://apache.org/hello_world_soap_http"/>

But this exposes your web service endpoint only to your localhost, aka you can access it only from the machine running FuseESB (if this is where your route is deployed).
In order to be able to access it from anywhere, use 0.0.0.0 instead of localhost in the address property.

18 April 2013

H2 console

Bambitroll @ 14:11
The H2 console application nicely lets you interact with any of your locally created H2 databases. It gives you access via your web browser so it also works for remote machines, as long as you can start the console application there and access the box via HTTP.
More info on the H2 website.

The you get something which looks like this:



For reference:
The name H2 stands for Hypersonic 2, however H2 does not share code with Hypersonic SQL or HSQLDB. H2 is built from scratch.
More info from wikipedia here.

13 June 2012

XML indenting/formatting

Bambitroll @ 14:56

Working with XML can be a pain, but it is even more so if the XML you get it not properly formatted or indented.

Editors support this more or less properly.

This simple command line will give you something nice:
$ xmllint --format original.xml > nice.xml

P.S: make sure you run this in order to have xmllint installed
apt-get install libxml2-utils

14 March 2012

Private repositories in Sonatype Nexus

Bambitroll @ 16:36
Sonatype Nexus is an artifact repository manager very much used when maven is your build tool for a java project. You can use it as a proxy for other repositories but also to host your own artifacts.

By default in Nexus, all repositories are reachable via the anonymous user.

Here is what to do to configure your Nexus server so that (at least) some of your repositories will require credentials to access them.

First you have to prevent the anonymous user to have access to all the repositories:
  1. Create a new Privilege that gives access to you public group (or individual repositories) NOTE: Assigning access to a group is equivalent to assigning these privileges to all of the repositories in the group.
    1. Login to nexus as an administrator.
    2. Click on Privileges in the left menu.
    3. Click Add.
    4. Use the following values:
      1. Name: M2 Public Repositories (Group)
      2. Description: Access to Public Repositories (Group)
      3. Repository: Public Repositories (Group)
      4. Repository Target: All (Maven2)
    5. Save.
  2. Repeat the previous step for all you public groups and/or respositories.
  3. Create a new Role and assign this new privilege to it.
    1. Click on Roles in the left menu.
    2. Click Add.
    3. User the following values:
      1. Role Id: repo-public-read
      2. Name: Repo: All Public Repositories (read)
      3. Description: Read only access to all public repositories.
      4. Session Timeout: 60
      5. Selected Roles / Privileges: M2 Public Repositories (Group) - (read)
      6. NOTE: Include all of the roles you created in the first step.
    4. Save.
  4. Remove the Grant Read all role from the Anonymous user and add the new role.
    1. Click on Users in the left menu.
    2. Click on the anonymous user.
    3. Remove the role 'Repo: All Repositories (read)'
    4. Add the role 'All Public Repositories (read)'
    5. Save.

Once you have done this, you need to create users to access all the non public repositories.
  1. You first create a privilege for your given repo (Security->Privileges then Add)
  2. Then a role (add the view and read part of the privilege created above)
  3. And finally a user and assign him the role just create
  4. Repeat this for as many times as you have repos. If you want to have several users for one repo, then just repeat 3.


The final step is to configure the client side with the proper settings in .m2/settings.xml and in the project pom.xml. like this:
settings.xml
<servers>
   <server>
       <id>myserver</id>
       <username>mrx</username>
       <password>pwd</password>
   </server>
<servers>
pom.xml
<repositories>
   <repository>
       <id>myserver</id>
       <name>myserver</name>
       <url>http://server:8081/nexus/content/repositories/your_repo</url>
   </repository>
</repositories>


Now your repository should be accessible with user and password only.


Some links:
Managing Security with Nexus
Depend on a password protected nexus repository
Can I make a repository private without disabling anonymous access?


02 August 2011

Hibernate/JPA Identity Generators

Bambitroll @ 16:05
While googling for some answer regarding JPA identity generators and how they behave according to which DB you use, I found this very nice blog post!

Very informative!

31 May 2011

SOAPUI and dynamic XML values

Bambitroll @ 17:44
I am currently using SOAPUI to load test an application via web services.
In order to do that, I need to be able to change the value of some XML elements on the fly.
Here is how you can do that:
- Create a Test Suite
- Create a Test Case (which will contains Test Steps)
- Your first test step will be a SOAP request called TestReq1
- In the Test Case Editor, you can create a Setup Script

Here is the Groovy script I use to set the value of one element dynamically (in this case the element attribute called Version):
def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
def holder = groovyUtils.getXmlHolder("TestReq1#Request")
holder.setNodeValue("//@Version","7.2")
holder.updateProperty()

At the moment the script is not so useful since it dynamically sets the element to the same value. But there are several strategies to use different values each time, from reading from a file or a DB to writing a little generator in Groovy.


Another thing which is useful with SOAPUI is to be able to create a mock-up response which is dynamically created according to the incoming data. Once you have created your mock-up service, here is how to fetch data from the request and paste it in the response. First the incoming XML and then the Groovy script changing the response (in this example both the request and the response have a Version attribute on an element and we simply return the value we get in):

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:abc="http://my.project.com/abc">
   <soapenv:Header/>
   <soapenv:Body>
      <abc:customerApp Version="${version}" />
   </soapenv:Body>
</soapenv:Envelope>

def holder = new com.eviware.soapui.support.XmlHolder(mockRequest.requestContent)
context.version = holder["//@Version"]

28 March 2011

Testing Databases with JUnit and Hibernate

Bambitroll @ 13:51
Here a 3 part blog post showing how to use JUnit and Hibernate (and Hypersonic) to test your database layer. Often using an ORM makes this superfluous but in some cases it can be an interesting to do.

Part 1
Part 2
Part 3

23 March 2011

Socket connection error when using remote JBoss AS

Bambitroll @ 10:05
I spent most if the afternoon on this problem yesterday and I thought I
could just share the solution since this might be a good tip for everyone...

I have Arquillian tests which used to run flawlessly and which stopped
working out of the blue! The problem was, according to the JUnit logs, a
socket connection error when using remote JBoss AS.

After a lot of searching and great help from Aslak Knutsen on
#jbosstesting, I found out that the problem was in /etc/hosts!


More info and the solution from JBoss community.

28 February 2011

Improving your laptop battery life

Bambitroll @ 08:14
-
According to this page, you should avoid charging your laptop if its battery is nearly full. Indeed, it destroys your battery capacity in the long run.

Running Ubuntu on a Thinkpad (mine is a T510), you can set up the thresholds for when to start and stop charging yourself.

Here is how:
# aptitude install tp-smapi-dkms
# modprobe tp_smapi
# echo 40 > /sys/devices/platform/smapi/BAT0/start_charge_thresh
# echo 70 > /sys/devices/platform/smapi/BAT0/stop_charge_thresh
# cat /sys/devices/platform/smapi/BAT0/*_charge_thresh

Then to have the settings loaded at each startup:
# apt-get install sysfsutils
And add the following in /etc/sysfs.conf
# For a LiIon battery in a Thinkpad
devices/platform/smapi/BAT0/start_charge_thresh = 50
devices/platform/smapi/BAT0/stop_charge_thresh =85

All the info is here.

Once you are done are reboot the machine, you will need to run the "modprobe tp_smapi" command to be able to access /sys/devices/platform/smapi
-

23 February 2011

Arquillian testing - JBoss 5.1.0 - EJB3

Bambitroll @ 11:17
I started using Arquillian to test my business logic code directly in my running JBoss 5.1 instance, and after some fighting I got it working pretty nicely!

So here is what I had to do:
1. Add a profile for running the Arquillian Integration tests in Maven

<profiles>
   <profile>
      <id>it</id>
      <activation>
        <activeByDefault>false</activeByDefault>
      </activation>
      <build>
        <defaultGoal>verify</defaultGoal>
        <plugins>
          <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.6</version>
            <executions>
              <execution>
                <goals>
                  <goal>integration-test</goal>
                </goals>
              </execution>
            </executions>
          </plugin>
        </plugins>
        <testResources>
            <testResource>
                <directory>src/test/resources</directory>
            </testResource>
        </testResources>
      </build>
      <dependencies>
        <dependency>
            <groupId>org.jboss.arquillian.container</groupId>
            <artifactId>arquillian-jbossas-remote-5.1</artifactId>
            <version>1.0.0.Alpha4</version>
            <scope>test</scope>
          </dependency>
        <dependency>
          <groupId>org.jboss.jbossas</groupId>
          <artifactId>jboss-as-client</artifactId>
          <version>5.1.0.GA</version>
          <type>pom</type>
        </dependency>
      </dependencies>
    </profile>  
</profiles>

2. Make sure you are using the correct version of the javassist library in your business logic project. It has to be version 3.10.0.GA. The one coming with JBoss AS 5.1.0 GA is older (3.9.0) and will not work properly!

3. Write your test classes and run them using "mvn -Pit"!

4. Here is an example of a simple class just calling a EJB3 SLSB (test-persistence.xml has to be in src/test/resources together with jndi.properties):
import javax.naming.Context;
import javax.naming.InitialContext;

import junit.framework.Assert;
import myclasses.ejb.actions.interfaces.AuthenticatorLocal;
import myclasses.ejb.valueobjects.RequestorInfo;

import org.jboss.arquillian.api.Deployment;
import org.jboss.arquillian.junit.Arquillian;
import org.jboss.shrinkwrap.api.ArchivePaths;
import org.jboss.shrinkwrap.api.ShrinkWrap;
import org.jboss.shrinkwrap.api.spec.JavaArchive;
import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(Arquillian.class)
public class AuthenticatorBeanIT {
 @Deployment
 public static JavaArchive createDeployment() {
  return ShrinkWrap.create(JavaArchive.class, "testAuthenticatorBean.jar")
  .addClasses(AuthenticatorBean.class, AuthenticatorLocal.class, RequestorInfo.class)
  .addManifestResource("test-persistence.xml", ArchivePaths.create("persistence.xml"));
 }

 @Test
 public void isAuthenticatedTest() throws Exception {
  Context ctx = new InitialContext();
  AuthenticatorLocal auth;

  Object obj = ctx.lookup("AuthenticatorBean/local");

  auth = (AuthenticatorLocal) obj;
  RequestorInfo req = new RequestorInfo();

  // Trying with a valid user (according to import.sql!)
  req.setRequestorId("0123456789");
  req.setPassword("secret");
  Assert.assertTrue(auth.isAuthenticated(req));
  
  // Trying with a invalid user
  req.setRequestorId("0123456789");
  req.setPassword("incorrect");  
  Assert.assertFalse(auth.isAuthenticated(req));
 }
}


5. Here is an example of a class persisting something to the DB and cleaning up after itself (Note: you have to lookup the EntityManager and EntityTransaction yourself. They can not be injected since you are not in an EJB(:
import java.util.ArrayList;
import java.util.Date;
import java.util.List;

import javax.naming.Context;
import javax.naming.InitialContext;
import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.EntityTransaction;
import javax.persistence.Query;

import junit.framework.Assert;
import myclasses.ejb.actions.interfaces.MyClassLocal;
import myclasses.ejb.valueobjects.Info;

import org.jboss.arquillian.api.Deployment;
import org.jboss.arquillian.junit.Arquillian;
import org.jboss.shrinkwrap.api.ArchivePaths;
import org.jboss.shrinkwrap.api.ShrinkWrap;
import org.jboss.shrinkwrap.api.spec.JavaArchive;
import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(Arquillian.class)
public class BarCodeBoardingPassBeanIT {
 @Deployment
 public static JavaArchive createDeployment() {
  return ShrinkWrap.create(JavaArchive.class, "testBarCodeBoardingPassBean.jar")
  .addClasses(MyClassLocal.class)
  .addManifestResource("test-persistence.xml", ArchivePaths.create("persistence.xml"));
 }

 @Test
 public void persistBoardingPassInfoTest() throws Exception {
  Context ctx = new InitialContext();
  MyClassLocal mcl;

  Object obj = ctx.lookup("MyClasssBean/local");

  mcl = (MyClassLocal) obj;
  
  Info info = new Info("XXX");
  mcl.persistInfo(info);
  
  
  // Now checking that the data is properly persisted in the DB
  EntityManagerFactory emFactory = (EntityManagerFactory) ctx.lookup("java:/myTestEntityManagerFactory");
  EntityManager em = emFactory.createEntityManager();
  EntityTransaction tx = em.getTransaction();
  tx.begin();

  Query query = em.createQuery("From Info where text=:info");
  query.setParameter("info", "XXX");
  runQuery(query);
  
  // Cleaning up
  query = em.createQuery("delete from Info where text=:info");
  query.setParameter("info", "XXX");
  query.executeUpdate();
  
  tx.commit();
 }
 
 private void runQuery(Query query) {
  try {
   query.getSingleResult();
   Assert.assertTrue(true);
  } catch (Exception e) {
   Assert.assertTrue(false);
  }
 }
}

6. For the previous test class to work you need a persistence.xml which looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"
    version="1.0">

     <persistence-unit name="myTestPU" transaction-type="RESOURCE_LOCAL" >
        <provider>org.hibernate.ejb.HibernatePersistence</provider>
        <non-jta-data-source>java:/myAppDS</non-jta-data-source>
        <properties>
            <property name="hibernate.dialect" value="org.hibernate.dialect.PostgreSQLDialect"/>
            <property name="hibernate.current_session_context_class" value="jta"/>
            <property name="hibernate.show_sql" value="true"/>
            <property name="hibernate.format_sql" value="true"/>
            <property name="hibernate.default_schema" value="public"/>
            <property name="hibernate.hbm2ddl.auto" value="validate" />
            <property name="jboss.entity.manager.factory.jndi.name" value="java:/myTestEntityManagerFactory"/>
            
        </properties>
    </persistence-unit>

</persistence>

05 November 2010

Test syntax and xml publishing

Bambitroll @ 15:40
Use this little page to format the XML code for you


#! /bin/env python
def python_funct():
   a = a + b
   print "Hello Highlighted code"

class Foo(Bar):
   pass


<Location /svn>

  # Uncomment this to enable the repository
  DAV svn

  # The following three lines allow anonymous read, but make
  # committers authenticate themselves.  It requires the 'authz_user'
  # module (enable it with 'a2enmod').
  #<LimitExcept GET PROPFIND OPTIONS REPORT>
  Require valid-user
  #</LimitExcept> 

</Location>

23 August 2010

How to update Smooks in JBoss ESB

Bambitroll @ 10:51
The latest JBoss SOA platform (5.0) uses JBoss ESB 4.7 (https://www.jboss.com/products/platforms/soa/components/) which itself uses Smooks 1.2.3 (http://docs.jboss.org/jbossesb/docs/4.7/manuals/html/ReleaseNotes.html)
The latest and greatest Smooks is 1.3.1 and contains certain new features.
So if you absolutely need them in your JBoss Soa Platform 5.0, you will have to update Smooks manually. To do so, download the zip file from http://downloads.sourceforge.net/smooks/smooks-1.3.1.zip, unpack it and replace all the milyn-* from $JBOSS_ESB_HOME/server/default/deployers/esb.deployer/lib/ with the ones from the lib folder in the zip file.

Warning: This configuration is most likely not supported in case you need support from Red Hat.

14 June 2010

Agilo for Scrum

Bambitroll @ 15:29
I found a tool called Agilo (built on of Trac) which looks promising as a Scrum tool. It supports most of the Scrum features and is actually quite nice to use.

Here is a description regarding how to install Agilo

I used a lot of information from the following pages (mostly for Hardy but can be reused):
 http://trac.edgewall.org/wiki/0.11/TracOnUbuntu
 http://trac.edgewall.org/wiki/0.10.4/TracOnUbuntuHardy
 http://trac.edgewall.org/wiki/TracUbuntuMultipleProjects
 http://anantgarg.com/2009/03/25/subversion-trac-multiple-projects/
 http://www.agile42.com/cms/pages/download-install/
 http://specialbrands.net/2010/01/12/agilo-a-scrum-tool-agile-scrum/

Now for the step by step install (all commands are run as root!):

We start from a fresh Ubuntu 10.04 Lucid Lynx server

First we add Apache and some tools

apt-get install apache2 libapache2-mod-python libapache2-svn python-setuptools subversion python-subversion python-simplejson

Then we install trac

apt-get install trac

Then we configure Trac

mkdir /var/lib/trac
chown www-data:www-data /var/lib/trac
ln -s /var/lib/trac/ /trac
vi /etc/apache2/sites-available/trac and add this:

<VirtualHost *>
ServerAdmin user@email.com
ServerName you.servername.com
DocumentRoot /var/www
ErrorLog /var/log/apache2/error.trac.log
CustomLog /var/log/apache2/access.trac.log combined

<location /trac>
SetHandler mod_python
PythonInterpreter main_interpreter
PythonHandler trac.web.modpython_frontend
PythonOption TracEnvParentDir /var/lib/trac
PythonOption TracUriRoot /trac
PythonOption PYTHON_EGG_CACHE /tmp
</location>
# use the following for one authorization for all projects 
# (names containing "-" are not detected):
<LocationMatch "/trac/[[:alnum:]]+/login">
AuthType Basic
AuthName "trac"
AuthUserFile /etc/apache2/dav_svn.passwd
Require valid-user
</LocationMatch>
</VirtualHost *>

a2dissite default
a2ensite trac
/etc/init.d/apache2 reload

Then we configure SVN:

mkdir /var/lib/svn
svnadmin create /var/lib/svn/theproject
chown -R www-data /var/lib/svn


Then we create the Trac site for the project:

trac-admin /var/lib/trac/theproject initenv
chown -R www-data /var/lib/trac
vi /etc/apache2/mods-available/dav_svn.conf and add this:

<location /svn>
# Uncomment this to enable the repository
DAV svn

# Set this to the path to your repository
#SVNPath /var/lib/svn
# Alternatively, use SVNParentPath if you have multiple repositories under
# under a single directory (/var/lib/svn/repo1, /var/lib/svn/repo2, ...).
# You need either SVNPath and SVNParentPath, but not both.
SVNParentPath /var/lib/svn
#Provides directory listing of repositories
SVNListParentPath On

# Basic Authentication is repository-wide.  It is not secure unless
# you are using https.  See the 'htpasswd' command to create and
# manage the password file - and the documentation for the
# 'auth_basic' and 'authn_file' modules, which you will need for this
# (enable them with 'a2enmod').
AuthType Basic
AuthName "Subversion Repository"
AuthUserFile /etc/apache2/dav_svn.passwd
AuthzSVNAccessFile /etc/apache2/authz_svn.access

Require valid-user
</location>

/etc/init.d/apache2 reload


Now Agilo

Download and unpack Agilo Source Code and then run:
python setup.py install
vi /trac/theproject/conf/trac.ini and make sure that you have these lines

[components]
agilo.* = enabled
webadmin.* = enabled

Then run:
trac-admin /var/lib/trac/theproject upgrade
/etc/init.d/apache2 reload

Add the AccountManager plugin  http://trac-hacks.org/wiki/AccountManagerPlugin

Then you have to configure the access control for Trac (we use the same users for SVN and Trac):
htpasswd -cm /etc/apache2/dav_svn.passwd - NB: the -c creates the file so use it only once
vi /etc/apache2/authz_svn.access and have something like this:

[theproject:/]
theuser = rw


trac-admin /trac/theproject permission remove anonymous BROWSER_VIEW CHANGESET_VIEW FILE_VIEW LOG_VIEW MILESTONE_VIEW REPORT_SQL_VIEW REPORT_VIEW ROADMAP_VIEW SEARCH_VIEW TICKET_CREATE TICKET_MODIFY TICKET_VIEW TIMELINE_VIEW WIKI_CREATE WIKI_MODIFY WIKI_VIEW
trac-admin /trac/theproject permission add TRAC_ADMIN theadminuser
/etc/init.d/apache2 restart

YOU ARE DONE!

How to add new users (as root):
htpasswd -m /etc/apache2/dav_svn.passwd theuser (use -c for the first time to create the file)
vi /etc/apache2/authz_svn.access and add the new user there with rw
/etc/init.d/apache2 reload
trac-admin /trac/theproject permission add theuser the_perm with THE_PERM being TEAM_MEMBER, PRODUCT_OWNER, SCRUM_MASTER or TRAC_ADMIN

How to increase the default attachment size for documents the wiki
Change max_size value in the [attachment] section of the trac.ini of your site
More info on the online doc which comes with every new Trac site!

24 March 2010

JBoss and web-console

Bambitroll @ 11:06
I have been fighting with my JBoss 4.3 for a good hour now just because I want to access the web console located at htpp://localhost:8080/web-console/.
The problem is that no matter what I set for the user in the config file, it is not taken into account.
So here is how I got it to work:
- Go to $JBOSS_HOME/server/default/conf/props
- Add the files web-console-users.properties and web-console-roles.properties
- Edit  $JBOSS_HOME/server/default/conf/login-config.xml and make sure that the path to the 2 files above looks like props/web-console-*.properties.

For some fishy reasons, if you have the files right under conf/ and login-config.xml properly referring to them, it will not work. You have to have them in conf/props/ and update the reference in login-config.xml.

16 March 2010

Seamgen to Maven

Bambitroll @ 17:51
As a follow-up of my previous post, I generated a Seam 2.1.2 project via seamgen and tried to make it work in the maven structure I made earlier.
The biggest problem was with the config files and have richfaces properly working. But I managed and the tar of the project is here.

The files I had to change where persistence.xml, pages.xml, web/pom.xml, web.xml and components.xml.

Now download it, unpack, copy your Java classes and webapp generated by seamgen where they belong, run "mvn clean install package" and you should be able to deploy the .ear file without any problems (given that you have the corresponding *ds.xml file)

10 March 2010

Seam 2.0 and Maven

Bambitroll @ 16:54
Since I am playing with Maven and Seam, I had to manage to create a Seam 2.0 project which can be built with Maven (Seam 3.0 will be fully mavenized).

So here is what I found...
All you need to know is here:
http://seamframework.org/Documentation/IsThereAMaven2ProjectTemplateForSeamApplications

The first link (http://kasper.eobjects.dk/2009/04/seam-ejbs-and-ear-packaging-in-maven.html) works out of the box and uses sub-projects for the different modules (and Eclipse is ok with that!) but I am a bit unsure regarding the test part. It should work fine from the command line though.

The second link (http://www.smies.com/blog/?p=30) seems to have the test part built in but I have not tested it yet. It also uses a flat structure for the project which is a bit less nice...

So after a bit of work, I improved Kasper's setup so that the project now has a persistence unit defined (remember to add a proper *ds.xml file in your deploy folder) and is also able to do some sort of hot-deploy. You can get the tar file here.


For the hot-deploy part, I used something described here (http://relation.to/Bloggers/FastAndIncrementalPackagingInJBossTools) which copies files automatically upon changes from your workspace to your deploy folder (or any other folder of your choosing) according to some rules so you can filter the files you do not want.
This JBoss Tools might be interesting in other cases as well...

So to have the hot-deploy working, I configured the Project archives like this:
- one exploded archive for the ear
- one exploded archive for the jar, exploded inside the ear above
- one exploded archive for the war, exploded inside the ear above, and with 2 file sets (one taking all from web/src/main/webapp (aka the xhtml files) and the other taking all from web/target/JMSeamExample-web-1.0 minus **/*.*html (basically only the libs)).
An explanatory picture can be found here.

If one changes a Java class, the best is to right click on the project in Project archives and choose "Build full" to force a redeploy. Otherwise for xhtml files it happens by itself.
I know it is not has seamless as with the out of the box seam-gen integration but it is not too bad.