Early Adopter

My mum keeps telling me I used to be gifted child. Of course I don't remember everything I did at the age of three, but I seem to have signed up for the java.net forums at that time. Quite impressive!

Java XML Binding with Property Change Support

In an earlier post, I was talking about generating Java classes with xjc with property change support which is not enabled by default. This post explains the details, an in fact the Maven build explained below is a lot a easier than my original approch with Ant which required some downloads to set things up.

The general idea is same as described by Kohsuke Kawaguchi in a post of 2007, but his POM is broken with the current stage of the Maven repositories. In addition, there is a tweak with the PropertyListener customization.

So here is a little example schema:

<?xml version="1.0" encoding="UTF-8"?>
<schema xmlns="http://www.w3.org/2001/XMLSchema" xmlns:tns="http://www.example.org/person/" targetNamespace="http://www.example.org/person/">
    <complexType name="Person">
      <sequence>
        <element name="firstName" type="string"></element>
        <element name="lastName" type="string"></element>
        <element name="address" type="tns:Address"></element>
      </sequence>
    </complexType>

    <complexType name="Address">
      <sequence>
        <element name="street" type="string"></element>
        <element name="houseNumber" type="string"></element>
        <element name="city" type="string"></element>
        <element name="postalCode" type="string"></element>
        <element name="country" type="string"></element>
      </sequence>
    </complexType>
</schema>

We use the following file bindings.xjb to customize the Java classes generated by xjc. We make all classes serializable and include support for PropertyChangeListeners.

<?xml version="1.0" encoding ="UTF-8"?>
<jaxb:bindings
    schemaLocation="person.xsd"
    version="2.1"
    xmlns:li="http://jaxb.dev.java.net/plugin/listener-injector"
    xmlns:jaxb="http://java.sun.com/xml/ns/jaxb"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <jaxb:bindings node="/xs:schema">
    <jaxb:globalBindings>
      <jaxb:serializable uid="1" />
    </jaxb:globalBindings>
    <li:listener>java.beans.PropertyChangeListener</li:listener>    
  </jaxb:bindings>
</jaxb:bindings>

The following POM takes care of the code generation:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>jaxb-properties</groupId>
  <artifactId>jaxb-properties</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <name>JAXB PropertyListener Demo</name>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>2.3</version>
        <configuration>
          <source>1.6</source>
          <target>1.6</target>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.jvnet.jaxb2.maven2</groupId>
        <artifactId>maven-jaxb2-plugin</artifactId>
        <version>0.7.3</version>
        <executions>
          <execution>
            <goals>
              <goal>generate</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <extension>true</extension>
          <generatePackage>com.example.jaxb</generatePackage>
          <schemaIncludes>
            <schemaInclude>person.xsd</schemaInclude>
          </schemaIncludes>
          <bindingIncludes>
            <bindingInclude>bindings.xjb</bindingInclude>
          </bindingIncludes>
          <args>
            <arg>-Xinject-listener-code</arg>
          </args>
        </configuration>
        <dependencies>
          <dependency>
            <groupId>com.sun.xml.bind</groupId>
            <artifactId>jaxb-xjc</artifactId>
            <version>2.1.13</version>
          </dependency>
          <dependency>
            <groupId>org.jvnet.jaxb2-commons</groupId>
            <artifactId>property-listener-injector</artifactId>
            <version>1.1-SNAPSHOT</version>
            <exclusions>
              <exclusion>
                <groupId>com.sun.xml.bind</groupId>
                <artifactId>jaxb-xjc</artifactId>
              </exclusion>
            </exclusions>
          </dependency>
        </dependencies>
      </plugin>

    </plugins>
  </build>
</project>

You do need to use the 1.1-SNAPSHOT version of the property-listener-injector, as the 1.0 release only supports VetoableChangeListeners and not the PropertyChangeListeners we want to use. The <li:listener> element in the bindings file is used to define the listener class.

It is important to exclude the jaxb-xjc dependency from the property-listener-injector, or else it will try to download an non-existing snapshot version. Looks like something is broken in the java.net repository, but then again, working with snapshot releases is always dangerous....

Glassfish Logging with slf4j

Update 18 Dec 2010: There is an extended version of this article.

It can be annoying to have each third-party library or framework in your application logging to a different logfile. slf4j with its various adapters enables you to collect logging events from various APIs (org.slf4j.api, java.util.logging, org.apache.commons.logging) and redirect them to a logging backend of your choice.

I've always preferred log4j or, more recently, logback over java.util.logging, so after working with Glassfish v3 for a while, I tried to tweak it to use logback over slf4j and the jul-to-slf4j bridge.

To redirect java.util.logging used by Glassfish v3, you put the following libs on your classpath:

• jul-to-slf4j.jar
• slf4j-api.jar
• logback-classic.jar
• logback-core.jar

(You can replace the logback libs by any other supported backend like log4j, see the slf4j docs for more details.)
The problem with Glassfish is:

• You need to take care of its class loader hierarchy and make sure that the logging jars get picked up early enough.
• Glassfish does some all-too-clever logger manipulation in its LogManagerService which will get in your way if you don't like the Glassfish defaults: It redirects all System.out messages to a logger.

I did not manage to fully supersede Glassfish logging by slf4j and logback, but at least I can copy all Glassfish log events to my logback application log like this:

• Put the logging libs in [install-root]/lib/endorsed.
• Build a JAR containing your logback.xml configuration and put it in the same place.
• Edit an entry in [instance-root]/config/logging.properties, setting
  handlers = org.slf4j.bridge.SLF4JBridgeHandler

Now if you omit a ConsoleAppender from your logging configuration, the Glassfish server.log will no longer contain any log output from your application. It will still contain the log messages from Glassfish itself, but they will also be included in your own logback logfile.

jeeunit: In-Container Integration Testing for Java EE 6

Recently, I've started working with Java EE 6 in general and Glassfish v3 in particular. Some software engineering best practices do not really depend on the framework or even the language you work with, so I was looking for a convenient way of doing automatic integration tests.

I've been in the habit of using JUnit not just for unit tests, but also for integration or system tests, so this was a natural starting point.

Traditionally, Java EE containers were heavy-weight machinery so that people preferred writing their tests to run out-of-container, paying the price of emulating or mocking some of the container functionality.

Maybe I've been looking in the wrong places, but most of the out-of-container testing approaches like ejb3unit seem to carry more baggage than the container itself, at least when it comes to working with Glassfish v3.

Anyway, as I could not find a ready-to-go solution, I wrote a little JUnit extension called jeeunit together with an example project showing how to set things up with Glassfish v3 and Maven.

The jeeunit project is available from Google Code under an Apache License.

The Ultimate Software Metric

Misconceptions about Java Internationalization

Let me start with a joke:

What do you call someone who speaks three languages?
Trilingual.
What do you call someone who speaks two languages?
Bilingual.
What do you call someone who speaks one language?
American.

To be fair on Americans, even most of us multilingual Europeans tend to be biased when it comes to internationalization, tacitly assuming that text is written left-to-right and can be sorted from A to Z.

Most Java developers are familiar with resource bundles backed by properties files. The basics can be found in the Internationalization Trail of the Java Tutorial. Multilingual Java applications often come with a set of properties files, e.g.

• MyApp_de_AT.properties
• MyApp_de.properties
• MyApp_es.properties
• MyApp.properties

where MyApp.properties contains the "default" message resources in English, MyApp_de.properties and MyApp_es.properties contain the German and Spanish resources, respectively, and MyApp_de_AT.properties contains some country specific variants for the Austrian flavour of German. Usually, files for country specific variants are sparsely populated, containing only those properties that actually differ from the mainstream language version, like Jänner (de_AT) vs. Januar (de) vs. January (en).

However, you may be surprised in this case to end up with a German string even when you requested a resource for an English locale.

Assume nothing is a sound principle for robust software development, and you should not assume that English is the default or fallback language. In fact, the fallback for resources from an unsupported locale is the system default locale, which is based on the host environment.

See the documentation for ResourceBundle.getBundle() and Locale.getDefault() for more details.

So when the default locale of your system is de_DE and you request a resource for locale en_US, the lookup order for the properties files is

1. MyApp_en_US.properties
2. MyApp_en.properties
3. MyApp_de_DE.properties
4. MyApp_de.properties
5. MyApp.properties

Hence, ResourceBundle.getString() will return a German string from MyApp_de.properties, since the first three files do not exist and the English resources are preceded by the German ones in this sequence.

There are two solutions:

1. As a user, set your default locale to en when launching the application.
2. As a developer, make sure to provide a properties file for locale en (which may be empty).

The method for changing the default locale depends on your Java VM and your operating system. Setting the system property user.language may work on some platforms, but not with the Sun JDK 1.6.0 under Linux. Instead, you need to set the environment variable LANG before launching the Java VM.

The preferred solution is the second one, of course. Even when MyApp_en.properties is empty, it will be picked up as entry point for resource lookup. If a given key cannot be found in this file, the parent file MyApp.properties will be used as fallback, which is just the desired behaviour.

Editing Resource Bundles in Eclipse

Playing around with the Apache Roller blog engine, I noticed that some of the localized German text messages were missing or broken. Roller uses plain old Java resource bundles instead of the NLS mechanisms offered by Eclipse. Editing resource bundles for multiple languages in parallel is rather a pain with a plain text editor, so I was looking for an Eclipse plugin to do this job.

(Just to avoid any confusion, even though I've been writing a lot about OSGi bundles, the term bundle is only used in the sense of a resource bundle, or properties file, in this article.)

I found two solutions, both of which have minor bugs and lack some documentation but are very helpful nevertheless. And it turned out that the second solution uses code from the first one:

• Resource Bundle Editor from Sourceforge
• Messages Editor from the Eclipse Babel project

At first, I tried the Resource Bundle Editor. The latest version is from 2007, and some people have reported conflicts with newer Eclipse versions. I installed the Bundle in my Eclipse 3.5.1, and did not notice any version conflicts at all.

However, the Resource Bundle Editor does not parse the properties files correctly. It does not recognize exclamation marks as comment signs. For comment lines of the form

!some.key = some value

the editor will display a bogus key !some.key.

Looking at the sources, I found the the PropertiesParser class only recognizes a subset of the valid properties file syntax.

After that, I had a look at the Eclipse Babel editor. Unfortunately, the Babel project does not yet provide binary downloads, so you have to build the two plugins from source.

As it turned out, parts of the Babel sources are derived from the Resource Bundle Editor sources, and the same incomplete parser code is also used in the Eclipse project in class PropertiesDeserializer.

I changed a regular expression in the source to fix the "!"-problem. You can get the binary plugins including my patch from here:

• org.eclipse.babel.core_0.8.0.201002251355.jar
• org.eclipse.babel.editor_0.8.0.201002251355.jar

After installing the plugins, go to Window | Preferences | Messages Editor and deselect the option Setup validation builder on Java projects automatically, or else you may get lots of error markers on other properties files which are not used as message bundles at all. I also set the Reports severities to Ignore and the Displayed Locales to de to narrow the Editor display to the language I'm actually working on.

To edit a resource bundle, select the properties file in the Package Explorer and open it with the Messages Editor via the context menu.

Here is a screenshot of the Messages Editor in action:

With the additional toolbar buttons, you can limit the view to missing or unused translations.

Setting up Eclipse for Roller

There is an Eclipse plug-in for almost any task, and most of them do their job rather nicely. On the other hand, even some of the more or less official ones may give you a hard time if you try to use them in combination.

Recently, I've been playing around with Apache Roller, a Java blog engine, much like Blogger or Wordpress. This project is currently in beta for the next major release 5.0, so I checked out the sources from trunk, ran the Maven build, created a PostgreSQL database and got my own blog engine up and running within minutes.

Some minor things did not quite work as expected, so I thought I'd just create an Eclipse workspace for Roller and build and run it from there. As it turned out, the combination of Maven, Subversion, and a Web Application was a rather fatal mix, and it took me a day to figure out what was going wrong.

Most of this was not a Roller issue at all: Eclipse Web application tooling (WTP) and Maven Integration (m2eclipse) just make too many implicit and conflicting assumptions which make it hard to set things up correctly, so this article is really about working with a mavenized web application in Eclipse, and Roller is just an example.

There are a couple of threads on the Roller developer mailing list dealing with Eclipse setups, but none of them really provides a working solution, so maybe this post can fill gap.

Step 1: Get Eclipse and all required plug-ins

To avoid any conflicts with other plug-ins or features not required for this project, I used a separate Eclipse installation consisting of

• Eclipse for Java EE Developers 3.5.1
• Subversive SVN Team Provider 0.7.8
• Subversive SVN Connectors 2.2.1
• Maven Integration for Eclipse 0.10.0
• Maven Integration for WTP 0.10.0

Even if you do not intend to commit any changes from Eclipse, you will need the Subversion integration to deploy your web application from a Subversion working copy, otherwise Eclipse will try to pack the hidden .svn directories into the JARs and WARs and complain about duplicate paths.

Step 2: Set up m2eclipse

m2eclipse has a built-in pre-release version of Maven 3.0.0 which is not compatible with most existing projects based on Maven 2.x. Get a local installation of Maven 2.1.0 and define it as default for m2eclipse in Window | Preferences | Maven | Installations.

Step 3: Set up Tomcat

Download and install Tomcat 6.0.24 to a local directory. Create a Tomcat server instance for Eclipse via Window | Preferences | Server | Runtime Environments pointing to your Tomcat installation directory.

Install the additional prerequisites of Roller in the Tomcat lib directory:

• mail.jar
• activation.jar
• your JDBC driver

Step 4: Get Roller into your Eclipse workspace

Create a new empty workspace and switch to the SVN Repository Exploring perspective. There is supposed to be an integration of m2eclipse and Subversive, which I never managed to get to work, so this is why I use the following somewhat clumsy procedure to populate my workspace:

• Switch to the SVN Repository Exploring perspective and define a new repository location for https://svn.apache.org/repos/asf.
• Check out Roller from roller/trunk. This will create a new project roller-project in your workspace.
• Unfortunately, the Maven modules of this project do not yet appear as separate Eclipse projects. To change this, delete the project from your workspace and use File | Import | Maven | Existing Maven Projects. Select the workspace folder from your initial checkout.
• After this, you should have six Maven projects in your workspace, all shared via Subversive.

Step 5: Apply some fixes in the workspace

• Go to roller-weblogger-business and delete src/test/resources/org/apache/roller/weblogger/business/package.html, since this file would cause a clash with another copy from src/main/resources.
• Open /roller-weblogger-web/src/main/webapp/WEB-INF/security.xml and replace spring-security-2.0.1-openidfix.xsd by spring-security-2.0.4.xsd.
• Copy your roller-custom.properties to /roller-weblogger-web/src/main/resources.

Step 6: Configure your web application

• Open the project properties of roller-weblogger-web.
• Select the Java EE Module Dependencies and activate roller-planet-business, roller-core and roller-weblogger-business.
• Make sure that the resources from all dependent projects will get copied into the web application by modifying the build path settings of roller-planet-business, roller-weblogger-business and roller-weblogger-web. Select Java Build Path from the project properties and remove the Excluded: ** entry from src/main/resources for each of these projects.

Step 7: Run a Maven build

• Select roller-project/pom.xml. From the context menu, select Run As | Maven build...
• In the launcher dialog, fill in the goals clean install and (optionally) check Skip Tests to save some time during each build.
• When the build has completed, select all projects and press F5 so that Eclipse will see all the resources created by Maven.
• This step is required, since the Maven build generates some additional resources and runs the OpenJPA Enhancer. These two steps would not be handled by the Eclipse automatic build.

Step 8: Make sure that Eclipse picks up the generated resources

• Create a folder /roller-weblogger-web/src/main/sql and turn it into a source folder.
• Copy /roller-weblogger-business/target/dbscripts into this folder.

Step 9: Get Rolling!

• Select roller-weblogger-web and invoke Run As | Run on Server from the context menu.
• Select the Tomcat instance created in Step 3 and activate it as default if you like and click Finish.

Troubleshooting

• If you get stuck, clean the Tomcat instance. Open the Servers view and select Tomcat. From the context menu, invoke Clean...
• To check the web application assembled by Eclipse, have a look into <Eclipse workspace>/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/roller-weblogger-web/

All of this should have been a lot easier. m2eclipse will have to mature, and Eclipse should be more flexible about resource locations...

Running parameterized JUnit tests in parallel

We use JUnit 4 for Anaconda, not just for unit tests, but also for integration or system tests. Typically, we iterate over all features of a given class in a database and validate each feature.

Note: I'm using the term feature in the sense of map feature, not in the sense of application feature or implemented requirement.

A simple pattern for such tests is

public class FeatureTest
{ 
    @Test
    public void testAllFeatures()
    {
        for (Feature feature : findAllFeatures())
        {
            testOneFeature(feature);
        }
    }

    private void testOneFeature(Feature feature)
    {
        // some logic with one or more JUnit assertions
    }
}

Obviously, this naive approach has the following drawbacks:

• The test fails and terminates on the first incorrect feature. The remaining features will not be tested.
• All features get tested sequentially. This may take awfully long for a large database.

JUnit 4 has a specialized runner Parameterized for running all tests in a given class with different parameters from a given list of parameter sets. An instance of the test class is created for each parameter set, and the parameters are passed to the constructor via reflection:

@RunWith(Parameterized.class)
public class FeatureTest
{
    // This is the parameter for each instance of the test.
    private Feature feature;

    public FeatureTest(Feature feature)
    {
         this.feature = feature;
    }

    @Parameters
    public static Collection<Object[]> getParameters()     
    {         
        List<Feature> features = findAllFeatures();
        List<Object> parameters = new ArrayList<Object[]>(features.size());
        for (Feature feature : features)         
        {
             parameters.add(new Object[] { feature };
        }
        return parameters;
    }
 
    @Test
    public void testOneFeature()     
    {
        // assertions acting on the feature member
    }
} 

This solves the first problem: Each feature gets tested in its own test instance. Now if there is a large number of features or if each individual test is very expensive, we would like to run the test instances in parallel, using a thread pool, or maybe even a grid of multiple computers.

Browsing through the JUnit sources, I found a surprisingly easy way of parallelizing the tests with a thread pool, simply by using a custom runner:

@RunWith(Parallelized.class)
public class FeatureTest
{
   // same class body as above
}

All you need is a simple extension of the Parameterized runner:

public class Parallelized extends Parameterized
{
    
    private static class ThreadPoolScheduler implements RunnerScheduler
    {
        private ExecutorService executor; 
        
        public ThreadPoolScheduler()
        {
            String threads = System.getProperty("junit.parallel.threads", "16");
            int numThreads = Integer.parseInt(threads);
            executor = Executors.newFixedThreadPool(numThreads);
        }
        
        @Override
        public void finished()
        {
            executor.shutdown();
            try
            {
                executor.awaitTermination(10, TimeUnit.MINUTES);
            }
            catch (InterruptedException exc)
            {
                throw new RuntimeException(exc);
            }
        }

        @Override
        public void schedule(Runnable childStatement)
        {
            executor.submit(childStatement);
        }
    }

    public Parallelized(Class klass) throws Throwable
    {
        super(klass);
        setScheduler(new ThreadPoolScheduler());
    }
}

The RunnerScheduler interface is fairly new in JUnit and marked as experimental. I discovered it in the current version JUnit 4.8.1 and found it missing in JUnit 4.4.0 which we have been using so far. RunnerScheduler is also available in JUnit 4.7.0, but I did not check if this is the earliest version.

Eclipse Forms and Data Binding

Our map compiler Anaconda reads all parameters and settings from a configuration file, which over time has evolved from a simple Java properties file to a not-so-simple XML file which is validated by an XML schema.

To access the configuration at runtime, we use Java XML Bindings generated from our schema by xjc in a straightforward manner, without any fancy customizations.

Using the XML editing support in Eclipse, it is very easy to edit and validate a configuration file, at least from a developer perspective. However, our customers will not be too happy about editing large XML files by hand, so the idea is to develop a form-based configuration editor for our RCP application Anaconda Workbench, similar to the manifest editor of Eclipse PDE or the POM editor of m2eclipse.

These editors are based on Eclipse UI forms, another layer on top of SWT and JFace, which is obviously powerful enough for complex tasks, as demonstrated by the above examples. Less obviously, it is rather poorly documented. The Eclipse online help has just 10 brief pages about UI Forms and the Javadocs which, as usual, are not very useful for getting started.

The Eclipe Rich Client Platform book also has not more than one page on UI Forms and a link to an online article from 2004. On the Eclipse site, there are two more recent articles

• Eclipse Forms: Rich UI for the Rich Client
• Eclipse Forms: New in 3.3

Looking for futher tutorials, I came across Marco van Meegen's critical review Eclipse Forms im Härtetest. I decided to make up my own mind, but after implementing a few examples with Eclipse Forms, I largely agree to his criticism: the API forces you to write lots of repetitive code and it is not easy to figure out how to wire up the different classes to do your job.

To alleviate the shortcomings of Eclipse Forms, Marco created yet another layer called RCPForms. So I gave it a try, and I found it a lot easier to use than working with UI Forms directly. I had to use the sources from the Subversion trunk at Sourceforge, the older tagged or released versions do not seem to work with Eclipse 3.5.

Eclipse forms are usually wrapped by a ManagedForm which manages the state of the form parts and the underlying data models.

The form parts and the models can vary independently, and one model can be shared by multiple form parts.

A managed form is dirty, when one of its parts is more recent than the underlying model. Conversely, it is stale, when a change in the underlying model is not yet reflected on the UI.

To handle this form lifecycle, RCPForms expects the model to support PropertyChangeListeners, which again requires you to add some boilerplate code to your model beans. For my example with a JAXB model, I managed to tweak xjc to generate the required listeners, which is to be discussed in detail in a separate article.

Here is a screenshot of a simple example:

The warnings result from validators on missing mandatory fields.