Editing Resource Bundles in Eclipse

Playing around with the Apache Roller blog engine, I noticed that some of the localized German text messages were missing or broken. Roller uses plain old Java resource bundles instead of the NLS mechanisms offered by Eclipse. Editing resource bundles for multiple languages in parallel is rather a pain with a plain text editor, so I was looking for an Eclipse plugin to do this job.

(Just to avoid any confusion, even though I've been writing a lot about OSGi bundles, the term bundle is only used in the sense of a resource bundle, or properties file, in this article.)

I found two solutions, both of which have minor bugs and lack some documentation but are very helpful nevertheless. And it turned out that the second solution uses code from the first one:

• Resource Bundle Editor from Sourceforge
• Messages Editor from the Eclipse Babel project

At first, I tried the Resource Bundle Editor. The latest version is from 2007, and some people have reported conflicts with newer Eclipse versions. I installed the Bundle in my Eclipse 3.5.1, and did not notice any version conflicts at all.

However, the Resource Bundle Editor does not parse the properties files correctly. It does not recognize exclamation marks as comment signs. For comment lines of the form

!some.key = some value

the editor will display a bogus key !some.key.

Looking at the sources, I found the the PropertiesParser class only recognizes a subset of the valid properties file syntax.

After that, I had a look at the Eclipse Babel editor. Unfortunately, the Babel project does not yet provide binary downloads, so you have to build the two plugins from source.

As it turned out, parts of the Babel sources are derived from the Resource Bundle Editor sources, and the same incomplete parser code is also used in the Eclipse project in class PropertiesDeserializer.

I changed a regular expression in the source to fix the "!"-problem. You can get the binary plugins including my patch from here:

• org.eclipse.babel.core_0.8.0.201002251355.jar
• org.eclipse.babel.editor_0.8.0.201002251355.jar

After installing the plugins, go to Window | Preferences | Messages Editor and deselect the option Setup validation builder on Java projects automatically, or else you may get lots of error markers on other properties files which are not used as message bundles at all. I also set the Reports severities to Ignore and the Displayed Locales to de to narrow the Editor display to the language I'm actually working on.

To edit a resource bundle, select the properties file in the Package Explorer and open it with the Messages Editor via the context menu.

Here is a screenshot of the Messages Editor in action:

With the additional toolbar buttons, you can limit the view to missing or unused translations.

Setting up Eclipse for Roller

There is an Eclipse plug-in for almost any task, and most of them do their job rather nicely. On the other hand, even some of the more or less official ones may give you a hard time if you try to use them in combination.

Recently, I've been playing around with Apache Roller, a Java blog engine, much like Blogger or Wordpress. This project is currently in beta for the next major release 5.0, so I checked out the sources from trunk, ran the Maven build, created a PostgreSQL database and got my own blog engine up and running within minutes.

Some minor things did not quite work as expected, so I thought I'd just create an Eclipse workspace for Roller and build and run it from there. As it turned out, the combination of Maven, Subversion, and a Web Application was a rather fatal mix, and it took me a day to figure out what was going wrong.

Most of this was not a Roller issue at all: Eclipse Web application tooling (WTP) and Maven Integration (m2eclipse) just make too many implicit and conflicting assumptions which make it hard to set things up correctly, so this article is really about working with a mavenized web application in Eclipse, and Roller is just an example.

There are a couple of threads on the Roller developer mailing list dealing with Eclipse setups, but none of them really provides a working solution, so maybe this post can fill gap.

Step 1: Get Eclipse and all required plug-ins

To avoid any conflicts with other plug-ins or features not required for this project, I used a separate Eclipse installation consisting of

• Eclipse for Java EE Developers 3.5.1
• Subversive SVN Team Provider 0.7.8
• Subversive SVN Connectors 2.2.1
• Maven Integration for Eclipse 0.10.0
• Maven Integration for WTP 0.10.0

Even if you do not intend to commit any changes from Eclipse, you will need the Subversion integration to deploy your web application from a Subversion working copy, otherwise Eclipse will try to pack the hidden .svn directories into the JARs and WARs and complain about duplicate paths.

Step 2: Set up m2eclipse

m2eclipse has a built-in pre-release version of Maven 3.0.0 which is not compatible with most existing projects based on Maven 2.x. Get a local installation of Maven 2.1.0 and define it as default for m2eclipse in Window | Preferences | Maven | Installations.

Step 3: Set up Tomcat

Download and install Tomcat 6.0.24 to a local directory. Create a Tomcat server instance for Eclipse via Window | Preferences | Server | Runtime Environments pointing to your Tomcat installation directory.

Install the additional prerequisites of Roller in the Tomcat lib directory:

• mail.jar
• activation.jar
• your JDBC driver

Step 4: Get Roller into your Eclipse workspace

Create a new empty workspace and switch to the SVN Repository Exploring perspective. There is supposed to be an integration of m2eclipse and Subversive, which I never managed to get to work, so this is why I use the following somewhat clumsy procedure to populate my workspace:

• Switch to the SVN Repository Exploring perspective and define a new repository location for https://svn.apache.org/repos/asf.
• Check out Roller from roller/trunk. This will create a new project roller-project in your workspace.
• Unfortunately, the Maven modules of this project do not yet appear as separate Eclipse projects. To change this, delete the project from your workspace and use File | Import | Maven | Existing Maven Projects. Select the workspace folder from your initial checkout.
• After this, you should have six Maven projects in your workspace, all shared via Subversive.

Step 5: Apply some fixes in the workspace

• Go to roller-weblogger-business and delete src/test/resources/org/apache/roller/weblogger/business/package.html, since this file would cause a clash with another copy from src/main/resources.
• Open /roller-weblogger-web/src/main/webapp/WEB-INF/security.xml and replace spring-security-2.0.1-openidfix.xsd by spring-security-2.0.4.xsd.
• Copy your roller-custom.properties to /roller-weblogger-web/src/main/resources.

Step 6: Configure your web application

• Open the project properties of roller-weblogger-web.
• Select the Java EE Module Dependencies and activate roller-planet-business, roller-core and roller-weblogger-business.
• Make sure that the resources from all dependent projects will get copied into the web application by modifying the build path settings of roller-planet-business, roller-weblogger-business and roller-weblogger-web. Select Java Build Path from the project properties and remove the Excluded: ** entry from src/main/resources for each of these projects.

Step 7: Run a Maven build

• Select roller-project/pom.xml. From the context menu, select Run As | Maven build...
• In the launcher dialog, fill in the goals clean install and (optionally) check Skip Tests to save some time during each build.
• When the build has completed, select all projects and press F5 so that Eclipse will see all the resources created by Maven.
• This step is required, since the Maven build generates some additional resources and runs the OpenJPA Enhancer. These two steps would not be handled by the Eclipse automatic build.

Step 8: Make sure that Eclipse picks up the generated resources

• Create a folder /roller-weblogger-web/src/main/sql and turn it into a source folder.
• Copy /roller-weblogger-business/target/dbscripts into this folder.

Step 9: Get Rolling!

• Select roller-weblogger-web and invoke Run As | Run on Server from the context menu.
• Select the Tomcat instance created in Step 3 and activate it as default if you like and click Finish.

Troubleshooting

• If you get stuck, clean the Tomcat instance. Open the Servers view and select Tomcat. From the context menu, invoke Clean...
• To check the web application assembled by Eclipse, have a look into <Eclipse workspace>/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/roller-weblogger-web/

All of this should have been a lot easier. m2eclipse will have to mature, and Eclipse should be more flexible about resource locations...

Running parameterized JUnit tests in parallel

We use JUnit 4 for Anaconda, not just for unit tests, but also for integration or system tests. Typically, we iterate over all features of a given class in a database and validate each feature.

Note: I'm using the term feature in the sense of map feature, not in the sense of application feature or implemented requirement.

A simple pattern for such tests is

public class FeatureTest
{ 
    @Test
    public void testAllFeatures()
    {
        for (Feature feature : findAllFeatures())
        {
            testOneFeature(feature);
        }
    }

    private void testOneFeature(Feature feature)
    {
        // some logic with one or more JUnit assertions
    }
}

Obviously, this naive approach has the following drawbacks:

• The test fails and terminates on the first incorrect feature. The remaining features will not be tested.
• All features get tested sequentially. This may take awfully long for a large database.

JUnit 4 has a specialized runner Parameterized for running all tests in a given class with different parameters from a given list of parameter sets. An instance of the test class is created for each parameter set, and the parameters are passed to the constructor via reflection:

@RunWith(Parameterized.class)
public class FeatureTest
{
    // This is the parameter for each instance of the test.
    private Feature feature;

    public FeatureTest(Feature feature)
    {
         this.feature = feature;
    }

    @Parameters
    public static Collection<Object[]> getParameters()     
    {         
        List<Feature> features = findAllFeatures();
        List<Object> parameters = new ArrayList<Object[]>(features.size());
        for (Feature feature : features)         
        {
             parameters.add(new Object[] { feature };
        }
        return parameters;
    }
 
    @Test
    public void testOneFeature()     
    {
        // assertions acting on the feature member
    }
} 

This solves the first problem: Each feature gets tested in its own test instance. Now if there is a large number of features or if each individual test is very expensive, we would like to run the test instances in parallel, using a thread pool, or maybe even a grid of multiple computers.

Browsing through the JUnit sources, I found a surprisingly easy way of parallelizing the tests with a thread pool, simply by using a custom runner:

@RunWith(Parallelized.class)
public class FeatureTest
{
   // same class body as above
}

All you need is a simple extension of the Parameterized runner:

public class Parallelized extends Parameterized
{
    
    private static class ThreadPoolScheduler implements RunnerScheduler
    {
        private ExecutorService executor; 
        
        public ThreadPoolScheduler()
        {
            String threads = System.getProperty("junit.parallel.threads", "16");
            int numThreads = Integer.parseInt(threads);
            executor = Executors.newFixedThreadPool(numThreads);
        }
        
        @Override
        public void finished()
        {
            executor.shutdown();
            try
            {
                executor.awaitTermination(10, TimeUnit.MINUTES);
            }
            catch (InterruptedException exc)
            {
                throw new RuntimeException(exc);
            }
        }

        @Override
        public void schedule(Runnable childStatement)
        {
            executor.submit(childStatement);
        }
    }

    public Parallelized(Class klass) throws Throwable
    {
        super(klass);
        setScheduler(new ThreadPoolScheduler());
    }
}

The RunnerScheduler interface is fairly new in JUnit and marked as experimental. I discovered it in the current version JUnit 4.8.1 and found it missing in JUnit 4.4.0 which we have been using so far. RunnerScheduler is also available in JUnit 4.7.0, but I did not check if this is the earliest version.

Eclipse Forms and Data Binding

Our map compiler Anaconda reads all parameters and settings from a configuration file, which over time has evolved from a simple Java properties file to a not-so-simple XML file which is validated by an XML schema.

To access the configuration at runtime, we use Java XML Bindings generated from our schema by xjc in a straightforward manner, without any fancy customizations.

Using the XML editing support in Eclipse, it is very easy to edit and validate a configuration file, at least from a developer perspective. However, our customers will not be too happy about editing large XML files by hand, so the idea is to develop a form-based configuration editor for our RCP application Anaconda Workbench, similar to the manifest editor of Eclipse PDE or the POM editor of m2eclipse.

These editors are based on Eclipse UI forms, another layer on top of SWT and JFace, which is obviously powerful enough for complex tasks, as demonstrated by the above examples. Less obviously, it is rather poorly documented. The Eclipse online help has just 10 brief pages about UI Forms and the Javadocs which, as usual, are not very useful for getting started.

The Eclipe Rich Client Platform book also has not more than one page on UI Forms and a link to an online article from 2004. On the Eclipse site, there are two more recent articles

• Eclipse Forms: Rich UI for the Rich Client
• Eclipse Forms: New in 3.3

Looking for futher tutorials, I came across Marco van Meegen's critical review Eclipse Forms im Härtetest. I decided to make up my own mind, but after implementing a few examples with Eclipse Forms, I largely agree to his criticism: the API forces you to write lots of repetitive code and it is not easy to figure out how to wire up the different classes to do your job.

To alleviate the shortcomings of Eclipse Forms, Marco created yet another layer called RCPForms. So I gave it a try, and I found it a lot easier to use than working with UI Forms directly. I had to use the sources from the Subversion trunk at Sourceforge, the older tagged or released versions do not seem to work with Eclipse 3.5.

Eclipse forms are usually wrapped by a ManagedForm which manages the state of the form parts and the underlying data models.

The form parts and the models can vary independently, and one model can be shared by multiple form parts.

A managed form is dirty, when one of its parts is more recent than the underlying model. Conversely, it is stale, when a change in the underlying model is not yet reflected on the UI.

To handle this form lifecycle, RCPForms expects the model to support PropertyChangeListeners, which again requires you to add some boilerplate code to your model beans. For my example with a JAXB model, I managed to tweak xjc to generate the required listeners, which is to be discussed in detail in a separate article.

Here is a screenshot of a simple example:

The warnings result from validators on missing mandatory fields.

Eclipse RCP Application Branding

A while ago, I figured out how to use the branding features in Eclipse to make the About dialog in my Anaconda Workbench show all the included features with the appropriate icons and other information. This was based on Eclipse 3.4, and here is a screenshot of the results:



Clicking on Feature Details, you get a list of all branded features included in the product:


Not every feature is branded by default. You have to associate a branding plug-in to a given feature and include an about.ini file and an optional icon in the branding plug-in. I won't go into details for this Eclipse 3.4 example, seeing that things have changed for Eclipse 3.5, and the rest of this post relates to Eclipse 3.5.

What's new in 3.5

The Feature, Plug-ins and Configuration Details buttons have gone, now there is just a single Installation Details button. To see the installed features, click on one of the feature icons to open a dialog listing all features branded with the same icon.



(Actually, I'm not quite sure whether or not the icon name is the grouping criterion, but I guess so from my experiments.)

Selecting a feature from the list and clicking on Plug-in Details, you can display all plug-ins contained in the feature.

Things to note

Eclipse Help has just two rather brief topics on Product Branding and Customizing a product. Branding a product is straightforward if you use the Product Configuration Editor as explained in the first topic. Things get more complicated when your product contains multiple features or even third-party products.

For each feature to branded, you need a branding plug-in containing the following resources:

• about.ini
• about.properties (optional)
  
• a 32*32 icon

Make sure to include these resources in you build.properties so they will be exported to your plug-in JAR.

In about.ini, set the following two properties:

aboutText=Anaconda Workbench for Car Navigation Databases
featureImage=icons/anaconda32.png

You can use the optional about.properties to work with language dependent variables in about.ini. Set featureImage to the relative path of your icon resource.

By default, Eclipse assumes that the branding plugin for a feature com.acme.foo is also named com.acme.foo. If you want to assign a different name, you have to explicitly declare the branding plug-in in the feature editor.

I ran into a conflict between feature branding and product branding: My top-level product Anaconda Workbench contains another product udigLite, which in turn contains Geotools, Eclipse EMF and Eclipse RCP.

udigLite is a modified subset of uDig. uDig has a top-level feature named net.refractions.udig which a contains a branding plug-in of the same name with a product definition.

udigLite originally had a top-level feature net.refractions.udig.lite containing just a subset of the uDig plug-ins, including the net.refractions.udig plug-in which cannot be left out as it contains more than just branding information.

With this configuration, I ended up with two uDig features in the feature list: net.refractions.udig.lite as expected, and net.refractions.udig, although no such feature exists in my product.

It seems that a branding plug-in containing a product definition is automatically treated as a feature of the same name. I wonder if that's a bug or a feature (pun intended).

Anyway, to remove this duplicate, I dropped my net.refractions.udig.lite feature and changed the original net.refractions.udig feature to include just the required subset of plug-ins.

Another problem I ran into is bug 280186 or bug 289300, causing each feature to be listed twice in the exported product. To supress that, you can drop the org.eclipse.equinox.ds plug-in from your product, provided your application does not depend on Declarative Services.

Arrows with Styled Layer Descriptor

In our uDig plug-ins for the Anaconda Workbench, we use Styled Layer Descriptors (SLD) for the map styles.

One of the features we are currently working on is the direction attributes for one-way streets. Of course you want to visualize them on the map, but SLD does not directly support arrow decorations for lines.

I asked for advice on the uDig mailing list and got the answer in no time. You can either use an arrow character from a suitable font in a Text Symbolizer, or a clever combination of Line Symbolizers with different styles of dashed lines to emulate arrows.

Using an SLD snippet from the Geoserver blog, I got the following result:

[Sorry, screenshot deleted to avoid potential licence issues.]

Getting Started with OSGi Declarative Services

OSGi Declarative Services have been on my mental agenda for a while, and now I've started using them. I'm not going to write yet another tutorial on the subject, but there are some bits and pieces I could not find anywhere else that may be worth mentioning.

To get an overview of Declarative Services, have a look at Neil Bartlett's tutorial on EclipseZone, or at his more recent EclipseCon presentation. The former is based on Eclipse 3.2.2, the latter on the latest Eclipse 3.5 milestones, so there are some differences to the current release Eclipse 3.4.2. In particular, Eclipse 3.5 supports a newer version of the Declarative Services Specification (1.1 vs. 1.0 in Eclipse 3.4).

All of the following refers to Eclipse 3.4.2. For using Declarative Services with Equinox, you need the following bundles:

• org.eclipse.osgi
• org.eclipse.osgi.services
• org.eclipse.equinox.ds
• org.eclipse.equinox.util

The last two are not included in standard Eclipse distributions. Download them from the Equinox download area, or use the Eclipse update manager to install Equinox Bundles.

Follow the tutorials to implement a simple example. If you cannot figure out why your component does not start, enable console logging for the Service Component Runtime in your OSGi launcher by setting the following system properties.

• equinox.ds.debug=true
• equinox.ds.print=true

Make sure to include the OSGI-INF folder with your component descriptors into the binary build on the Build tab of the Eclipe manifest editor. This amounts to adding the folder to the bin.includes property in the build.properties.

Add a header Service-Component: OSGI-INF/components.xml to your manifest. (Choose any other filename if you like.)

Do not forget to specify the correct XML namespace http://www.osgi.org/xmlns/scr/v1.0.0 in your XML documents, or else your component definitions will not be recognized.

Seeing that Eclipse 3.4 has no graphical editors for the component descriptors, it is useful to install the Eclipse XML editor features and to work with XML validation on the component descriptors.

If your bundle contains more than one component, you can have one XML document per component and list them all in the manifest header, or you can have a single XML document containing all component descriptions, which is what I prefer.

To enable XML validation on a multi-component descriptor, write your own XML schema where the root element contains a sequence of 1..n component descriptors.

Here is an XML schema for this purpose:

<?xml version="1.0" encoding="UTF-8"?>
<schema xmlns="http://www.w3.org/2001/XMLSchema"
        xmlns:scr="http://www.osgi.org/xmlns/scr/v1.0.0" 
        targetNamespace="http://www.harmanbecker.com/anaconda/components" 
        xmlns:tns="http://www.example.org/components" 
        elementFormDefault="qualified">

   <import namespace="http://www.osgi.org/xmlns/scr/v1.0.0"/>

    <element name="components">
      <complexType>
        <sequence>        
          <element ref="scr:component" maxOccurs="unbounded"/>
        </sequence>
      </complexType>
    </element>
</schema>

Using this schema, a component descriptor for two components might look like this:

<?xml version="1.0" encoding="UTF-8"?>
<c:components xmlns:scr="http://www.osgi.org/xmlns/scr/v1.0.0"
      xmlns:c="http://www.harmanbecker.com/anaconda/components">


 <scr:component name="foo">              
   <implementation class="com.acme.foo.FooImpl"/>
   <service>
     <provide interface="com.acme.foo.Foo"/>
   </service>
 </scr:component>

 <scr:component name="bar">              
   <implementation class="com.acme.foo.BarImpl"/>
   <service>
     <provide interface="com.acme.bar.Bar"/>
   </service>
 </scr:component>

</c:components>

JDBC Drivers in OSGi

How do you create a JDBC connection for a given JDBC URL in an OSGi application? In particular, how do you avoid an explicit dependency of your application code on a given driver?

In a plain old classpath context, you would invoke DriverManager.getConnection(), maybe after loading the driver class using Class.forName(), if your driver does not support the Service Provider mechanism which is mandatory for JDBC 4.0.

The trouble is, even if your driver does provide the META-INF/services metadata, this does not work in OSGi, since DriverManager creates a number of class loader problems by using Class.forName() internally.

I'm working with a variant of the Zentus driver for SQLite, which I've modified slightly to build an OSGi bundle under Java 1.6. I added the service metadata, and indeed my application code now works without loading the driver via Class.forName() - as long as the code is running outside of OSGi.

There are the following issues:

1. The context class loader of the current thread is used to scan for META-INF/services resources and to load the referenced classes. This happens during initialization of DriverManager, so it will not catch resources from bundles which are not visible to the current context.
   


2. Additional drivers can register with DriverManager, they should do so in static initalization code, so that the driver gets registered when someone calls Class.forName("my.own.Driver").


3. DriverManager.getConnection() iterates over the registered drivers until it finds one that can process the given URL. Unfortunately, it does a fatal double check to see if the class of that driver is the same that would be obtained by the caller. This amounts to calling Class.forName("some.matching.Driver", ccl) where ccl is the caller's class loader.

(To get some information on what is happening, it is useful to call DriverManager.setLogStream(System.out).)

The third point means that DriverManager.getConnection() will always fail if the driver class is not visible to your bundle, so there really is no way to avoid a dependency.

Here is an outline of a partial solution:

• Create an OSGi bundle for each JDBC driver you want to use. Add a bundle activator that calls Class.forName("some.jdbc.Driver"). This will register the driver with DriverManager when the bundle is started.
• For each bundle that needs to create connections using DriverManager.getConnection(), add optional dependencies on the JDBC drivers packages for all drivers that you are planning to use.

This is rather ugly, because you need to edit your client bundle manifest whenever you want to add support for an additional driver. But at least you can deploy your system with any subset of the defined drivers, maybe with just a single driver.

If you do not need to support legacy code, do not bother using DriverManager and the Service Provider mechanism. After all, this is just a poor man's service registry, and you are much better off using the OSGi Service Registry.

I'll give an example in one of my next posts.

Maven Plugin for Eclipse Source Bundles

The Maven Source Plugin creates a JAR containing the sources of a project and attaches it to the main artifact of the project.

It would be nice if you could directly use this JAR as source attachment for a bundle in your Eclipse target platform, but Eclipse 3.4 expects a special Source Bundle format with specific headers in the manifest. There is a feature request to support this directly in the Maven Source Plugin.

To fill this gap, I had a look at the sources of the Maven Source and Archiver plugins and wrote my own maven-sourcebundle-plugin. This plugin is available for public use as a subproject of the DataScript project hosted at BerliOS.

There is no public Maven repository for this project yet, but it is contained in the latest binary release rds-bin-0.30, or you can get the sources from Subversion.

3D Rendering in uDig

I've started experimenting with Java3D rendering in uDig. The goal is to use Java3D as a rendering engine first for plain old 2D data, and then for real 3D data in the next step.

More details can be found in the uDig wiki.