Around the World in Java

Integration Testing for JBoss AS7, Tomcat and Weld SE

2012-01-25T22:17:00.000+01:00

Initially, the jeeunit Integration Testing framework was exclusively focused on Java EE 6, supporting GlassFish 3.x, which was the only Java EE 6 compliant server at that time. Support for Resin 4.x has been added in subsequent releases.

With the current release 0.9.1, jeeunit supports JBoss AS 7, as well as alternative containers and injection methods beyond the scope of Java EE 6. You can now run jeeunit tests on Tomcat 6 and 7 and Weld SE containers.

Tomcat can be combined either with Spring 3.1 or with CDI (Weld Servlet) to inject dependencies into jeeunit tests.

And there's more to come: While jeeunit will continue a life of its own for a while, I'm planning to merge it step by step into Pax Exam, the OSGi testing framework of the OPS4J community.

Pax Exam and jeeunit both implement an in-container testing approach - while Pax Exam focuses on OSGi alone, jeeunit now supports various other containers, but no OSGi at all.

The Pax Exam/jeeunit merger opens interesting perspectives for hybrid applications, i.e. enterprise applications composed of traditional WARs and OSGi bundles. GlassFish 3.x supports this hybrid application model, implementing a subset of the OSGi Enterprise specifications.

For Pax Exam, the road towards the next major release 3.0.0 will be marked by a sequence of milestone releases, each of which is to incoporate a new container adapted from jeeunit.

A Pax Exam GlassFish Test Container is the goal of the first proof-of-concept milestone 3.0.0.M1. This is work in progress on a dedicated branch exam3-milestones in the Pax Exam GitHub repository.

Stay tuned...

Troubleshooting VisualVM Remote Connections

2012-01-03T19:21:00.000+01:00

I was trying to use VisualVM to monitor the memory usage of a build job on our build server. According to the documentation, this should require nothing but a jstatd daemon running on the remote machine.

For some reason, I could not see any remote applications in my local VisualVM. It was not a port issue, all the required ports were reachable via telnet.

Googling around, this article finally gave my the important clue: Our network does not have consistent DNS names, so I had to connect to the remote machine via IP address. Setting the java.rmi.server.hostname property for jstatd to the IP address solved the problem.

Summary

On the remote machine

Create policy file for jstatd

grant codebase "file:${java.home}/../lib/tools.jar" {
    permission java.security.AllPermission;
};

Start jstatd with this hostname file and the IP address as hostname

jstatd -J-Djava.security.policy=jstatd.all.policy -J-Djava.rmi.server.hostname=192.168.0.123

On the local machine

Start jvisualvm and add a Remote Host with the given IP address. A jstatd connection will be established by default. You should now see your remote applications.

Spring Integration Tests with Real Transactions

2011-12-21T11:01:00.000+01:00

Spring's Test Context has a @Transactional annotation for wrapping tests in a transaction started and rolled back by the test container, to keep the tests from modifying the database. This is just what you need for deterministic, repeatable database tests.

On the other hand, due to this approach, the transaction boundaries in your test system differ from the ones in your production system, which can lead to errors in production which were never noticed in your tests suites.

I wouldn't go as far as saying that transactional tests should be considered harmful, but at least you should be aware of the side effects which may or may not be harmless in your specific use case.

For tests with real transactions, you need to take care of cleaning up the database yourself.

JUnit rules are a neat way of doing this. Here is an example:

Using the CleanDatabaseRule

@RunWith(SpringJUnit4ClassRunner.class) 
@ContextConfiguration("/META-INF/spring/test-context.xml") 
public class RealTransactionTest { 

    @Rule 
    @Inject 
    public CleanDatabaseRule cleanDatabase; 

    @Inject
    private UserDao userDao;

    @Test 
    public void testFindUsers() { 
        userDao.createUsers("wilma", secret); 
        assertThat(userDao.findAllUsers().size(), is(1));
    } 
}

This test class has the usual @RunWith and @ContextConfiguration annotations, to enable the Spring Test Context, but the @Transactional annotation is missing

The CleanDatabaseRule is marked with @Rule. JUnit requires all rule members to be public. Given that our rule internally works with a persistence unit to be injected by Spring, the rule itself is configured as a singleton Spring bean to be injected into our test class.

This rule takes care of deleting all database content before and after each test method.

CleanDatabaseRule Implementation

Our CleanDatabaseRule extends JUnit's ExternalResourceRule which has before() and after() methods for dealing with a given external resource (the database in our case) before and after running a test method.

The level of cleanup performed by this rule is up to you: you can drop the database and create a new one, you can restore a given database dump, or simply truncate all tables, which is what I'm doing here.

The following example uses MySQL syntax to disable all foreign keys before deleting the tables:

public class CleanDatabaseRule extends ExternalResource { 

    @PersistenceUnit 
    private EntityManagerFactory emf; 


    @Override 
    protected void before() { 
        EntityManager em = emf.createEntityManager(); 
        em.getTransaction().begin(); 

        // disable foreign keys
        em.createNativeQuery("SET FOREIGN_KEY_CHECKS = 0").executeUpdate(); 
        truncateTables(em); 

        // reenable foreign keys 
        em.createNativeQuery("SET FOREIGN_KEY_CHECKS = 1").executeUpdate(); 

        em.getTransaction().commit(); 
   }

    @Override 
    protected void after() { 
        before();
    }
}

Note that we're injecting an EntityManagerFactory via @PersistenceUnit instead of the usual EntityManager This is because the entity managers injected by Spring do not permit manual transactions. Using an EntityManagerFactory, we can create our own EntityManager and control or own transactions.

In a database schema with lots of foreign key constraints, it is useful to disable them while cleaning up, to avoid the headache of having to delete tables in the correct order.

The truncateTables() method is left as an exercise. :-)

XVisitor: Visitor Pattern for JAXB

2011-12-02T19:23:00.001+01:00

XVisitor is a plugin for the xjc code generator to enrich the JAXB model generated from an XML schema so that users can work with the Visitor pattern on this model.

Working with large XML documents is a significant part of my job at the moment, and I've found that both JAXB and the Visitor pattern provide useful type-safe abstractions compared to working with DOM and XPath only. This was the motivation for creating XVisitor as an open source project with an Apache License.

XVisitor is hosted by the OPS4J community, which offers first class infrastructure for Apache-licensed Java projects, including Confluence, GitHub, Hudson, JIRA and Maven Central releases.

For more details about XVisitor, check out the wiki.

Eclipse Performance on KDE and Ubuntu 11.10

2011-11-16T20:51:00.001+01:00

If you suffer from poor graphics performance when using Eclipse under KDE, especially when scrolling large trees, check your GTK+ Style.

After changing the style from oxygen-gtk to QtCurve, my Eclipse is now running on steroids.

I'm using Eclipse Indigo 3.7.1 on Kubuntu 11.10 amd64 with Oracle JDK 1.6.0_26 and an NVIDIA GeForce 8400 GS graphics adapter. I never used to have any issues on Kubuntu 10.04 LTS which is still running smoothly on another partition of my disk.

I had suspected the NVIDIA drivers, tried different JREs (including 32 bit versions) and Eclipse releases, none of which made a difference.

Finally, changing various configuration options by trial and error, I found out that oxygen-gtk seemed to be the cause.

To optimize your installation, install the qtcurve package, and select it in K Menu | System Settings | Appearance | GTK+ Appearance | Widget Style. Then restart Eclipse and enjoy.

JBoss AS 7: Catching up with Java EE 6

2011-10-23T20:49:00.001+02:00

In my Java EE 6 server comparison of June 2011, JBoss AS 6.0.0 was not exactly the shining star, both in terms of performance and usability.

The next major release JBoss AS 7, claiming to be lightning fast, was published in July 2011, followed by two maintenance updates in August and September.

Time to take another look at JBoss and check if it now compares more favourably to other servers.

This post is based on JBoss AS 7.0.2 (Full Profile) and the same real-world application which I had ported from Spring to Java EE 6 for the GlassFish/Resin/JBoss AS 6 comparison.

With some minor modifications, I was able to deploy the same WAR on JBoss AS 7 and to repeat the measurements I did for JBoss AS 6.

Compared to JBoss AS 6 which did not even offer a Getting Started Guide, the documentation of JBoss AS 7 has improved a lot, but there are still quite a few gaps to be closed. At any rate I found sufficient input for my experiments in the following two documents:

Preparing the server

My application contains a JPA 2.0 persistence unit developed on Hibernate 3.6.x and MySQL 5.1, so first of all I had to configure the data source and the JDBC driver.

To do so, I added this section to the main configuration file of JBoss AS 7 in standalone/configuration/standalone.xml:

<datasource jndi-name="jdbc/myapp" pool-name="myapp" enabled="true" jta="true" use-java-context="true" use-ccm="true">
    <connection-url>
        jdbc:mysql://localhost/myapp
    </connection-url>
    <driver-class>
        com.mysql.jdbc.Driver
    </driver-class>
    <driver>
        mysql-connector-java-5.1.13.jar
    </driver>
    <transaction-isolation>
        TRANSACTION_READ_COMMITTED
    </transaction-isolation>
    <pool>
        <min-pool-size>
            4
        </min-pool-size>
        <max-pool-size>
            30
        </max-pool-size>
        <prefill>
            true
        </prefill>
        <use-strict-min>
            false
        </use-strict-min>
        <flush-strategy>
            FailingConnectionOnly
        </flush-strategy>
    </pool>
    <security>
        <user-name>
            MYUSER
        </user-name>
        <password>
            SECRET
        </password>
    </security>
</datasource>
<drivers>
    <driver name="mysql-connector-java-5.1.13.jar" module="com.mysql"/>
</drivers>

Then I copied both the JDBC driver mysql-connector-java-5.1.13.jar and my application myapp.war to standalone/deployments/ and started the server via

bin/standalone.sh

Troubleshooting

Of course, this did not work out of the box. The first exception I had was

java.lang.ClassCastException: org.dom4j.DocumentFactory cannot be cast to org.dom4j.DocumentFactory

This looked like a classloader conflict for dom4j. My WAR has a dom4j.jar in its WEB-INF/lib, but dom4j is also provided by JBoss as an transitive dependency of Hibernate. To resolve the conflict, I rebuilt my WAR without dom4j.

The next exception was

java.lang.ClassNotFoundException: com.sun.xml.bind.Locatable

My original application included jaxb-impl-2.2.2.jar, which I'd had to remove for JBoss AS 6, so I re-added this dependency to my WAR and redeployed it.

JBoss AS 7 marks any failed deployment with a foo.war.failed file in the deployment directory and will not attempt to redeploy the application until the marker file is deleted.

After restarting the server, the next problem was a

org.jboss.weld.exceptions.DeploymentException: WELD-001408 Unsatisfied dependencies for type [MyType] with qualifiers [@Default] 
at injection point [[field] @Inject private MyClient.myType]

This was confusing. I had beans.xml files in all my application modules, and the same WAR did not cause any CDI exceptions on other servers. So I enabled DEBUG logging for Weld to find out what was going on by adding

<logger category="org.jboss.weld">
    <level name="DEBUG"/>
</logger>

to the logging section of standalone.xml. This did not have the desired effect. I also had to change the

<level name="INFO"/>

child elements of the console-handler and the periodic-rotating-file-handler from INFO to ALL. (The Logging Configuration documentation does not mention this, but some additional information is hidden in the Getting Started Guide, section Configure Logging in JBoss Application Server 7.)

Now the DEBUG output from Weld revealed the cause of the problem:

17:33:39,799 DEBUG [org.jboss.weld.ClassLoading] (MSC service thread 1-8) WELD-000119 Not generating any bean definitions 
from MyType because of underlying class loading error
Caused by: java.lang.NoClassDefFoundError: Lorg/dom4j/Document;

Strange, dom4j had caused a classloader conflict at first, and now it was no longer visible at all? I had a closer look at Class Loading in AS7 to understand the background of this.

In short, JBoss AS 7 has a module structure, and server modules are not visible to applications by default, unless they are imported or exported explicitly.

Both dom4j and jaxb-impl are JBoss modules, so I re-deleted jaxb-impl and jaxb-api from my application and made the JBoss modules globally visible by adding the following to standalone.xml:

<subsystem xmlns="urn:jboss:domain:ee:1.0">
    <global-modules>
        <module name="org.dom4j" slot="main"/>
        <module name="com.sun.xml.bind" slot="main"/>
    </global-modules>
</subsystem>

(Global visibility is not the best solution, of course, but for this experiment I would rather modify the server configuration than add JBoss-specific manifest headers or deployment descriptors to my application WAR.)

Finally, I had another ClassNotFoundException for an internal Xerces class from the JDK required by my application. (Yes, such a dependency is bad practice, but as I said above, I'm using a real-world application for these tests...)

I made this class visible to my application by adding

<path name="com/sun/org/apache/xerces/internal/impl/io"/>

to modules/sun/jdk/main/module.xml - no idea if you are supposed to do it this way, but it works for me...

Now at last I was able to deploy and run my application and take some measurements.

Performance measurements

The following table compares some performance indicators for JBoss AS 7.0.2, JBoss AS 6.0.0 and GlassFish 3.1.1. For JBoss AS 6, I did not repeat the measurements but simply copied the results of June 2011. For GlassFish, I had been working with a pre-release build at that time, so I've now re-run my tests with the official GlassFish 3.1.1 release.

	JBoss AS 7.0.2	JBoss AS 6.0.0	GlassFish 3.1.1
Empty server startup time	1.9 s	12 s	3.2 s
Empty server heap memory	10.5 MB	100 MB	26.5 MB
Empty server PermGen memory	36.3 MB	70 MB	28.4 MB
MyApp deployment time	5.8 s	47 s	13 s
Server + MyApp restart time	8 s	30 s	14.5 s
Server + MyApp heap memory	52.8 MB	236 MB	55.3 MB
Server + MyApp PermGen memory	80.9 MB	175 MB	84.5 MB
MyApp redeployment time	3.5 s	30 s	7 s

All in all, there is an amazing improvement of performance and memory footprint in JBoss AS 7, compared to JBoss AS 6. JBoss AS 7 is now at a competitive level with Resin and Glassfish and actually outperforms Glassfish in almost all of these tests.

I don't know if this indeed qualifies as lightning fast, but it is now safe to say that JBoss AS 6 was blatantly fat and slow, as confirmed by its own successor.

Redeployment

I tested redeploying my application by simply touching the WAR file in the deployments directory and watching the memory usage via jvisualvm.

I could see the PermGen usage grow at a constant rate, and after a couple of redeployments, there was an OutOfMemoryError, so JBoss AS 7 appears to have a new classloader leak. (The same test did pass successfully on JBoss AS 6.)

Eclipse Integration

In contrast to the drastic improvements of the server itself, I could find no significant progress with the Eclipse integration. I installed the JBoss AS Tools from JBoss Tools 3.3.0.M3 into a fresh copy of Eclipse Indigo 3.7.1 with m2e 1.0 and m2e-wtp 0.14.0.

I was able to deploy my application directly from the workspace, but editing and saving a source file did not redeploy the application. An Incremental Publish did not help either. Only a Full Publish was sufficient to redeploy the application.

This is a major productivity issue for Eclipse users. Redeploy-on-save works smoothly with the built-in Tomcat integration of Eclipse WTP, so something must be broken in JBoss AS Tools.

JPA Providers

JBoss AS 7.0.2 comes bundled with Hibernate 4.0.0.CR2. This is a bad smell - an official product release should never contain any release candidates of upstream components, especially not for components originating from the same company.

The good news is that my application which was developed on Hibernate 3.6.0 works without problems on this Hibernate 4.x release candidate.

But the real bad news is that JBoss AS 7 does not currently support other persistence providers like Eclipselink, OpenJPA or DataNucleus (according to the JPA Reference Guide). With GlassFish and Resin, you can simply drop the JARs of your preferred provider and its dependencies in a designated folder of your server installation and edit your persistence.xml to override the default provider of the server.

JBoss AS 7 appears to require an adapter per persistence provider, which to me looks like an unfortunate and unnecessary design decision.

At any rate, this lack of flexibility is a severe restriction. In most of my projects, I had to replace Hibernate by OpenJPA due to a number of bugs related to new JPA 2.0 features.

Conclusion

Given the drastic performance improvements in JBoss AS 7, which was released only six months and a half after JBoss AS 6, I can only wonder why RedHat and the JBoss community ever spent any resources on an AS 6 release which clearly was not competitive, instead of focusing on AS 7.

Anybody looking for a production-quality Java EE 6 server in the first half of 2011 was better off with GlassFish 3.1.1 or one of its pre-releases.

With AS 7, JBoss is now back in the game, it is surprisingly lean and fast, and it even has the potential to take over the lead from GlassFish.

But performance is not the only factor that counts. The documentation continues to be sketchy and far below the standard of JBoss AS 5.

Different projects have different needs, but for me, the top-level performance of JBoss AS 7 is more than outweighed by classloader leaks, productivity issues of the Eclipse integration and lack of support for JPA providers other than Hibernate.

Each of these is currently a blocker for using JBoss AS 7 in production. GlassFish 3.x has taken more than a year to reach production quality. Let's see if JBoss AS 7.x can do the same lightning fast.

Eclipse Maven Integration Extensions

2011-08-29T23:41:00.000+02:00

For me, the most important new feature in Eclipse 3.7.0 (Indigo) is the improved Maven Integration, formerly developed by Sonatype under the name of m2eclipse, now an official Eclipse subproject under the name of m2e.

m2eclipse and Eclipse sometimes used to have different opinions on what was going on in the workspace, and so we had to go through series of refresh/update dependencies/update configuration/rebuild voodoo (or "m2eclipse dance" as some called it) to get projects in a good state. (Quote from the m2e Wiki).

m2e has a different approach. First of all, it complains when detecting a Maven plugin in one of your POMs that it cannot handle out of the box. m2e flags your POM with an error marker that will not go away until you tell m2e what do to about the unknown plugin. The easy way out is a Quick Fix to ignore the plugin, which means you'll have to run the corresponding Maven goal manually when you need it.

However, when the Maven plugin in question generates source code or post-processes byte code, this is usually not sufficient - ideally, Eclipse and m2e should pick up the required build steps from your POM automatically.

This issue is addressed by m2e extensions, which provide the missing link between Eclipse project builders and Maven plugins. An m2e extension for a given Maven plugin not only invokes the required mojos but also informs Eclipse about new source or binary folders created by the mojos, so that Eclipse and Maven can stay in sync.

My projects use a number of Maven plugins that are not supported by m2e out of the box, e.g. for JAXB and OpenJPA, so I started creating some custom extensions for these plugins, based on a GitHub fork of the official m2e extensions from Sonatype.

See the wiki page at GitHub for more details. The GitHub project also contains an Eclipse update site (aka p2 repository) for installing the plugins. Time permitting, I'll try to make these extensions available via the Eclipse marketplace.

For any fixes, additions or new extensions, feel free to send my pull requests.

Wicket and OSGi

2011-06-25T19:43:00.002+02:00

After playing around with the OSGi JPA and JTA components of Apache Aries for a while, it was only natural to try and add a web layer to my sample projects. The Aries samples all use plain old servlets and JSPs, so I wondered if Wicket would work in this context.

So far, I had only ever used Wicket in standard Java EE web applications, but I had noticed the Wicket JARs come with an OSGi manifest, so someone must have tried to use Wicket in OSGi before.

Then I noticed the ominous

DynamicImport-Package: *

manifest header, which is not the best way of interacting with client code, and what's worse, Wicket does not even import its own static dependencies like org.slf4j, but relies on the catch-all dynamic import.

A discussion on the Wicket Users mailing list confirmed my suspicion that this had to be related to Wicket's page deserialization requiring to load classes from user bundles.

Anyway, I had a feeling it shouldn't be too hard to make this work without dynamic imports. In addition, it would be nice to inject OSGi services into Wicket components, just like CDI or Spring beans.

So I started coding, and by now, there is a first proof-of-concept sample using Wicket on top of Aries, Jetty, OpenJPA, Derby and Equinox. The sample code is included in my OSGi Enterprise sandbox project at Google Code. More details can be found in the OSGi Enterprise wiki.

Update 10 Jul 2011: After some polishing, my Wicket/OSGi glue code has made it into the Wicketstuff project - see the OSGi Integration wiki page for more details. This subproject targets the Wicket 1.5 release and will automatically be integrated with the next release or release candidate of Wicketstuff.

Snapshot artifacts are currently available from the Sonatype OSS repository.

The first shot in the OSGi Enterprise project works with Wicket 1.4.x.

wicket-osgi from Wicketstuff cannot be used with Wicket 1.4.x.

Java EE 6 Server Comparison: Conclusion

2011-06-07T18:58:00.001+02:00

Lessons Learned

Portability is not just a Java EE marketing promise. My application now runs on three different Java EE 6 servers, using the same WAR file, without any compile-time configuration.

An application is not portable until it has been ported. Of course, this is true for any specification with different implementations, not just for Java EE. Each of the three servers helped me discovering incorrect API usage in my application that just happened to work on another server.

Certification is not reliable. My application hit at least one major specification violation in each of the three Java EE 6 certified servers. Thus, the test coverage in the Java EE TCKs must be insufficient.

A certification process for an open specification should be open. Implementations of the specification may be closed source, but the test suites (TCKs) should be open source, and the user community should have the chance to contribute new test cases.

The next version of the Java Community Process is moving in the right direction, but if you read the proposal carefully, you will notice a lot of half-hearted may's and should's, which is not enough. TCKs shall be open, period.

(Note: Most of the JCP site is currently unavailable, due to restructuring - I read JSR 348 a few days ago. Oracle, did you ever hear about high availability and Cool URLs don't change?)

And the winner is...

Each of the three servers has its strong points and its weaknesses.

Only JBoss has an official release that is able to run my application (with a workaround). On the other hand, its memory footprint is enormous, the deployment speed is inacceptable and the weakest point is the lack of up-to-date documentation. The Eclipse integration is severely limited in not allowing to republish applications.

Resin has the smallest footprint by far, clearly in terms of memory, less clearly in terms of deployment speed, where Glassfish is close up and might scale better than Resin for larger applications due to Resin's code generation during deployment. Resin needs a few more releases to mature. Its Eclipse integration is too unstable for serious use.

Glassfish is the leader in end-to-end usability. Its ease of use and its excellent documentation are invaluable for promoting the benefits of Java EE 6, keeping the barrier low for newcomers.

Its speed is excellent (though outrivalled by Resin), its memory footprint needs to be improved, but is still a lot smaller compared to JBoss. Important to note (though without any impact on my experiments) is the fact that Glassfish is the only one of the three certified for the Java EE 6 Full Profile.

Compared to JBoss and Resin, Glassfish has the best Eclipse integration (but the standard Eclipse WTP Tomcat integration is still better).

Glassfish needs to shift the balance from new features to stability. There has been no production quality 3.x release so far. Glassfish 3.1.1 is on its way, it has a chance of becoming the first really stable release, and I'd rather wait for it a little longer to make sure that all major known bugs are fixed.

Glassfish would benefit from quarterly maintenance releases and more transparency in community communications.

All in all, if you ask me

Which server would you choose for an application to go live next week?
Which server would you choose for developing a new project to go live next year?

then my answer is Glassfish for both. And if I couldn't use Glassfish, I would choose JBoss for next week and Resin for next year.

Executive Summary

Glassfish is Business Class, JBoss is Baroque, Resin is Zen.

Read the whole story

Java EE 6 Server Comparison: Resin

2011-06-07T18:53:00.001+02:00

The first time I read about Caucho Resin 4.x was last summer, when I was looking for alternative CDI implementations, i.e. other than Weld. Resin has its own CDI implementation named CanDI, and I was bold enough to give it a try, knowing that Resin 4.x was not yet Java EE 6 certified and probably not stable enough.

I didn't get very far at that time, CanDI looked ok, but the JPA API was broken and @Singleton @Startup beans did not work, so I left it at that and made a mental note to check back after the first stable release.

Now that Resin 4.0.17 passed the Java EE 6 Web Profile certification in May 2011, it's a good time to give it another try.

Considering that Caucho focuses on the Web Profile and has no intention to support the Full Profile, you'd expect Resin to be more lightweight than Glassfish or JBoss. In fact, Resin is amazingly small and easy to use, the perfect example to destroy common belief that Java Enterprise servers are heavyweight and slow.

The Resin download package has 23 MB (compared to 44 MB for Glassfish Web Profile, 78 MB for Glassfish Full Profile, and 173 MB for JBoss).

Resin has a very small number of third party dependencies. The largest part of Resin is a single resin.jar of less than 11 MB, the rest is Eclipselink (JPA), Mojarra (JSF), Java Mail and a couple of Java EE API JARs.

This fact is worth mentioning, not just as an achievement of the Resin development team, but also as a benefit for the application developer: The risk of a version conflict for a library used both by the application and the server is almost zero, given that Resin has little or no external dependencies.

Preparing the server

To configure my data source, I added the MySQL JDBC driver to Resin's lib directory and then added the following section to conf/resin.xml:

<database>
    <jndi-name>jdbc/tcm</jndi-name>
    <driver type="com.mysql.jdbc.jdbc2.optional.MysqlConnectionPoolDataSource">
      <url>jdbc:mysql://localhost/myapp</url> 
      <user>myuser</user>
      <password>****</password>
    </driver>
    <prepared-statement-cache-size>8</prepared-statement-cache-size>
    <max-connections>20</max-connections>
    <max-idle-time>30s</max-idle-time>
  </database>

Then I started the server from the command line:

bin/resin.sh console

and copied my WAR to the webapps directory for deployment.

Troubleshooting

Resin's approach to proxy creation for EJBs and CDI beans is Java code generation and compilation at deployment time. In the current release 4.0.18, the EJB code generator is broken for some use cases with generic methods.

After I posted my problem in the Resin forum, the development team advised me to use the current snapshot which solved the problem at least partially, and for the remaining cases, I temporarily removed some generic parameters in a few source files of my application.

For the rest of my experiments, I used the pre-4.0.19 snaphot version labelled resin-4.0.s110531.

Besides the generic methods arguments, the code generator complained about a checked exception in a @PostConstruct method, but it was right in doing so: the PostConstruct Javadoc clearly states that methods with this annotation must not throw checked exceptions, so I wrapped my checked exception in an unchecked exception, and the problem was gone. (Glassfish and JBoss did not complain about the checked exception.)

The only remaining problem was a version conflict for the SLF4J API. Resin includes slf4j-1.6.1.jar in webapp-jars whereas my application requires 1.5.11. I deleted the 1.6.1 version from webapp-jars and added the Hibernate, Javassist and JAXB JARs I had needed to exclude from my WAR for JBoss.

Other problems like EL in JSPs not working turned out to be a side effect of a broken Eclipse plugin version.

Eclipse Integration

In fact, the Eclipse plugin was my biggest source of troubles with Resin. The current release 4.0.16 from Caucho's Update Site did not work at all in the default (i.e. WAR) deployment mode. After enabling directory-based deployment in the Server Runtime dialog, the plugin would deploy my application, but its update behaviour was a complete mystery to me. Sometimes it seemed to deploy out-dated versions of classes, including some classes I had already deleted.

The WAR deployment has been fixed in the current 4.0.19 snapshot version of the Eclipse plugin which can be installed from the Snapshot Update Site.

After deleting the 4.0.16 version and installing the 4.0.19 snapshot, the Eclipse plugin finally started to work more or less reliably in the default mode. The directory mode continues to be broken, causing all sorts of mysterious side effects.

With the default mode, there are still a number of issues:

Editing a source file of the top-level module redeploys the application as expected, editing a source of a dependent module does not.
Redeploying the application causes the EJB Proxies and JSPs to be regenerated and recompiled. For some reason, this process is much slower when running under the Eclipse plugin, compared to (re)deploying an application from the command line. Increasing MaxPermSize in the server launcher did not help.
After two or three redeployments from Eclipse, Resin dies with a PermGen size OutOfMemoryError.

From the command line, I was able to redeploy my application a couple of times without any apparent memory leaks.

Performance Measurements

Empty server startup time: 3.3 s
Empty server memory usage: 22 MB heap, 27 MB PermGen
MyApp deployment time: 8 s
Server + MyApp restart time: 9 s
Server + MyApp memory usage: 44 MB heap, 58 MB PermGen
MyApp redeployment time: 8 s
Server + MyApp startup time from Eclipse: 30 s
Server + MyApp memory usage from Eclipse: 245 MB heap, 124 MB PermGen
MyApp redeployment on Eclipse save: 25 s, heap reduced to 85 MB

It is strange that Resin works rather smoothly from the commandline and has no memory issues at all, while slowing down on code generation, using excessive amounts of memory and throwing PermGen exceptions in combination with the Eclipse plugin.

Summary

Resin is easy to use and incredibly lightweight. The documentation is rather terse and not always up-to-date, but all in all usable. The Resin Forum was extremely helpful, my questions were answered quickly and some of the bugs I found during this investigation have already been fixed by the Caucho team in the meantime.

It would be foolish to recommend any server for production use before having worked with it in a real project for at least a couple of months, but it is safe to say that Resin looks extremely promising and is fun to work with.

The sore point at the moment is the Eclipse integration, which is currently not stable enough for day-to-day development, so this is a real blocker, unless you are happy using command line tools, some other IDE, or maybe some other server for development and Resin for testing and production only.

Resin 4.0.x releases have been published once a month since February 2010, and if the team can keep up this pulse rate both for the server and the Eclipse plugin, I'm quite confident that Resin will be a first class Java EE 6 Web Profile platform within a few months.

Read the whole story

Java EE 6 Server Comparison: JBoss

2011-06-07T18:46:00.001+02:00

I never did any serious work with JBoss AS 5 or earlier versions. At the peak of my frustration with Glassfish last summer after the 3.0.1 release when CDI continued to be unusable, I knew that JBoss would be no help as it also included Weld, the main source of trouble. Besides, given the number of problems in released and certified Glassfish, I dreaded an even larger number of problems in an unreleased, uncertified and undocumented JBoss.

When JBoss AS 6.0.0 was finally released late in December 2010, with surprisingly little marketing noise and community echo, I briefly had a look at the release and was disappointed to find out that it was still largely undocumented.

Five months later, there has been no maintenance release, and the documentation is not up-to-date, even the Getting Started Guide still refers to JBoss AS 5.

Anyway, having read Arun Gupta's and Dave Johnson's summaries of their first steps with JBoss, I felt should be able to come to grips with it.

Preparing the server

I copied the MySQL JDBC driver to server/default/lib. To configure my datasource, I copied docs/examples/jca/mysql-xa-ds.xml to server/default/deploy and edited the copy. The most important and non-obvious changes are

<jndi-name>jdbc/myapp</jndi-name>
<use-java-context>false</use-java-context>

All the JBoss datasource examples use a simple JNDI name foo and then include a reference

<jta-data-source>java:/foo</jta-data-source>

in META-INF/persistence.xml. The problem is how to configure a data source with a cross-platform JNDI name that will work on Glassfish, JBoss and Resin alike.

I tried using a resource-ref-name indirection in my web.xml combined with a container specific deployment descriptor, without success. Finally, I found this simple solution with the use-java-context tag in this blog comment.

Running the application

To run my application on JBoss, I copied my WAR to server/default/deploy and then started the server from the command line:

bin/run.sh

Troubleshooting

At first, I all got was a flood of error messages. which I found hard to interpret. For every problem I ran into, it was easy to find similar questions in the JBoss forums or on stackoverflow, but most of the time either without adequate answers, or with solutions referring to JBoss 4 or 5 that did not seem to apply to JBoss 6.

The situation was not helped by the extreme verbosity of the console log messages, and by a sluggish startup. Before starting to deploy my application, or more likely, while scanning the WAR, JBoss takes a deep thought for about 20 seconds with CPU load > 100 %, printing no message whatsoever.

Anyway, most of the problems turned out to be caused by classloader conflicts, which I could solve by excluding a number of libraries from my WAR, but there remained an issue with JNDI lookup exceptions, caused by a known bug in JBoss: Injecting EJBs into the web layer with @Inject results in JNDI lookup exceptions, since JBoss starts the web container before the EJB container by default.

I could not use the producer method workaround mentioned in the JIRA issue, since my web component uses the CDI-to-Spring bridge and is not itself a CDI bean archive. My solution was to create a ServletContextListener class which injects all required session beans with an old-school @EJB annotation. This listener is configured to run before Spring's ContextLoaderListener.

JBoss also complained about an invalid XML namespace definition in my web.xml, and it was right, the namespace was wrong - Glassfish and Resin did not seem to care.

Classloader issues

Unlike Glassfish and Resin, JBoss adopts the Servlet Specification recommendation and does not use a parent-first strategy for class lookups. As a result, there were lots of ClassCastExceptions for org.hibernate.* or javax.* classes included in my WAR. To get rid of these, I excluded all Hibernate, JAXB and Geronimo Spec libraries from my WAR by changing the scope in the Maven POM to provided.

As an alternative, I tried to force JBoss to do a parent-first lookup. The Administration and Configuration Guide does not cover the subject, but following the instructions in an article on JBoss class loading, I added a configuration file WEB-INF/jboss-classloading.xml to my WAR:

<classloading xmlns="urn:jboss:classloading:1.0"
       domain="myapp.war"
       parent-first="true">
</classloading>

This indeed had an effect on classloading, but only produced a new exception

java.lang.NoSuchMethodError: org.objectweb.asm.ClassWriter.(I)V

caused by an older version of asm.jar used by JBoss which is incompatible to the version 3.1 required by my application.

I did not try to upgrade the asm.jar in the JBoss installation for fear of breaking the server, so I ended up excluding the relevant libraries from my WAR, and consequently I had to copy most of these libraries to the appropriate server library directory in Glassfish and Resin to make my WAR portable.

Later, trying to redeploy my application, simply by running

touch server/default/deploy/myapp.war

I ran into OutOfMemoryErrors. A heap dump confirmed my suspicion that some classloaders did not get garbage collected. Taking a closer look with the Eclipse Memory Analyzer, I found a class org.jboss.interceptor.util.InterceptionTypeRegistry holding an indirect reference to an obsolete class loader.

Taking another look at my WAR, I noticed a jboss-interceptor.jar including the javax.interceptor package, a transitive dependency of the CDI-Spring-Bridge. After excluding this library from my WAR, I still had an OutOfMemoryError on redeployment, but I could see in jvisualvm that now at least some classes had been unloaded, so I edited bin/run.conf to increase MaxPermSize to 384MB and the heap size to 1GB. With this modification, I was able to redeploy my application multiple times in a row without any apparent classloader leaks.

So while it is certainly not recommended to include another copy of javax.interceptor in your WAR, this still leaves the question why JBoss loads classes from this package via the WAR classloader and not via its container classloader.

The Web Profile

Unlike Glassfish with its two separate distributions for the Full Profile and the Web Profile, JBoss offers a single distribution including different server configurations, which can be selected by a command line option of the startup script.

JBoss comes with 5 different server configurations. Surprisingly, none of them exactly matches the Web Profile. There is a configuration called jbossweb-standalone which launches a web container but does not include CDI support.

I used the default configuration to launch my application. The default configuration includes the Web Profile but also other services like JMS which are not part of the Web Profile.

See this thread in the JBoss Forum for a more detailed discussion.

Having the JBoss command line options in mind, I once suggested in the Glassfish forum to let users select the desired profile in a similar way. It is now somewhat ironic to find out that JBoss doesn't really let you do that.

Eclipse Integration

Quite some time ago, I made the mistake of installing the entire JBoss Tools set into what must have been Eclipse Galileo, only to find my workspace cluttered almost beyond recognition by tools and menus I had never intended to use.

So this time I took care to only install JBoss AS Tools 2.2.0 from the JBoss Helios Update Site. I was confused (but no longer surprised) by the feature description "Supports JBoss AS 4.x and 5.x". The plugin does support JBoss AS 6 also, at least to some extent.

I was able to launch my application from the workspace using Run on Server. The startup time was between 45-50 s, just the same as from the command line. There was no big difference in the heap size either.

Editing and saving a source file of my top level project should have triggered a redeploy, but didn't. In the Server view, my application still had the status Synchronized. Clicking the Publish button had no effect. Even cleaning the top-level web project did not cause the application to be redeployed.

I also tried reloading a page of my application in the browser, hoping that some hot swap magic might have updated my application without redeploying it.

But no. Stopping and restarting the server seems to be the only way of publishing the current version from Eclipse, so this is at least a critical bug, if not a blocker.

Performance Measurements

Empty server startup time: 12 s
Empty server memory usage: 100 MB heap, 70 MB PermGen
MyApp deployment time: 30 s
Server + MyApp restart time: 47 s
Server + MyApp memory usage: 236 MB heap, 175 MB PermGen
MyApp redeployment time: 30 s
Server + MyApp startup time from Eclipse: 47 s
Server + MyApp memory usage from Eclipse: 258 MB heap, 181 MB PermGen
MyApp redeployment on Eclipse save: failed

Summary

JBoss has taken a long time to finally release a Java EE 6 server with JBoss AS 6.0.0, but quality is worth waiting for, and the server itself appears mature and stable.

This is really the best thing I can say about JBoss. Otherwise, JBoss is too fat, too slow, too verbose, poorly documented and simply not enjoyable to work with.

While Red Hat have pointed out that their business focus is on JBoss AS 7, whereas JBoss AS 6 is "just" a Community release, I'm still somewhat surprised that the JBoss community do not promote their server more actively.

Compared to the first class professional documentation of Glassfish and a whole lot of tutorials and videos on almost each and every aspect of Glassfish, Red Hat and/or the JBoss Community do not appear to have a very active interest in widening their audience, or motivating Glassfish users to give JBoss a try.

For experienced JBoss AS 5 users, upgrading to JBoss AS 6 may be the easiest way to start using Java EE 6 features without losing time on getting familiar with a new server like Glassfish or Resin.

For Java EE newcomers, starting on JBoss AS 6 is probably the hardest route to take, and frustration with the Java EE 6 server may quickly turn into frustration with Java EE 6 as such and may reinforce common prejudice.

The JBoss Eclipse integration is currently not usable for day-to-day development, unless you are ready to shutdown and restart the server instead of simply republishing your application. Even if republishing worked as expected, having to wait 40 seconds for a redeployment is no fun.

All in all, while JBoss may still be the world's number one Java application server in terms of installed instances, its glory seems to have faded. The boss is still around, but the juniors have taken over the business.

Read the whole story

Java EE 6 Server Comparison: Glassfish

2011-06-07T18:37:00.002+02:00

In April 2010, I started a Java EE 6 project, and Glassfish 3.0 was the obvious server choice, or rather, the only option at that time. For the next couple of months, I spent a considerable amount of time not working on my application, but analyzing, reporting and working around bugs in the server, mainly in the CDI and JPA areas. This appeared to be the price you had pay for being an early adopter of the new Java Enterprise standard.

There was an unbearably long gap between the 3.0.1 release in June 2010 and the 3.1 release in February 2011. Glassfish 3.0.1 was still buggy, and the only solution to get my project done on Glassfish 3.0.1 was to stop using CDI completely, reverting to Java EE 5 style @EJB and @Resource injections.

I was hoping for Glassfish 3.1 to be the first stable release without blocker bugs - but it was not.

Weld, the CDI reference implementation shared by Glassfish and JBoss, is still broken in Glassfish 3.1. The problem can be fixed by replacing weld-osgi-bundle.jar with a newer version, or by using a more recent promoted build of Glassfish.

So this article is based on Glassfish 3.1.1-b06 which includes Weld-1.1.1.Final and to my great relief, this version seems to fix all bugs and memory issues encountered in previous releases.

Glassfish has an extensive documentation set, and this blog already has a tutorial on setting up Eclipse and Glassfish, so I'll only briefly touch the basics in this article.

Preparing the Server

The first thing I do with every Glassfish installation is changing its logging configuration from JUL to SLF4J and Logback, since the default output is pretty unreadable and my application uses SLF4J anyway. Actually, my posts on this topic belong to the most popular articles on this blog, so it seems I'm not alone.

The next thing to do (with any server, not only Glassfish) is to copy the JDBC driver for your database to the appropriate location where the container classloader can pick it up before loading your web application. So I copied the MySQL driver to domains/domain1/lib.

After that I was ready to start Glassfish using

bin/asadmin start-domain -v

and to configure my data source using the Admin Console web interface at http://localhost:4848.

Selecting Resources | JDBC | JDBC Connection Pools, I created a MySQL connection pool and then mapped it to the JNDI name jdbc/myapp via Resources | JDBC | JDBC Resources.

Troubleshooting

At this point, I was ready to deploy my web app, either by uploading it via the Admin Console, or by copying it to domains/domain1/autodeploy, or directly from my Eclipse workspace using Run on Server (which of course requires installing the latest Glassfish plugin for Eclipse).

My WAR still included the full set of third-party transitive dependencies from the original Maven build for Tomcat and Spring, some of which are also used by Glassfish itself, with different versions, so I prepared myself for a number of classloader conflicts.

Actually, there were none. Glassfish uses the standard parent classloader delegation model by default, instead of following the recommendation of the Servlet Specification. (You can override the default by a configuration switch.) So even though my webapp included a whole lot of Java EE API JARs from org.apache.geronimo.specs and its own JAXB version, Glassfish would simply ignore the classes included in my WAR and load the ones from its own modules.

My WAR also included Hibernate and its dependencies. As my META-INF/persistence.xml did not explicitly define a persistence provider, Glassfish at first used its built-in provider Eclipselink. This was enough to exhibit a JPQL bug in my application. (Hibernate is known to accept broken JPQL syntax for backward compatibility with its own HQL language, and there is no configuration option to enforce standard compliance.) Eclipselink also complained about a missing SEQUENCE table in my database schema, so I left it at that and added

<provider>org.hibernate.ejb.HibernatePersistence</provider>

to my persistence.xml

After that, I could deploy my web app and open its start page in my browser. Only when undeploying or redeploying the application, there was an error message about an unloaded Hibernate class (which I cannot remember from Glassfish 3.0.1). Anyway, I knew that bundling the JPA provider in the WAR is not the recommended way, so I excluded Hibernate and its dependencies from my WAR and copied the corresponding JARs to domains/domain1/lib.

Finally, I had to solve a runtime issue caused by a version conflict in Woodstox. Glassfish includes Woodstox 3.2.1, whereas my application requires Woodstox 4.1.1 with some new parser options. To get on with my experiments, I simply deleted the missing option from my source code and filed an enhancement request for Glassfish to upgrade Woodstox in its 3.1.1 release.

Eclipse Integration

Complementing the broken 3.1 release, the Glassfish Eclipse plugin was also broken in its default configuration at that time, failing to deploy applications, but this could be worked around by disabling the directory deployment option in the plugin's Server configuration. The current version Oracle Glassfish Server Tools 1.7.1 (released in May 2011) works without problems.

A point to be improved is the publishing behaviour of the plugin. My application has some 30 Maven modules, represented as individual Eclipse projects. When the application is started via the Eclipse plugin, editing and saving a class from the top-level web project will publish and redeploy the application. Editing a class in any other module does not trigger a redeploy, even after clicking the Publish button. You have to clean the top-level project to force a redeploy.

Editing a JSP page or a CSS file in the top-level project also causes a redeploy. This could be optimized by simply publishing the changed resource to the running application.

Performance Measurements

Empty server startup time: 3.1 s
Empty server memory usage: 37 MB heap, 28 MB PermGen
MyApp deployment time: 13 s
Server + MyApp restart time: 15 s
Server + MyApp memory usage: 156 MB heap, 83 MB PermGen
MyApp redeployment time: 10 s
Server + MyApp startup time from Eclipse: 14 s
Server + MyApp memory usage from Eclipse: 170 MB heap, 89 MB PermGen
MyApp redeployment on Eclipse save: 10.5 s

Glassfish is really fast, but I'm not satisfied with its memory footprint, compared to the original Tomcat/Spring version of my application. This is a known issue which can be narrowed down to a single internal metadata cache. The Oracle team is currently working on this, and I'm hoping to see the memory footprint cut drastically in the forthcoming 3.1.1 release.

[Update 10 Jun 2011: Tim Quinn from Oracle committed some changes to reduce the memory footprint. I've retested my application with the nightly build glassfish-3.1.1-b07-06_09_2011, and the heap is now less than 70 MB. Great news!]

I have redeployed my application 20 times in a row, and there were no classloader or other memory leaks.

Summary

Glassfish is more than the Java EE 6 Reference Implementation, it is the reference for Java EE 6 servers. Of course, Glassfish is not perfect and it may be second to one or the other of its competitors in certain areas, but the total package of documentation, community support, first-class technology, and an ever growing number of tutorials, presentations and videos provides a very productive basis for Java Enterprise development.

Regarding the shared Sun/Oracle background, Glassfish may have some natural bias towards Netbeans, but nevertheless the Glassfish Eclipse plugin is stable and rich enough for professional use.

Most of the things that could be improved in and about Glassfish are organizational rather than technical aspects.

The most pressing issue is release management. I cannot help feeling that the 3.0, 3.0.1 and 3.1 releases were more determined by timelines and management requirements than by stability. There was an urgent need for a 3.0.2 maintenance release that never came. Working with 3.1 pre-release promoted builds was a tedious two steps forward, one step back experience.

From this background, the announcement that the Glassfish 3.1.1 release will be delayed to "include more changes requested by the community" is very good news.

Since its acquisition of Sun, Oracle has not exactly gained the best of reputations in dealing with Open Source projects. There are plenty of reasons for criticizing Oracle's moves in the Java Community during the last twelve months, but lack of support for Glassfish is definitely not one of them.

One of the less successful innovations was the migration of the java.net infrastructure, running the Glassfish forums, among others. The forum continues to be a useful information source, but its usability has degraded a lot.

Anyway, the Glassfish community is not a one-way road, you do get feedback in the forum, both from users and from Oracle engineers, most of the bugs I reported have been fixed, many of them within a few days, and even though Oracle may be more of a benevolent dictator than a neutral community sponsor, I don't really mind as long as the entire ecosystem is thriving.

Read the whole story

Java EE 6 Server Comparison: Introduction

2011-06-07T17:54:00.001+02:00

A year and a half after the Java EE 6 specification release, there are now three Open Source servers certified for the Java EE 6 Web Profile:

Glassfish 3.x
JBoss 6.x
Resin 4.0.x

Time to take a look at these three servers in direct comparison and do some experiments, not with a toy or benchmark application developed for this purpose, but with a real-life enterprise application.

Taking an application developed on server X and porting it to servers Y and Z might cause some bias towards server X, so my scenario is the following:

MyApp is an enterprise application originally developed on Tomcat 6 and Spring 3, using JPA 2.0 (Hibernate on MySQL), declarative transactions, dependency injection, and Spring MVC in the web layer. In the persistence and service layers, this application has no compile time dependencies on Spring, thanks to the fact that Java EE 6 annotations like @Inject or @TransactionAttribute are also supported by Spring.
MyApp is built with Maven, my development environment is Eclipse 3.6.2 with m2eclipse.

The goals of my experiments:

Make MyApp run on a Java EE 6 server (using JPA, JTA, EJB, CDI, Servlet, JSP) and restrict the run-time use of Spring to the web layer, using a CDI-to-Spring bridge.
Build a WAR that can be deployed to each of Glassfish, JBoss and Resin.
Work with the Eclipse plugin for each server, compare the edit-save-deploy cycles and developer productivity in general.
Work with the stand-alone servers, and compare startup and deployment times and memory usage.

Disclaimer: I started working with Glassfish in March 2010 and I haven't really used either JBoss or Resin before these experiments, so I may be biased towards Glassfish. I'm not in any way affiliated with Oracle, Red Hat or Caucho, and the views expressed in this blog are my own and not my employer's.

A note on Java EE 6 Profiles and certification: Glassfish is of course certified both for the Full Profile and for the Web Profile. JBoss AS 6.0.0.Final was announced as a fully certified implementation of the Java EE 6 Web Profile, but for some reason it is not yet listed on the official Java EE compatibility page. JBoss also claims to support the Full Profile, but has not taken the pains of undergoing certification for it. Resin is explicitly focused on the Web Profile and has recently been certified. Resin also supports JMS, which is not part of the Web Profile.

MyApp only requires the Web Profile. For my experiments, I've used the Full Profile installation of Glassfish, the default server configuration of JBoss, and the Open Source Resin installation (not Resin Pro).

Performance Measurement Scenarios

For each server, I ran the following tests:

Empty server startup time. ("Empty" means without any applications deployed by the user, running the server out of the box, which may include some management applications.)
Empty server total heap and PermGen size. (After starting the server and forcing a garbage collection from jvisualvm.)
MyApp deployment time (copying it to the autodeploy directory of the running server).
Server + MyApp restart time (restarting the server with my deployed application).
Server + MyApp total heap and PermGen size. (After starting the server with the deployed applicaton and forcing a garbage collection from jvisualvm).
MyApp redeployment time (after touching myapp.war in the autodeploy directory).
Server + MyApp startup time from Eclipse. (Using Run on Server on the top-level web project).
Server + MyApp total heap and PermGen size, running from Eclipse (after forcing GC as above).
MyApp redeploy time after changing a source file in Eclipse.
Checking for memory leaks after redeploying.

Read the whole story

Java EE 6 Integration Testing on Resin and Glassfish

2011-06-06T12:33:00.000+02:00

jeeunit, my Java EE 6 Integration Testing project, now supports Caucho Resin 4.0.18 in addition to Glassfish 3.1.

Another new feature is a @Transactional annotation which can be used to wrap test methods in an auto-rollback transaction.

The current release jeeunit 0.8.0 is available from Maven Central and from the jeeunit Project Page on Google Code.

Using CDI from Spring

2011-06-01T13:46:00.000+02:00

Spring and Java EE are converging in many areas, mainly due to CDI. In fact, by confining yourself to a suitable subset of annotations and other configuration mechanisms, you can design your persistence and service layers to run either on Java EE 6 or on Spring 3, simply by selecting the appropriate set of libraries.

The web layer is a different story: if your web framework is tightly coupled to either Java EE 6 or Spring, it will be hard or even impossible to simply change the container, but even in the web layer, there are solutions like Apache Wicket which work well both with Spring and Java EE, all you need is some glue code to access the dependency injection container.

I'm currently migrating a web application from Tomcat/Spring to Glassfish, and since this application is based on Spring MVC, I cannot completely replace Spring in this step. The goal of the migration is to drop all Spring dependencies from all components except the web frontend and to somehow make the service beans (stateless session beans and CDI managed beans) visible to the Spring web application context.

Looking for a solution, I found Rick Hightower's post on CDI and Spring living in harmony and the related CDISource project. The basic idea is to use a Spring BeanFactoryPostProcessor to populate Spring's application context with a factory bean for each CDI bean obtained from the CDI BeanManager, thus making all CDI beans accessible as Spring beans.

It was very easy to integrate the CDISource Spring Bridge into my project, I just cloned their Git repository, ran the Maven build, added


    <dependency>
      <groupId>org.cdisource.springbridge</groupId>
      <artifactId>springbridge</artifactId>
      <version>1.0-SNAPSHOT</version>
    </dependency>

to my own POM and included

<bean class="org.cdisource.springintegration.CdiBeanFactoryPostProcessor"/>

in my Spring configuration.

This was enough to get going, but I soon found out that some standard but not-so-basic use cases are not yet supported, e.g.

@Stateless EJBs @Injected into CDI beans.
Multiple beans of the same type with @Named or other qualifiers.
Producer methods

So I started hacking the CdiBeanFactoryPostProcessor to make it work for my application. Here is the solution:

package org.cdisource.springintegration;

import java.lang.reflect.Type;
import java.util.Set;

import javax.enterprise.inject.spi.Bean;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.config.BeanFactoryPostProcessor;
import org.springframework.beans.factory.config.ConfigurableListableBeanFactory;
import org.springframework.beans.factory.support.BeanDefinitionBuilder;
import org.springframework.beans.factory.support.DefaultListableBeanFactory;

public class CdiBeanFactoryPostProcessor implements BeanFactoryPostProcessor {

    private static Logger logger = LoggerFactory.getLogger(CdiBeanFactoryPostProcessor.class);

    private boolean useLongName;

    private BeanManagerLocationUtil beanManagerLocationUtil = new BeanManagerLocationUtil();

    @Override
    public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory)
            throws BeansException {

        DefaultListableBeanFactory factory = (DefaultListableBeanFactory) beanFactory;

        Set<Bean<?>> beans = beanManagerLocationUtil.beanManager().getBeans(Object.class);
        for (Bean<?> bean : beans) {
            if (bean instanceof SpringIntegrationExtention.SpringBean) {
                continue;
            }

            if (bean.getName() != null && bean.getName().equals("Spring Injection")) {
                continue;
            }
            logger.debug("bean types = {}", bean.getTypes());
            Class<?> beanClass = getBeanClass(bean);
            BeanDefinitionBuilder definition = BeanDefinitionBuilder
                    .rootBeanDefinition(CdiFactoryBean.class)
                    .addPropertyValue("beanClass", beanClass)
                    .addPropertyValue("beanManager", beanManagerLocationUtil.beanManager())
                    .addPropertyValue("qualifiers", bean.getQualifiers()).setLazyInit(true);
            String name = generateName(bean);
            factory.registerBeanDefinition(name, definition.getBeanDefinition());
            logger.debug("bean name = {}, bean class = {}", bean.getName(), beanClass.getName());
        }
    }

    private Class<?> getBeanClass(Bean<?> bean) {
        Class<?> klass = Object.class;
        for (Type type : bean.getTypes()) {
            if (type instanceof Class) {
                Class<?> currentClass = (Class<?>) type;
                if (klass.isAssignableFrom(currentClass)) {
                    klass = currentClass;
                }
            }
        }
        return klass;
    }

    private String generateName(Bean<?> bean) {
        String name = bean.getName() != null ? bean.getName() : generateNameBasedOnClassName(bean);
        return name;
    }

    private String generateNameBasedOnClassName(Bean<?> bean) {
        Class<?> beanClass = getBeanClass(bean);
        return !useLongName ? beanClass.getSimpleName() + "FactoryBean" : beanClass.getName()
                .replace(".", "_") + "FactoryBean";
    }

    public void setUseLongName(boolean useLongName) {
        this.useLongName = useLongName;
    }

}

The main change is in getBeanClass(): It is important to note that the CDI API Bean.getBeanClass() does not return the class of the given bean: the result is the class in which the bean is defined. In particular, for producer methods, the result is the class containing the method and not the method return type.

Bean.getTypes() returns all classes and interfaces of potential injection points that the bean will match. This set of types does not always include the bean's implementation class, e.g. for a stateless session bean with a local business interface, the bean types will only be java.lang.Object and the business interfaces (and any superinterfaces of the business interface).

getBeanClass() currently selects the most specialized CDI bean type and uses this type to create the corresponding Spring factory bean.

The next point to note are qualifiers: I added a qualifiers property to the CdiFactoryBean and changed its getObject() method to use the qualifiers for looking up the bean in the CDI bean container:

package org.cdisource.springintegration;

import java.lang.annotation.Annotation;
import java.util.Set;

import javax.enterprise.inject.spi.BeanManager;

import org.cdisource.beancontainer.BeanContainer;
import org.cdisource.beancontainer.BeanContainerImpl;
import org.springframework.beans.factory.FactoryBean;
import org.springframework.beans.factory.InitializingBean;

public class CdiFactoryBean implements FactoryBean<Object>, InitializingBean {

    private Class<?> beanClass;
    private boolean singleton = true;
    private BeanContainer beanContainer;
    private BeanManager beanManager;
    private Set<Annotation> qualifiers;

    @Override
    public void afterPropertiesSet() throws Exception {
        if (beanManager == null)
            throw new IllegalStateException("BeanManager must be set");
        beanContainer = new BeanContainerImpl(beanManager);
    }

    public void setBeanClass(Class<?> beanClass) {
        this.beanClass = beanClass;
    }

    @Override
    public Object getObject() throws Exception {
        return beanContainer.getBeanByType(beanClass, qualifiers.toArray(new Annotation[] {}));
    }

    @Override
    public Class<?> getObjectType() {
        return beanClass;
    }

    @Override
    public boolean isSingleton() {
        return singleton;
    }

    public void setSingleton(boolean singleton) {
        this.singleton = singleton;
    }

    public void setBeanManager(BeanManager beanManager) {
        this.beanManager = beanManager;
    }

    public void setQualifiers(Set<Annotation> qualifiers) {
        this.qualifiers = qualifiers;
    }
}

I'm sure this is not the end of the story, both Spring and CDI are complex enough to allow use cases where this bridge is likely to break, but it now works perfectly for me in my real-life application.

Eclipse Helios with Firefox 4 on Ubuntu Natty

2011-05-25T17:08:00.002+02:00

I've just finished installing Kubuntu 11.04 on a spare partition, and I'm rather pleased with it, there were absolutely no problems with the usual suspects like X driver, sound or fake RAID.

The only regression I noted is that Eclipse 3.6.2 no longer supports an internal web browser. Ubuntu 11.04 comes with Firefox 4.x, and this is not currently supported by Eclipse SWT.

This thread gave me the hint how to fix it:

Install package libwebkitgtk-1.0-0 (do not use the newer version libwebkitgtk-3.0-0)
Add the following to eclipse.ini: -Dorg.eclipse.swt.browser.UseWebKitGTK=true
Create a symlink in /usr/lib to satisfy the SWT dependencies:
ln -sf libwebkitgtk-1.0.so.0.6.0 libwebkit-1.0.so.2

Restart Eclipse and check Window | Preferences | General | Web Browser. The radio button Use internal web browser should now be enabled.

Update 2 Jul 2001: After upgrading to Eclipse Indigo 3.7.0, the internal browser works out of the box. I did not have to modify eclipse.ini. But it seems Eclipse now uses WebKit by default, so you still need to install the WebKit library and create the symlink if you haven't done so before.

reFit Plugin for Jenkins

2011-04-03T23:13:00.000+02:00

After working with Hudson (now Jenkins) as a user for almost three years, I've now created my first plugin for publishing Fit test reports for Jenkins projects.

The plugin works with reFit 1.7.0 or higher. It is published on the Jenkins Community infrastructure, separate from the reFit project on Google Code. You can install the plugin from the Jenkins update centre.

For more details, see the reFit Plugin page in the Jenkins Wiki.

JAXB Marshalling with Custom Namespace Prefixes

2011-03-20T14:04:00.005+01:00

The Problem

You have an XML schema with multiple XML namespaces.
You generate a JAXB model with xjc.
You build a JAXB document model and use the JAXB Marshaller to create XML from the model.
You want to override the default namespace prefixes ns1, ns2, ... created by the Marshaller.

Solution 1

You can implement a NamespacePrefixMapper and pass it to the Marshaller

Marshaller m = context.createMarshaller();
    NamespacePrefixMapper mapper = new MyNamespacePrefixMapper();
    m.setProperty("com.sun.xml.internal.bind.namespacePrefixMapper", mapper);

Drawbacks:

You need to implement the Mapper.
The Marshaller property is implementation dependent.
When you use a standalone version of the JAXB Reference Implementation and not the one embedded in Oracle's JDK 6, the property name must be replaced by com.sun.xml.bind.namespacePrefixMapper.

Solution 2

Manually create the package-info.java source file which is normally generated by xjc:

@XmlSchema(namespace = "urn:example.com:foo",
    xmlns = { 
        @XmlNs(namespaceURI = "http://example.com/namespaces/bar", prefix = "bar"),
        @XmlNs(namespaceURI = "urn:example.com:foo", prefix = "")
    },
    elementFormDefault = javax.xml.bind.annotation.XmlNsForm.QUALIFIED)

package com.example.foo.jaxb;

import javax.xml.bind.annotation.XmlNs;
import javax.xml.bind.annotation.XmlSchema;

This is enough to make the Marshaller use the prefixes defined in the @XmlNs annotations.

Now you only need to make sure that xjc does not overwrite or duplicate the package-info.java source file by setting the -npa option.

Working with Maven and the maven-jaxb2-plugin, you simply put your customized package-info.java under src/main/java and add the -npa option to the plugin configuration:

<plugin>
        <groupId>org.jvnet.jaxb2.maven2</groupId>
        <artifactId>maven-jaxb2-plugin</artifactId>
        <version>0.7.5</version>
        <executions>
          <execution>
            <goals>
              <goal>generate</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <extension>true</extension>
          <args>
            <arg>-npa</arg>
          </args>
        </configuration>
      </plugin>

Update 21 March 2011: This approach does not work with the JAXB version contained in Oracle's JDK 1.6.0_24. You have to use a more recent version of the JAXB reference implementation by including the following dependency:

<dependency>
      <groupId>com.sun.xml.bind</groupId>
      <artifactId>jaxb-impl</artifactId>
      <version>2.2.2</version>
    </dependency>

Transparent Asynchronous Remoting via JMS

2011-03-13T20:49:00.002+01:00

The Scenario

A client needs to invoke a service interface with the following restrictions:

The service implementation is running on a remote machine.
This fact is transparent to the client, i.e. any service method invocation is just like a local method invocation.
The service methods are executed asychronously on the server, method invocations on the client return immediately (fire-and-forget).
All service methods have void return types.
The service invocations shall be transported via JMS.

Background

The post is a sequel to my previous article on Transparent Asynchronous Remoting, the only difference in this scenario is the addition of the JMS requirement.

This sounds more like an implementation detail than a requirement, but in fact there can be good reasons for mandating the use of JMS, e.g.

You want to decouple service providers and clients: a client may invoke a service method even when the service provider is temporarily not available.
The system's communication infrastructure is based on JMS anyway.
You want to monitor the communication.

The Solution with Apache Camel, Spring and Glassfish MQ

There is no out-of-the box solution for this scenario either in Java EE 6 or in Spring.

Java EE 6 has an @Async annotation for asynchronous invocations of remote EJBs, but there is no way to use a JMS transport simply by configuration.

Spring Remoting provides transparent proxies for remote services using various transports, including JMS, but these proxies are always synchronous.

I've tried to solve this problem by modifying or extending some of the Spring JMS helper classes, but this turned out to be not quite trivial. The next thing I tried was Spring Integration, but this seems to be lacking the transparent proxy feature and only provides a partial solution for services with single-parameter methods.

In the end, I turned to Apache Camel, which solves the problem rather nicely, once you find the relevant bits of example code and documentation. camel-example-spring-jms was a good starting point. The setup explained in the rest of this post differs from the Camel example in the following points:

The communication is asynchronous.
The Spring contexts are defined by Java Configuration instead of XML.
The JMS provider is Glassfish MQ, not Active MQ.
Apache Commons Logging is replaced by slf4j and logback.
Dependency injection is based on the JSR-330 @Inject annotation.

Overview

Apache Camel can be regarded as a giant plumbing framework for message-based communication between endpoints of all sorts. You create endpoints supporting a given communication protocol, you connect them with routes, and you may insert filters to modify the messages on their way.

Camel directly supports Spring beans as endpoints and uses proxies to adapt method invocations to Camel messages and vice versa. Hence, the setup for our scenario is rather similar to Spring Remoting.

On the server side, the service implementations are wrapped in proxies which are connected to JMS queues by Camel routes.

On the client side, you create proxies for the remote services and inject them into your application classes. The proxies are directly connected to JMS service URLs.

In addition, you need a JMS provider. The original Camel/Spring/JMS example uses an ActiveMQ broker embedded into the server. I'm using a stand-alone Glassfish MQ broker, just to show that Camel also works with other JMS implementations.

The API

Following the tradition of "Hello World" examples, we create a HelloService to say hello to a given person, and we add a GoodbyeService.

package com.blogspot.hwellmann.camel.greeter.api;

@InOnly 
public interface HelloService {
    void sayHello(Person person);
}

package com.blogspot.hwellmann.camel.greeter.api;

@InOnly 
public interface GoodbyeService {
    void sayGoodbye(Person person);
}

@InOnly is a Camel annotation indicating that a given method or, in this case, all methods of the given interface, take input parameters only and that the caller shall not wait for any reply. Thanks to this annotation, our scenario is indeed asynchronous.

By default, Camel uses standard Java serialization for transporting method parameters, so all parameter objects need to be Serializable:

package com.blogspot.hwellmann.camel.greeter.api;

import java.io.Serializable;

public class Person implements Serializable {

    private static final long serialVersionUID = 1L;

    private String firstName;
    private String lastName;
     
    public Person(String firstName, String lastName) {
        this.firstName = firstName;
        this.lastName = lastName;
    }

    // ...getters and setters...
}

The Server

package com.blogspot.hwellmann.camel.greeter.server;

public class HelloServiceImpl implements HelloService {

    public void sayHello(Person person) {
        try {
            Thread.sleep(1000);
            System.out.println(String.format("Hello, %s %s!", 
                    person.getFirstName(), person.getLastName()));
        }
        catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

The GoodbyeService is just the same. Now here is the Spring configuration for the server:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:context="http://www.springframework.org/schema/context" 
  xsi:schemaLocation="http://www.springframework.org/schema/beans
         http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
         http://www.springframework.org/schema/context
         http://www.springframework.org/schema/context/spring-context-3.0.xsd">

  <context:annotation-config  />
  <bean class="com.blogspot.hwellmann.camel.greeter.server.GreeterServerSpringConfig"/>

</beans>

package com.blogspot.hwellmann.camel.greeter.server;

@Configuration
public class GreeterServerSpringConfig {
    
    @Inject
    private ApplicationContext applicationContext;

    @Bean
    public ConnectionFactory cf() throws JMSException {
        com.sun.messaging.ConnectionFactory cf = new com.sun.messaging.ConnectionFactory();
        cf.setProperty(ConnectionConfiguration.imqAddressList, "localhost:7676");
        return cf;
    }
    
    @Bean
    public HelloService helloService() {
        return new HelloServiceImpl();
    }
    
    @Bean
    public GoodbyeService goodbyeService() {
        return new GoodbyeServiceImpl();
    }
    
    @Bean
    public JmsComponent jms() throws JMSException {
        JmsComponent jms = new JmsComponent();
        jms.setConnectionFactory(cf());
        return jms;
    }
    
    @Bean
    public RouteBuilder serverRoutes() {
        return new ServerRoutes();
    }
    
    @Bean
    public CamelContext camelContext() throws Exception {
        CamelContextFactoryBean factory = new CamelContextFactoryBean();
        factory.setApplicationContext(applicationContext);
        factory.setId("jms-server");
        SpringCamelContext context = factory.getContext();
        serverRoutes().addRoutesToCamelContext(context);
        return context;
    }    
}

The ServerRoutes bean defines two simple Camel routes connecting the JMS queues to our service implementations:

package com.blogspot.hwellmann.camel.greeter.server;

import org.apache.camel.builder.RouteBuilder;

public class ServerRoutes extends RouteBuilder {
    
    @Override
    public void configure() throws Exception {
        from("jms:queue:hello").to("helloService");
        from("jms:queue:goodbye").to("goodbyeService");
    }
}

Camel has its own org.apache.camel.spring.Main class which by default creates a Spring application context from all XML files found in META-INF/spring on the classpath. This is enough to launch the server.

In addition, you need to launch the Glassfish MQ message broker on the default port 7676. To do so, you can simply start a complete Glassfish instance, which has an embedded MQ broker, or you may use a standalone Glassfish MQ broker by running GLASSFISH_HOME/mq/bin/imqbrokerd, where GLASSFISH_HOME refers to your Glassfish installation directory. (Tested with Glassfish 3.1 and Glassfish 3.0.1).

The Client

This is the client for saying hello and goodbye to three persons:

package com.blogspot.hwellmann.camel.greeter.client;

public class GreeterClient implements Runnable {
    
    private static Logger log = LoggerFactory.getLogger(GreeterClient.class);

    @Inject
    private HelloService helloProxy;
    
    @Inject
    private GoodbyeService goodbyeProxy;
    
    private Person donald = new Person("Donald", "Duck");
    private Person mickey = new Person("Mickey", "Mouse");
    private Person charlie = new Person("Charlie", "Brown");

    public void run() {
        log.info("invoking greeter");
        
        helloProxy.sayHello(donald);
        helloProxy.sayHello(mickey);
        helloProxy.sayHello(charlie);

        goodbyeProxy.sayGoodbye(donald);
        goodbyeProxy.sayGoodbye(mickey);
        goodbyeProxy.sayGoodbye(charlie);

        log.info("done");
    }
        
    public static void main(String[] args) {
        ApplicationContext context = new AnnotationConfigApplicationContext(GreeterClientSpringConfig.class);
        GreeterClient client = context.getBean(GreeterClient.class);
        client.run();
    }    
}

Look Mum, no XML! The Spring application context for the client is completely defined in a Java configuration class:

package com.blogspot.hwellmann.camel.greeter.client;

@Configuration
public class GreeterClientSpringConfig {
    
    @Inject
    private ApplicationContext applicationContext;

    @Bean
    public ConnectionFactory cf() throws JMSException {
        com.sun.messaging.ConnectionFactory cf = new com.sun.messaging.ConnectionFactory();
        cf.setProperty(ConnectionConfiguration.imqAddressList, "localhost:7676");
        return cf;
    }
    
    @Bean
    public GreeterClient greeterClient() {
        return new GreeterClient();
    }
    
    @Bean
    public JmsComponent jms() throws JMSException {
        JmsComponent jms = new JmsComponent();
        jms.setConnectionFactory(cf());
        return jms;
    }
    
    @Bean
    public HelloService helloProxy() throws Exception {
        CamelProxyFactoryBean factory = new CamelProxyFactoryBean();
        factory.setServiceInterface(HelloService.class);
        factory.setServiceUrl("jms:queue:hello");
        factory.setCamelContext(camelContext());
        factory.afterPropertiesSet();
        return (HelloService) factory.getObject();
    }
    
    @Bean
    public GoodbyeService goodbyeProxy() throws Exception {
        CamelProxyFactoryBean factory = new CamelProxyFactoryBean();
        factory.setServiceInterface(GoodbyeService.class);
        factory.setServiceUrl("jms:queue:goodbye");
        factory.setCamelContext(camelContext());
        factory.afterPropertiesSet();
        return (GoodbyeService) factory.getObject();
    }
    
    @Bean
    public CamelContext camelContext() throws Exception {
        CamelContextFactoryBean factory = new CamelContextFactoryBean();
        factory.setApplicationContext(applicationContext);
        factory.setId("jms-client");
        factory.afterPropertiesSet();
        return (CamelContext) factory.getObject();
    }
}

The POM

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.blogspot.hwellmann</groupId>
  <artifactId>spring-camel-jms</artifactId>
  <version>0.0.1</version>
  <name>Spring Camel JMS Example</name>
  <properties>
    <camel.version>2.6.0</camel.version>
    <spring.version>3.0.5.RELEASE</spring.version>
  </properties>
  <dependencies>
    <dependency>
      <groupId>org.apache.camel</groupId>
      <artifactId>camel-core</artifactId>
      <version>${camel.version}</version>
      <exclusions>
        <exclusion>
          <groupId>commons-logging</groupId>
          <artifactId>commons-logging-api</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.apache.camel</groupId>
      <artifactId>camel-jms</artifactId>
      <version>${camel.version}</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-context</artifactId>
      <version>${spring.version}</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-core</artifactId>
      <version>${spring.version}</version>
      <exclusions>
        <exclusion>
          <groupId>commons-logging</groupId>
          <artifactId>commons-logging</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.apache.servicemix.bundles</groupId>
      <artifactId>org.apache.servicemix.bundles.cglib</artifactId>
      <version>2.2_1</version>
    </dependency>    
    <dependency>
      <groupId>org.apache.geronimo.specs</groupId>
      <artifactId>geronimo-atinject_1.0_spec</artifactId>
      <version>1.0</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-api</artifactId>
      <version>1.6.0</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>jcl-over-slf4j</artifactId>
      <version>1.6.0</version>
    </dependency>
    <dependency>
      <groupId>ch.qos.logback</groupId>
      <artifactId>logback-core</artifactId>
      <version>0.9.28</version>
    </dependency>
    <dependency>
      <groupId>ch.qos.logback</groupId>
      <artifactId>logback-classic</artifactId>
      <version>0.9.28</version>
    </dependency>
    <dependency>
      <groupId>com.sun.messaging.mq</groupId>
      <artifactId>imq</artifactId>
      <version>4.4.2</version>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
          <source>1.6</source>
          <target>1.6</target>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

Running the example

Now if you start the broker, the server and the client in this order, you'll see the following output from the client...

20:21:46.789 [main] INFO  c.b.h.c.greeter.client.GreeterClient - invoking greeter
20:21:47.016 [main] INFO  c.b.h.c.greeter.client.GreeterClient - done

...and from the server:

20:21:47.976 [DefaultMessageListenerContainer-1] INFO  c.b.h.c.g.server.HelloServiceImpl - Hello, Donald Duck!
20:21:47.995 [DefaultMessageListenerContainer-1] INFO  c.b.h.c.g.server.GoodbyeServiceImpl - Goodbye, Donald Duck!
20:21:48.998 [DefaultMessageListenerContainer-1] INFO  c.b.h.c.g.server.HelloServiceImpl - Hello, Mickey Mouse!
20:21:49.003 [DefaultMessageListenerContainer-1] INFO  c.b.h.c.g.server.GoodbyeServiceImpl - Goodbye, Mickey Mouse!
20:21:50.005 [DefaultMessageListenerContainer-1] INFO  c.b.h.c.g.server.HelloServiceImpl - Hello, Charlie Brown!
20:21:50.009 [DefaultMessageListenerContainer-1] INFO  c.b.h.c.g.server.GoodbyeServiceImpl - Goodbye, Charlie Brown!

The client terminates before the server has even processed the first message. By default, Camel generates a single-threaded consumer for each queue, so the order of the Hellos is preserved, but they get processed in parallel with the Goodbyes.

Conclusion

Using either Java EE 6 or Spring 3, the standard framework APIs are not sufficient to implement transparent asynchronous communication via JMS.

However, with Apache Camel on top of Spring 3, a plug-and-play solution is easy to set up.

Hamburg Elections Powered by MySQL

2011-02-20T19:54:00.000+01:00

Following a link on Google to the intermediate results of today's elections in Hamburg on the official website of Statistikamt Nord (Hamburg and Schleswig-Holstein Statistics Office), I found this:

Ok, the good news is that our taxes don't get wasted on expensive Oracle licenses...

Building a Web Application with Wicket, Spring and JPA

2011-02-20T14:36:00.002+01:00

This is a tutorial on setting up a web application stack with Wicket, Spring and JPA, with a special focus on avoiding Spring XML configuration in favour of Java configuration, and on using the JSR-330 @Inject annotation both in the web and service layers.

Martijn Dashorst's blog on Wicket/Spring/Hibernate configurationwas the starting point for my setup, and as I hate "oodles of XML" as much as he does, I replaced all the XML bean declarations by a @Configuration class, a new feature in Spring 3.0.x.

Let us start with the web.xml deployment descriptor, for hooking both Wicket and Spring into our servlet container:

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
  xmlns="http://java.sun.com/xml/ns/javaee"
  xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" 
  xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
  version="2.5">
  
  
  <display-name>WicketSpring</display-name>

  <context-param>
    <param-name>contextConfigLocation</param-name>
    <param-value>WEB-INF/app-context.xml</param-value>
  </context-param>


  <filter>
    <filter-name>WicketSpring</filter-name>
    <filter-class>org.apache.wicket.protocol.http.WicketFilter</filter-class>
    <init-param>
      <param-name>applicationFactoryClassName</param-name>
      <param-value>org.apache.wicket.spring.SpringWebApplicationFactory</param-value>
    </init-param>
  </filter>

  <filter-mapping>
    <filter-name>WicketSpring</filter-name>
    <url-pattern>/*</url-pattern>
  </filter-mapping>

  <listener>
    <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
  </listener>

</web-app>

The ContextLoaderListener will create the Spring application context as soon as the servlet context is available. The contextConfigLocation parameter defines the location of the XML configuration file for the Spring context - unfortunately, we cannot fully drop XML configuration in Spring 3.0.

Then we set up the usual WicketFilter and map all requests to this filter. Instead of providing the applicationClassName as in a vanilla Wicket setup, we configure a SpringWebApplicationFactory which will look up a Wicket WebApplication bean from the Spring context.

The Spring application context in WEB-INF/app-context.xml is rather short:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xmlns:context="http://www.springframework.org/schema/context"
     xmlns:tx="http://www.springframework.org/schema/tx"
     xsi:schemaLocation="http://www.springframework.org/schema/beans
         http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
         http://www.springframework.org/schema/tx
         http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
         http://www.springframework.org/schema/context
         http://www.springframework.org/schema/context/spring-context-3.0.xsd">

  <context:annotation-config/>
  <context:property-placeholder location="file:${wicketspring.config}"/>
  <tx:annotation-driven/>
  <bean class="com.blogspot.hwellmann.wicketspring.beans.WicketSpringConfig"/>      
</beans>

First, we enable configuration by annotations so we can do the rest of the setup in Java. The property-placeholder defines the location of a properties file outside of the WAR which we use to provide the JDBC settings. tx:annotation-driven enables declarative transactions by annotations. Finally, we define just one bean WicketSpringConfig which does the rest of the work in Java:

@Configuration
public class WicketSpringConfig {
    
    @Value("${wicketspring.jdbc.url}")
    private String jdbcUrl;
    
    @Bean
    public WicketSpringApplication wicketSpringApplication() {
        return new WicketSpringApplication();
    }
    
    @Bean
    public BookDao bookDao() {
        return new BookDao();
    }
    
    @Bean
    public DataSource dataSource() {
        return new SimpleDriverDataSource(new org.apache.derby.jdbc.EmbeddedDriver(), jdbcUrl);
    }

    @Bean
    public PlatformTransactionManager transactionManager() {
      return new JpaTransactionManager(entityManagerFactory());
    }
    
    @Bean
    public EntityManagerFactory entityManagerFactory() {
        LocalContainerEntityManagerFactoryBean bean = new LocalContainerEntityManagerFactoryBean();
        bean.setDataSource(dataSource());
        bean.setPersistenceProvider(new HibernatePersistence());
        bean.afterPropertiesSet();
        return bean.getObject();
    }    
}

There are some points to note about the entityManagerFactory bean: We use a Spring factory bean to construct a JPA EntityManagerFactory, so the Spring class should really be called LocalContainerEntityManagerFactoryFactoryBean (yuck) or LocalContainerEmfFactoryBean for short.

The Spring Reference Manual does not explain how to use factory beans in Java configurations. For factory beans in XML configurations, bean.getObject() is called automatically by the container to produce the real bean from the factory. Apparently, you have to do this explicitly in Java configurations, and afterPropertiesSet() is a prerequisite for calling getObject().

The @Value annotation on the jdbcUrl field is used to access a value from the properties file referenced in the XML context.

In our Wicket application class, we define a SpringComponentInjector which will inject Spring beans into our Wicket components:

public class WicketSpringApplication extends WebApplication {
    
    @Autowired
    private BookDao bookDao;
    
    @Override
    protected void init() {
        super.init();
        
        addComponentInstantiationListener(new SpringComponentInjector(this));
        bookDao.createBooks();
    }

    @Override
    public Class getHomePage() {
        return BooksPage.class;
    }
}

WicketSpringApplication itself is not a Wicket component, but it is a Spring bean, so the required bookDao is injected automatically by the Spring container. The Wicket components like BooksPage, on the other hand, are instantiated by Wicket outside of the Spring container, so we need to configure a ComponentInstantiationListener to inject Spring beans into our Wicket components.

The SpringComponentInjector is defined in the wicket-spring extension library. It will inject all Wicket component fields annotated by @SpringBean with beans from the Spring context. This is an example of a Wicket page with an injected Spring DAO:

public class BooksPage extends WebPage {
    
    @SpringBean
    private BookDao bookDao;
    
    public BooksPage() {
        
        add(new PropertyListView<Book>("book", bookDao.findBooks()) {

            @Override
            protected void populateItem(ListItem<Book> item) {
                item.add(new Label("author"));
                item.add(new Label("title"));
            }            
        });        
    }
}

The DAO itself is nothing special. It has an injected EntityManager and automatic transactions by means of the @Transactional annotation:

public class BookDao {
    
    @PersistenceContext
    private EntityManager em;
    
    private static String[][] parameters = {
        {"Thomas Mann", "Buddenbrooks"},
        {"Heinrich Mann", "Der Untertan"},
        {"Günter Grass", "Die Blechtrommel"}
    };
    
    @Transactional
    public void createBooks() {
        for (String[] param : parameters) {
            Book book = new Book(param[0], param[1]);
            em.persist(book);
        }        
    }    
    
    public List<Book> findBooks() {
        
        TypedQuery<Book> query = em.createQuery("select b from Book b", Book.class);
        return query.getResultList();
    }    
}

Now we're ready to launch our simple Web applications which will display a list of books from the database. I'm leaving out the Book entity and the BooksPage.html markup, which are completely trivial.

This is how the wicket-spring integration works out of the box, but I'm not quite satisified with that. There are two different annotations @Autowired and @SpringBean doing basically the same thing and tying my application classes to Spring.

However, I would like to preserve the freedom to run my application either on Spring or on Java EE, an approach I explained in another post.

More precisely, I would like to replace both @Autowired and @SpringBean by the standard @javax.inject.Inject annotation from JSR-330, and also replace Spring's @Transactional by @javax.ejb.TransactionAttribute.

Spring 3.0 honours the @Inject and @TransactionAttribute annotations, provided they are on the classpath, so all we need to do is add the corresponding Apache Geronimo Java EE specification JARs to our project.

Making wicket-spring recognize @Inject instead of @SpringBean requires a few more steps, which I'll just outline:

Create the classes Jsr330SpringComponentInjector, Jsr330SpringInjector and Jsr330ProxyFieldValueFactory as copies of SpringComponentInjector, SpringInjector and AnnotProxyFieldValueFactory in wicket-spring.
Replace references to SpringBean in Jsr330ProxyFieldValueFactory by references to Inject.
In our modified copies, reference the modified copies instead of the originals.

Not very elegant, but it works. All the copying could easily be avoided by modifying the wicket-spring sources, but this is currently out of scope.

With this approach, I've created a functional example project, using Eclipse, Maven, Tomcat, Hibernate and Derby as further ingredients. Here is my Maven POM:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.blogspot.hwellmann.wicketspring</groupId>
  <artifactId>wicketspring-web</artifactId>
  <packaging>war</packaging>
  <version>0.0.1-SNAPSHOT</version>
  <properties>
    <spring.version>3.0.5.RELEASE</spring.version>
    <wicket.version>1.4.15</wicket.version>
  </properties>
  <dependencies>
    <dependency>
      <groupId>org.apache.wicket</groupId>
      <artifactId>wicket</artifactId>
      <version>1.4.15</version>
    </dependency>
    <dependency>
      <groupId>org.apache.wicket</groupId>
      <artifactId>wicket-spring</artifactId>
      <version>1.4.15</version>
    </dependency>
    <dependency>
      <groupId>org.apache.wicket</groupId>
      <artifactId>wicket-ioc</artifactId>
      <version>1.4.15</version>
      <exclusions>
        <exclusion>
          <groupId>cglib</groupId>
          <artifactId>cglib-nodep</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-context</artifactId>
      <version>${spring.version}</version>
      <exclusions>
        <exclusion>
          <groupId>commons-logging</groupId>
          <artifactId>commons-logging</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-web</artifactId>
      <version>${spring.version}</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-orm</artifactId>
      <version>${spring.version}</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>jcl-over-slf4j</artifactId>
      <version>1.5.11</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-api</artifactId>
      <version>1.5.11</version>
    </dependency>
    <dependency>
      <groupId>ch.qos.logback</groupId>
      <artifactId>logback-core</artifactId>
      <version>0.9.20</version>
    </dependency>
    <dependency>
      <groupId>ch.qos.logback</groupId>
      <artifactId>logback-classic</artifactId>
      <version>0.9.20</version>
    </dependency>
    <dependency>
      <groupId>javax.servlet</groupId>
      <artifactId>servlet-api</artifactId>
      <version>2.5</version>
      <type>jar</type>
      <scope>provided</scope>
    </dependency>
    <dependency>
      <groupId>org.apache.geronimo.specs</groupId>
      <artifactId>geronimo-atinject_1.0_spec</artifactId>
      <version>1.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.geronimo.specs</groupId>
      <artifactId>geronimo-ejb_3.1_spec</artifactId>
      <version>1.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.derby</groupId>
      <artifactId>derby</artifactId>
      <version>10.7.1.1</version>
    </dependency>
    <dependency>
      <groupId>org.hibernate</groupId>
      <artifactId>hibernate-entitymanager</artifactId>
      <version>3.5.6-Final</version>
    </dependency>
  </dependencies>
</project>

As my maths books used to say, "the details are left as a trivial exercise to the reader", but I hope I've provided enough details to get anyone started...

Maven Snapshot Repositories with Nexus

2011-02-07T23:15:00.001+01:00

When you have a Maven build which depends on more than one external repository, it is usually a good idea to work with a repository manager. A repository manager also enables you to share your artifacts with other team members.

Nexus Open Source is probably the most popular Maven repository manager. Once it's set up correctly, it requires little or no maintenance, at least that's my experience after using it in different projects over the last three years.

I've just finished setting up Nexus for my current project, but this time, I spent almost a day chasing a problem related to snapshot repositories.

I had set up the Public Repositories group of Nexus as a mirror for Maven central and configured the Nexus Snapshots repository as an additional snapshot-enabled repository for the snapshots from my own project:

<mirrors>
    <mirror>
      <id>mynexus</id>
      <mirrorOf>central</mirrorOf>
      <name>Nexus</name>
      <url>http://nexus.example.com/content/groups/public</url>
    </mirror>
  </mirrors>

  <profile>
    <id>mysnapshots</id>
    <repositories>
      <repository>
        <id>mysnapshots</id>
        <url>https://nexus.example.com/content/repositories/snapshots</url>
        <snapshots>
          <enabled>true</enabled>
        </snapshots>        
       </repository>
     </repositories>
  </profile>

  <activeProfiles>
    <activeProfile>mysnapshots</activeProfile>
  </activeProfiles>

I did not enable snapshots for Central as suggested in the Nexus book, to avoid picking up any snapshots except my own. This setup worked fine in my local workspace and on our build server, but then my off-site co-workers started complaining about build failures.

They had error messages of the kind

Unable to find resource 'com.example.foo:jar:0.0.1-SNAPSHOT' in repository mysnapshots (http://nexus.example.com/content/repositories/snapshots)

even though they could see the required artifact in the given repository via the Nexus interface. Or so we thought...

It turned out that we'd simply misinterpreted the Maven console messages:

Downloading: http://nexus.example.com/content/repositories/snapshots/com/example/foo/0.0.1-SNAPSHOT/foo-0.0.1-SNAPSHOT.pom

should have given us a clue to the root cause, seeing that snapshot resources in remote repositories always have the SNAPSHOT suffix replaced by a timestamp, so the message really should have been something like

Downloading: http://nexus.example.com/content/repositories/snapshots/com/example/foo/0.0.1-SNAPSHOT/foo-0.0.1-20110207.090228-284.pom

The problem was the configuration of the Public Repositories group in Nexus which includes the hosted Snapshots repository by default. This appears to have led to the following scenario:

Client requests com.example.foo:jar:0.0.1-SNAPSHOT.
Maven tries to locate this artifact in Central.
Central is mirrored to Nexus Public Repositories, which includes Nexus Snapshots, which hosts the requested artifact.
Thus, Maven has found a match in Central and tries to download it from Central.
Now since Snapshots are not enabled for Central, the SNAPSHOT to timestamp conversion does not take place and Maven tries to download a non-existing resource, failing with a somewhat misleading error message.

Solution:

After deleting the Snapshots repository from the Public Repositories group in Nexus, all builds were working fine on all sites.

Transparent Asynchronous Remoting

2011-01-22T20:20:00.001+01:00

The Scenario

A client needs to invoke a service interface with the following restrictions:

The service implementation is running on a remote machine.
This fact is transparent to the client, i.e. any service method invocation is just like a local method invocation.
The service methods are executed asychronously on the server, method invocations on the client return immediately (fire-and-forget).
All service methods have void return types.

The Solution with Java EE 6

With Java EE 6, the solution is quite simple, thanks to the @Asynchronous annotation introduced in EJB 3.1. You simply create a remote session bean and add this annotation to any business method or to the entire class. The container will then execute the given methods asynchronously.

Patrick Champion has a complete example in his blog, so there's no need for me to provide any sample code.

When your client is also a Java EE 6 application, you can simply @Inject the service interface and use a @Produces annotation on an @EJB reference somewhere else to direct the client to the appropriate implementation.

In addition, to avoid hardcoding the service URL in your client, you should define a local JNDI alias in your Java EE 6 container for the address of the remote service implementation, so you can move the remote implementation to another host without recompiling your client.

The Solution with Spring 3.0

Spring does not have an out-of-the box solution for this scenario. Spring Remoting provides transparent proxies for remote services, but these proxies are always synchronous. Since Spring 3.0, there annotation support for asynchronous execution, but this only applies to local beans.

However, it is not hard to combine these two features with some glue code to implement a solution for our scenario.

Overview

Spring Remoting requires you to do some plumbing both on the server and client sides. On the server side, you expose your service beans wrapped in a HttpInvokerServiceExporter. On the client side, you need to configure a local proxy for each remote service using an HttpInvokerProxyFactoryBean.

Now this HttpInvokerProxyFactoryBean lets you provide a custom HttpInvokerRequestExecutor, so the idea is to build an AsyncHttpRequestExecutor which delegates the remote invocation to an AsyncHttpRequestWorker annotated with @Async, so that the remote invocation will indeed be executed asynchronously.

Here is an example with two remote services - I'm using two services to make clear which of the beans and helper objects need to be constructed per service and which can be shared. The application contexts will use Java configuration to minimize the amount of XML.

The API

Following the tradition of "Hello World" examples, we create a HelloService to say hello to a given person, and we add a GoodbyeService.

package com.blogspot.hwellmann.greeter.api;

public interface HelloService {

    void sayHello(Person person);
}

package com.blogspot.hwellmann.greeter.api;

public interface GoodbyeService {

    void sayGoodbye(Person person);
}

Note that Spring Remoting uses standard Java serialization for transporting method parameters, so all parameter objects need to be Serializable:

package com.blogspot.hwellmann.greeter.api;

import java.io.Serializable;

public class Person implements Serializable {

    private static final long serialVersionUID = 1L;

    private String firstName;
    private String lastName;
     
    public Person(String firstName, String lastName) {
        this.firstName = firstName;
        this.lastName = lastName;
    }

    // ...getters and setters...
}

The Server

package com.blogspot.hwellmann.greeter.server;

import com.blogspot.hwellmann.greeter.api.HelloService;
import com.blogspot.hwellmann.greeter.api.Person;


public class HelloServiceImpl implements HelloService {

    public void sayHello(Person person) {
        try {
            Thread.sleep(1000);
            System.out.println(String.format("Hello, %s %s!", 
                    person.getFirstName(), person.getLastName()));
        }
        catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

The GoodbyeService is left as an exercise to the reader ;-) Now here is the Spring configuration for the server:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:context="http://www.springframework.org/schema/context" 
  xsi:schemaLocation="http://www.springframework.org/schema/beans
         http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
         http://www.springframework.org/schema/context
         http://www.springframework.org/schema/context/spring-context-3.0.xsd">

  <context:annotation-config  />
  <bean class="com.blogspot.hwellmann.greeter.server.GreeterServerSpringConfig"/>

</beans>

package com.blogspot.hwellmann.greeter.server;

import java.io.IOException;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import com.blogspot.hwellmann.async.HttpServiceExporterFactoryBean;
import com.blogspot.hwellmann.greeter.api.GoodbyeService;
import com.blogspot.hwellmann.greeter.api.HelloService;
import com.sun.net.httpserver.HttpServer;

@Configuration
@SuppressWarnings("restriction")
public class GreeterServerSpringConfig {
    
    
    @Bean
    public HelloService helloService() {
        return new HelloServiceImpl();
    }
    
    @Bean
    public GoodbyeService goodbyeService() {
        return new GoodbyeServiceImpl();
    }
    
    @Bean
    public HttpServer httpServer() throws IOException {
        HttpServiceExporterFactoryBean factory = new HttpServiceExporterFactoryBean();
        factory.setPort(9999);
        
        factory.add(HelloService.class, helloService());
        factory.add(GoodbyeService.class, goodbyeService());

        factory.afterPropertiesSet();
        return factory.getObject();
    }    
}

I'm using a simple standalone HTTP server for this example. The Spring reference manual shows how to configure a dispatcher servlet instead when the services are running in a web container anyway.

The Client

This is the client for saying hello and goodbye to three persons:

package com.blogspot.hwellmann.greeter.client;

import javax.inject.Inject;

import org.springframework.context.ApplicationContext;
import org.springframework.context.support.GenericXmlApplicationContext;

import com.blogspot.hwellmann.greeter.api.GoodbyeService;
import com.blogspot.hwellmann.greeter.api.HelloService;
import com.blogspot.hwellmann.greeter.api.Person;

public class GreeterClient implements Runnable {

    @Inject
    private HelloService helloProxy;

    @Inject
    private GoodbyeService goodbyeProxy;

    private Person donald = new Person("Donald", "Duck");
    private Person mickey = new Person("Mickey", "Mouse");
    private Person charlie = new Person("Charlie", "Brown");

    public void run() {
        System.out.println("invoking greeter");
        
        helloProxy.sayHello(donald);
        helloProxy.sayHello(mickey);
        helloProxy.sayHello(charlie);
        
        goodbyeProxy.sayGoodbye(donald);
        goodbyeProxy.sayGoodbye(mickey);
        goodbyeProxy.sayGoodbye(charlie);

        System.out.println("done");
    }
    
    public static void main(String[] args) throws InterruptedException {
        ApplicationContext context = new GenericXmlApplicationContext("/META-INF/spring/client-context.xml");
        GreeterClient client = context.getBean(GreeterClient.class);
        client.run();
    }    
}

And here is the Spring Configuration:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:context="http://www.springframework.org/schema/context" 
  xmlns:task="http://www.springframework.org/schema/task"
  xsi:schemaLocation="http://www.springframework.org/schema/beans
         http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
         http://www.springframework.org/schema/task
         http://www.springframework.org/schema/task/spring-task-3.0.xsd
         http://www.springframework.org/schema/context
         http://www.springframework.org/schema/context/spring-context-3.0.xsd">

  <context:annotation-config  />
  <task:annotation-driven proxy-target-class="true"/>
  <bean class="com.blogspot.hwellmann.greeter.client.GreeterClientSpringConfig"/>

</beans>

Note the <task:annotation-driven> element which is required to enable the @Async annotation.

package com.blogspot.hwellmann.greeter.client;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.remoting.httpinvoker.HttpInvokerProxyFactoryBean;
import org.springframework.remoting.httpinvoker.SimpleHttpInvokerRequestExecutor;

import com.blogspot.hwellmann.async.AsyncHttpRequestExecutor;
import com.blogspot.hwellmann.async.AsyncHttpRequestWorker;
import com.blogspot.hwellmann.greeter.api.GoodbyeService;
import com.blogspot.hwellmann.greeter.api.HelloService;

@Configuration
public class GreeterClientSpringConfig {
    
    private <T> T createHttpInvokerProxy(Class<T> service, String url) {
        HttpInvokerProxyFactoryBean factory = new HttpInvokerProxyFactoryBean();
        factory.setServiceInterface(service);
        factory.setServiceUrl(url);
        factory.setHttpInvokerRequestExecutor(asyncHttpRequestExecutor());
        factory.afterPropertiesSet();
        return service.cast(factory.getObject());        
    }
    
    private <T> T createHttpInvokerProxy(Class<T> service) {
        String url = "http://remotehost:9999/remoting/" + service.getName();
        return createHttpInvokerProxy(service, url);
    }
    
    @Bean
    public HelloService helloProxy() {
        return createHttpInvokerProxy(HelloService.class);
    }
    
    @Bean
    public GoodbyeService goodbyeProxy() {
        return createHttpInvokerProxy(GoodbyeService.class);
    }
    
    @Bean
    public SimpleHttpInvokerRequestExecutor asyncHttpRequestExecutor() {
        return new AsyncHttpRequestExecutor();
    }
    
    @Bean
    public AsyncHttpRequestWorker asyncWorker() {
        return new AsyncHttpRequestWorker();
    }

    @Bean
    public GreeterClient greeterClient() {
        return new GreeterClient();
    }    
}

You'll probably want to use a PropertyPlaceholderConfigurer to inject the remote host name from a properties file.

The Glue

Here is a helper class reducing the amount of boilerplate code for exporting your services on the server:

package com.blogspot.hwellmann.async;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

import org.springframework.remoting.httpinvoker.SimpleHttpInvokerServiceExporter;
import org.springframework.remoting.support.SimpleHttpServerFactoryBean;

import com.sun.net.httpserver.HttpHandler;

@SuppressWarnings("restriction")
public class HttpServiceExporterFactoryBean extends SimpleHttpServerFactoryBean {

    public static final String DEFAULT_CONTEXT_ROOT = "/remoting/";
    
    private Map<String, HttpHandler> contexts = new HashMap<String, HttpHandler>();
    
    private String contextRoot = DEFAULT_CONTEXT_ROOT;

    
    public <T> void add(Class<T> klass, T service) {

        SimpleHttpInvokerServiceExporter exporter = new SimpleHttpInvokerServiceExporter();
        exporter.setService(service);
        exporter.setServiceInterface(klass);
        exporter.afterPropertiesSet();

        contexts.put(contextRoot + klass.getName(), exporter);
    }
    
    @Override
    public void afterPropertiesSet() throws IOException {
        setContexts(contexts);
        super.afterPropertiesSet();
    }
    
    public void setContextRoot(String contextRoot) {
        this.contextRoot = contextRoot;
        if (! contextRoot.endsWith("/")) {
            this.contextRoot += "/";
        }
    }
}

This is the AsyncHttpRequestExecutor:

package com.blogspot.hwellmann.async;

import java.io.ByteArrayOutputStream;
import java.io.IOException;

import javax.inject.Inject;

import org.springframework.remoting.httpinvoker.HttpInvokerClientConfiguration;
import org.springframework.remoting.httpinvoker.SimpleHttpInvokerRequestExecutor;
import org.springframework.remoting.support.RemoteInvocationResult;

public class AsyncHttpRequestExecutor extends SimpleHttpInvokerRequestExecutor {

    @Inject
    private AsyncHttpRequestWorker worker;
    
    @Override
    public RemoteInvocationResult doExecuteRequest(HttpInvokerClientConfiguration config,
            ByteArrayOutputStream baos) throws IOException, ClassNotFoundException {
        worker.executeAsyncRequest(config, baos);
        return new RemoteInvocationResult(null);
    }    
}

And this is the worker which gets proxied by Spring to run asynchronously:

package com.blogspot.hwellmann.async;

import java.io.ByteArrayOutputStream;
import java.io.IOException;

import org.springframework.remoting.httpinvoker.HttpInvokerClientConfiguration;
import org.springframework.remoting.httpinvoker.SimpleHttpInvokerRequestExecutor;
import org.springframework.scheduling.annotation.Async;

public class AsyncHttpRequestWorker extends SimpleHttpInvokerRequestExecutor {

    @Async
    public void executeAsyncRequest(HttpInvokerClientConfiguration config,
            ByteArrayOutputStream baos) {
        try {
            super.doExecuteRequest(config, baos);
        }
        catch (IOException exc) {
            exc.printStackTrace();
        }
        catch (ClassNotFoundException exc) {
            exc.printStackTrace();
        }
    }
}

That's all. Starting the server and then the client, you'll see the following output from the client...

invoking greeter
done

...and from the server:

Hello, Mickey Mouse!
Bye, Charlie Brown!
Bye, Mickey Mouse!
Hello, Donald Duck!
Hello, Charlie Brown!
Bye, Donald Duck!

The execution order on the server is indeterminate.

Traditional Approaches

In existing software systems, it is not uncommon to see a similar scenario implemented by means of a JMS message queue.

Now there's nothing wrong with JMS, but I think it's simply overkill for our simple remoting scenario, where persistence or multiple subscribers are not required. Besides, JMS does not address transparency, you have to marshall your method parameters into messages and vice versa.

The Lingo project used to provide transparent proxies on top of JMS, but I would not recommend it. Lingo has not been maintained since 2006, the Maven artifacts are unusable due to a broken POM referencing snapshot artifacts, and it has dependencies on outdated Spring versions, so if your own system uses any recent Spring version, you'll have to patch Lingo due to some API changes in Spring.

Apache Camel is sometimes perceived as an up-to-date alternative to Lingo, but in fact it does not provide transparent proxies.

As a sidenote, Javalobby has an interesting discussion thread on Why are Java developers ignorant of JMS and messaging in general?, which is five years old but still worth reading.

Conclusion

Using either Java EE 6 or Spring 3, the standard framework APIs are sufficient to implement transparent asynchronous communication with a remote system. Third-party frameworks or additional systems like a JMS server are not required.

While Java EE 6 has a standard solution for this scenario, there is no direct equivalent in Spring 3. Anybody still claiming that Spring is "simpler than Java EE"?

reFit: Acceptance Testing for Java EE 6, Spring and OSGi

2011-01-18T22:07:00.000+01:00

Even if you don't practice test-driven development, you are certainly familiar with the JUnit family of testing frameworks (including ports to other languages like cppunit, NUnit, or independent but similar approaches like TestNG).

This post is about the Fit framework family, which has a somewhat different focus and is not just another JUnit clone. In a nutshell, Fit represents expected and actual test results in tables, and you do not have to be a programmer to read or write them.

The following table has one column of inputs and two columns of outputs:

Each row of this table can be regarded as an acceptance test case. You can feed this table to a given system and compare the outputs:

It's only a couple of weeks ago I was introduced to Fit in my current project, and I must admit my first thought was "Why don't you just write a @Parameterized JUnit test?"

The answer is, JUnit is for developers, and developers only; Fit is for customers and developers.

You still need a developer to implement the skeleton logic of a test (called fixture in Fit), but anyone can write new test cases by adding rows or columns to a given Fit table.

The Fit method and the original Java implementation were created by Ward Cunningham in or around 2002, the project is hosted on Sourceforge and has been inactive since 2008.

The Fitnesse project integrates Fit with a Wiki, it includes a modified version of the original Fit implementation and is under active development. However, Fit and Fitnesse are incompatible, Fit lets you write your test tables in plain old HTML, whereas Fitnesse supports its own Wiki syntax only.

Our project has a fair amount of plain old HTML Fit tests, and our production code uses Spring, so it was a fairly natural idea to inject Spring beans from the system under test into our Fit test code.

Spring is just one (and not really my favourite) framework for dependency injection (and lots of others things). so I inevitably started thinking about how to use Fit together with Java EE or OSGi.

Thinking was followed by coding (yeah, it can be the other way round sometimes...), and this led to a project called reFit hosted on Google Code.

reFit is based on the latest Fit sources from SourceForge, it provides up-to-date Maven artifacts on Maven Central and integrations with Java EE 6, Spring and OSGi Declarative Services. The Java EE integration reuses parts of my jeeunit project.

In addition, reFit lets you run Fit tests under a JUnit wrapper or from a Maven plugin, both with or without Spring integration. reFit includes ready-to-run example code for Glassfish 3.1, Spring 3, Equinox and Weld SE.

There's more to come, I'm currently experimenting with a Web frontend and a WYSIWYG HTML editor.

For more details, check out the reFit Wiki.

Eclipse Meta-Error

2011-01-04T23:08:00.000+01:00

Maven Plugin Inheritance

2011-01-01T22:00:00.000+01:00

Writing Plugins for Maven is not hard, and Maven: The Complete Reference has an entire chapter on this subject. Of course, nothing is ever complete in real life or in software development, and one of the topics not covered in the book is plugin inheritance.

The scenario:

There is a (fictitious) maven-tea-plugin which takes a number of configuration parameters and has a tea goal. It is implemented by a TeaMojo.
You are writing a maven-greentea-plugin which has almost but not quite the same behaviour as the maven-tea-plugin. It takes the same parameters and has the same goals.
The natural idea is to derive a GreenTeaMojo from the TeaMojo and to override one or two methods of the base class, reusing all the parameters.

Unfortunately, this does not work, at least not out of the box. The problem is:

Maven uses poor man's Javadoc annotations to inject configuration parameters into Mojo classes.
This is still true in Maven 3.0, even though the traditional Plexus container has now been replaced by Guice.
The maven-plugin-plugin (i.e. the plugin responsible for plugin building) needs to scan the Mojo sources for Javadoc annotations, but the maven-tea-plugin sources are not visible during the build of the maven-greentea-plugin.

Now the maven-inherit-plugin from OPS4J fills the gap by looking up the plugin descriptor from the maven-tea-plugin and merging it with any additional parameters defined in the maven-greentea-plugin. To use it, just follow the documentation.

There is just one point to note: You have to add an @extendsPlugin annotation to your GreenTeaMojo to indicate the plugin you want to extend, but the value of this annotation is not quite obvious. It has to be either the plugin artifact id, or the plugin short name, tea in our case, but this requires the artifact id to be maven-tea-plugin or tea-maven-plugin.

Java EE 6 Integration Testing Easier Than Ever

2010-12-28T21:18:00.000+01:00

With jeeunit 0.6.0, you can write and run JUnit integration tests for Java EE 6 components as easy as this:

@RunWith(JeeunitRunner.class)
class MySessionBeanTest {
    
    @Inject
    private MySessionBean mySessionBean;

    @Test
    public void aSimpleTest() {
        boolean result = mySessionBean.doSomething();
        assertTrue(result);
    }
}

The JeeunitRunner builds a WAR on the fly, launches an embedded container, deploys the WAR and delegates all test methods to a proxy runner within the container.

With this approach, you can run your test class (or even a package with multiple test classes) from Eclipse via the context menu. The container is only launched once for all classes.

jeeunit currently supports Glassfish 3.1-b33 or higher. It uses the new Embedded Glassfish API introduced in b33.

Compared to earlier jeeunit versions, it is no longer required to build a test WAR manually or to provide a test suite for running multiple test classes.

Thanks to the excellent service provided by Sonatype OSS Repository Hosting, jeeunit is now available from Maven Central.

Glassfish Logging with slf4j (Part 2)

2010-12-18T20:50:00.001+01:00

My previous post on the same subject is one of the most popular articles in this blog, and since Glassfish still does not officially support slf4j, this update may be helpful to anyone hitting my blog via Google trying to improve their logging experience with Glassfish.

To redirect (almost) all log messages from Glassfish and your applications to the same log file and/or console, follow this recipe:

Copy these JARs to [install-root]/lib/endorsed:

jul-to-slf4j.jar
slf4j-api.jar
logback-classic.jar
logback-core.jar

Build a JAR containing your logback.xml configuration at root level and put it in the same place. [Update 26 Jul 2011: Actually, this is no longer required: Simply create a logback configuration file somewhere in your local file system and add the system property -Dlogback.configurationFile=file:/path/to/my/logback.xml to your Glassfish configuration.]
Define a new system property in the jvm-options section of your domain.xml:
-Djava.util.logging.config.file=${com.sun.aas.instanceRoot}/config/my_logging.properties
Create the corresponding file my_logging.properties with the following contents:

handlers = org.slf4j.bridge.SLF4JBridgeHandler
com.sun.enterprise.server.logging.GFFileHandler.flushFrequency=1
com.sun.enterprise.server.logging.GFFileHandler.file=${com.sun.aas.instanceRoot}/logs/server.log
com.sun.enterprise.server.logging.GFFileHandler.rotationTimelimitInMinutes=0
com.sun.enterprise.server.logging.GFFileHandler.logtoConsole=false
com.sun.enterprise.server.logging.GFFileHandler.rotationLimitInBytes=2000000
com.sun.enterprise.server.logging.GFFileHandler.alarms=false
com.sun.enterprise.server.logging.GFFileHandler.formatter=com.sun.enterprise.server.logging.UniformLogFormatter
com.sun.enterprise.server.logging.GFFileHandler.retainErrorsStasticsForHours=0

When you restart Glassfish, you should only see two log messages in the old java.util.logging format, all the rest goes via slf4j and logback to the appenders configured in your logback.xml. The two remaining log messages are issued before Glassfish reads the domain.xml configuration with the adapted configuration, so there is no easy way of avoiding them.

I have been using this approach on various Glassfish versions from 3.0.1 to 3.1-b33. It is rather ugly, but nonetheless effective.

All this may sound rather mysterious, so I'd better add some explanations:

jul-to-slf4j.jar is a bridge for redirecting java.util.logging via the slf4j API to any logging backend of your choice (logback in this case). To make the redirection work, the bridge has to be visible to the classloader evaluating the logging configuration. Since Glassfish starts logging very early in its bootstrapping phase, the lib/endorsed folder is the only place where the bridge has the desired effect.

Unfortunately, class file folders seem to be unsupported for endorsed libraries. Of course, it would be easier to simply put the logback.xml file in lib/endorsed/classes where you can directly edit it. But for a quick change to your logging configuration, you can always use an editor which supports editing ZIP/JAR file contents in place.

I created the contents of my_logging.properties by trial and error, starting with a one-liner containing only the handler. This caused a couple of exceptions as Glassfish seemed to be missing the GFFileHandler settings, so I copied them over from the original logging.properties.

The same procedure should also work for other slf4j backends, so you could replace the logback JARs by a log4j.jar.

An Update on Memory Issues in Glassfish and Weld

2010-12-18T18:47:00.000+01:00

Using the most recent Glassfish milestone build 3.1-b33, I took another look at the memory issues that had plagued my project on Glassfish 3.0.1. The last build I tried was b30, which did not even let me deploy my application.

Just in time for Christmas, there are Good Tidings - Weld has improved a lot, and I can no longer see any suspicious memory usage related to Weld. I took my Wicket+EJB+OpenJPA+PostgreSQL application packaged in a single WAR and deployed it to several different Glassfish versions from the command line. Then I used the Eclipse Memory Analyzer to a create a heap dump of the running Glassfish instance.

The Total Heap column in the following table is the total size of the heap as reported by the Memory Analyzer. The Shallow, Retained, and Percentage Columns refer to the amount of heap memory consumed by instances of org.jboss.weld.introspector.weld.jlr.WeldClassImpl

Glassfish Version	Total Heap	Shallow	Retained	Percentage
3.0.1	412.1 MB	84.680	378.215.056	87.52 %
3.1-b26	335.3 MB	85.144	205.527.048	58.38 %
3.1-b33	149.6 MB	84.216	7.041.704	4.49 %

The retained heap of an object is the set of objects that will be garbage collected when the given parent object is garbage collected.

So in b33, Weld still holds 5 % of the total heap - whether or not this is still too much or just reasonable is surely debatable. But I'm really happy to see that Weld has improved a lot, so I would no longer consider it as a no-go.

The version of my application I used for these measurements is the last one that actually used Weld, before I decided in August to stop using CDI to get rid of most memory issues.

I took some more measurements with the successor version in which CDI was eliminated essentially just by replacing @Inject by @EJB throughout the sources. This is really the only difference between the two versions, and here are the total heap sizes:

Glassfish Version	with @Inject	with @EJB
3.0.1	412.1 MB	44.4 MB
3.1-b33	149.6 MB	138.3 MB

The illustrates, as claimed in my previous post, that I was able to save 90 % of heap memory by not using CDI on Glassfish 3.0.1. Doing the same on Glassfish 3.1-b33, I can still save about 9 %.

But I was really surprised to see the heap usage of my application increase by a factor of 3 between Glassfish 3.0.1 and 3.1-b33.

The Memory Analyzer reveals the culprit to be org.glassfish.hk2.classmodel.reflect.impl.TypesCtr with a retained heap of about 80 MB.

I have no clue what this class is doing, but it smells like another case of an oversized reflective model of the application, which was just the problem with earlier Weld versions.

GLASSFISH-15266 is the related issue in Glassfish JIRA.

The Interface Antipattern

2010-12-10T22:18:00.000+01:00

Keeping interfaces and implementations separate is a useful design pattern for building modular systems. If your clients only depend on a service interface, you can switch service implementations without your clients even noticing.

Now there can be too much of a good thing, and the quality of a software system is certainly not measured by the ratio of interfaces to classes.

When you are planning to implement a FrobnicatorService, think twice before creating an IFrobnicatorService interface and a FrobnicatorServiceImpl. How many different Frobnicator implementations are there going to be? If you do need at least two implementations with distinct behaviour, then go ahead and create the interface. If there's only one implementation, then don't bother with the interface.

Resist the temptation of speculative generalization: "Oh, there might be a performance bottleneck, and in that case an alternative CacheingFrobnicatorServiceImpl might help, so I really should start with a service interface now." Think twice: you ain't gonna need it.

And if you do need the interface, you can still pull it out when you need it. Most IDEs have an automatic refactoring Extract Interface. (Unfortunately, none of the major Java IDEs seems to have the opposite refactoring Merge Interface and Implementation.)

Even in Java EE, the times of interface inflation are gone with EJB 3.1, thanks to the no-interface local view. For a stateless session bean, it is enough to implement

@Stateless
public class FrobnicatorService {
    
    public void frobnicate() {
        ...
    }
}

Every public method of this class will be part of the implicit local business interface.

In OSGi, a service does not need to have a separate interface, you can register any class as a service. Even when using Declarative Services, don't get fooled by the XML syntax of the Service Component Descriptor:

<service>
  <provide interface="com.example.FrobnicatorService"/>
</service>

The provide element has an interface attribute, but the attribute value can be any old class.

The same is true in Spring: There is no rule forcing you to inject Spring beans only via their interface. Any old class will do, even when working with automatic transaction proxies (just make sure that CGLIB is on your classpath).

So remember the KISS principle and kill some of the interfaces you don't really need!

Glassfish and Weld: New builds, new bugs

2010-12-01T09:54:00.000+01:00

My article about memory issues with CDI/Weld seems to have caused rather a wave. Someone (not me) posted a link on DZone, which triggered an unprecedented amount of traffic on this blog, there were discussions both in the Glassfish and Weld communities, and I was very happy to see that Weld appears to have made significant progress in terms of resource usage.

So the good news is: First, things are moving in the right direction with Weld, and second, blogging helps - at least in this case, it seems to have been a lot more effective than my previous bug reports and forum messages.

Unfortunately, there is also some bad news. Several users have already reported regressions with Glassfish 3.1-b29 and b30 related to Weld. My application fails to deploy on b30 with a stack trace which looks like a case of GLASSFISH-13131. So I'd better wait and try again with Glassfish 3.1 milestone 8.

Java EE and Spring: Why I couldn't care less

2010-11-29T22:24:00.000+01:00

Java EE or Spring - for an independent software architect, the question is just as relevant as Catholic or Protestant to an atheist. The question has recently been discussed in a number of articles mostly taking one side or the other:

In this post, I'll try to point out why I'm not fully happy with either of the two.

If there's one thing that matters to me, it's modularity. To put my cards on the table, I'm a strong OSGi supporter, but I'm not even religious about that - in fact, Living without OSGi (which is what I've been doing for nine months now) would be a nice headline for one of my next posts.

Seen from outside, Spring and Java EE share a number of shortcomings:

Preaching to the Converted

When did you last visit the official homepages of Java EE or Spring? Take a moment to click on the links and imagine you've never heard about either. After visiting the homepages, you are none the wiser, all you get is the usual marketing blurb and a couple of newsbites completely meaningless to the uninitiated.

Lack of focus

Java EE and Spring both are animals of the rare species Sus ovipara lactifera velluta unknown in English speaking countries which we call eierlegende Wollmilchsau in German. In plain English, they both try to be everything to everyone and end up doing nothing in particular.

In contrast, OSGi has a clear mission statement The Dynamic Module System for Java and provides a link to its key benefits right on its homepage.

Bloat

Ever heard about lean development? Let's have a look at the official Javadocs:

Java EE 6: 1597 classes
Spring 3.0.5: 2312 classes

Legacy

Backward compatibility is a good thing for veteran users of a framework. Users don't want to change all of their application code just for upgrading to a new framework version.

On the other hand, backward compatibility can be extremely confusing to new users: There's two or three solutions for the same kind of problem, the legacy one is not marked as legacy or deprecated clearly enough, tutorials and example code still use the old style, and you can only resort to your favourite search engine or to trial and error to make things work consistently.

Love Thy User

Don't just list what you can do for your users, ask them what they want you to do for them. Looking at the Java EE and Spring reference documentation, I can see that both frameworks solve lots of problems I've never had in my life.

On the other hand, looking for a solution for some specific standard kind of problem, it is often surprisingly hard to find a comprehensive code example.

User documentation should be structured by user stories, not by technology.

Enough Rope

Both Spring and Java EE give you enough rope to hang yourself. It's easy to write messy and ugly code on top of both frameworks, and I've seen enough of that.

You can write loosely coupled and elegant applications on top of either, but it takes a good architect to set up guidelines and restrict the number of choices.

Development Process Integration

As an application developer, you don't just write code to run on a given framework. You want to run integration tests on your applications, you want to deploy and debug them directly in your favourite IDE, and you want your batch build and continuous integration to be able to deploy your applications.

When choosing a framework, always consider the entire process chain - some lack of tooling for your favourite framework may be reason enough to pick the second best instead.

Modularity

My ideal framework/container/app server - pick any name you like - would allow me to write truly modular applications, enforcing modularity at design time, at compile time, and at run time. The framework itself would be modular, allowing me to pick out those and only those modules actually required by my application.

It would actively support OSGi as the most mature module system for Java, and maybe a few others on top.

I expect some further degree of convergence between Java EE, Spring and OSGi within the next two years or so. There has been a lot slideware and a number of proof-of-concept or early adopter projects, but my feeling is that none of this has reached production quality yet.

To sum up, frameworks in general are alive and kicking, but the age of monolithic all-in-one frameworks is definitely over.

Java EE 6 Integration Testing with Spring

2010-11-28T20:03:00.000+01:00

Half a year ago, I created and published the jeeunit project to fill in some gaps for running integration tests on Java EE 6 components in an embedded EJB container, mainly targeted at Embedded Glassfish 3.x. One of the alternatives I considered and dismissed before writing my own solution was the Spring Test Context.

At the time, I was left with the impression you had to pollute your Java EE components with Spring annotations like @Transactional to make declarative transactions work in the Spring container, and I did not (and still do not) like the idea of writing XML just to create a couple of Java beans and wire them up.

After taking a closer look at Spring 3.0.x these days, I realized a number of facts which are not quite obvious from the Spring reference manual:

You can do most of your Spring configuration (but sadly not all of it!) in Java instead of XML, which makes your application context type-safe and easier to read, and gives you refactoring support from your IDE.
Spring supports some (but not all) of the Java EE 6 annotations as alternatives to its own flavours.
You can run Stateless Session Beans and JPA entities in a Spring container with declarative transactions without introducing dependencies on Spring APIs or annotations.

jeeunit now includes an example project jeeunit-example-test-spring demonstrating this approach. The details are documented in the jeeunit Wiki page Testing on Spring.

This solution is not intended to replace the original in-container-testing approach of jeeunit, because transactional Spring beans and EJBs have different run-time semantics. However, for many integration tests, these differences are irrelevant.

Using the Spring Test Context instead of Embedded Glassfish and jeeunit has two main advantages:

Spring starts noticeably faster than Embedded Glassfish.
Launching a single test from your suite is much easier than with jeeunit. E.g. in Eclipse, you can simply use Run as JUnit Test just as with plain old unit tests.

CDI - A Major Risk Factor in Java EE 6

2010-11-04T15:40:00.002+01:00

Update 18 Dec 2010: This post mainly relates to Glassfish 3.0.1. As of 3.1-b33, the situation has improved a lot.

Evaluating platforms and architectures for a new enterprise web application earlier this year, around March/April, scheduled to be released in 2011, I was bold enough to settle on Java EE 6.

In retrospective, I do not regret this decision, but I had to spend a significant amount of time analyzing and debugging platform problems. Based on this experience with Java EE 6, which may not be representative of course, the only advice I can give is: When Java EE 7 is released, wait a year or two before using it for production, unless you have the time and resources to act as beta tester.

The Java EE 6 specifications were released in December 2009, together with Glassfish v3, the first "production-quality release" of a Java EE 6 server. "Marketing release" would have been more appropriate, since going by the number of bugs in Glassfish 3.0, the main driving factor for the release date was quite obviously not a quality benchmark, but simply the need for Sun as the Java EE champion to release a working server simultaneously with the specifications.

The Problem

Java EE 6 was advertised as the lightest Java Enterprise Edition ever, with Contexts and Dependency Injection (CDI, JSR-299) as one of the most prominent innovations. However, for my project, CDI in the guise of Weld, its reference implementation, turned out to be the largest single source of problems. In addition to a number of functional bugs where injection would not work in certain scenarios, there are at least two severe memory issues:

Weld causes memory leaks on application redeployment.
Even in the first deployment of an application, Weld uses enormous amounts of memory.

The first issue had plagued us for months, and my team had grown accustomed to kill all Glassfish and Eclipse processes after three or four redeployments and to re-initialize their entire workspace, which was rather a pain.
Investigating the memory leaks using the Eclipse Memory Analyzer, I realized that Weld claimed almost 90 % of the heap memory of my application. Most of this was due to instances of org.jboss.weld.introspector.weld.jlr.WeldClassImpl, which appears to be a reflective model of a class inspected by the CDI BeanManager. Some instances of this class were as large as 1 MB.

I should add that our application uses Wicket for the web front end, injecting EJBs into custom Wicket components. Custom components inherit from Wicket base classes with a fairly deep inheritance hierarchy and a large number of methods. This accounts for the size of the reflective models.

Our usage of CDI was fairly basic, so it was just an hour's work to replace all @Inject annotations by Java EE 5 style @EJB annotations. After that, the heap usage of our application was reduced by 90 %, we no longer had any memory leaks, and we finally had a hassle-free edit-save-debug cycle in our development environment.

The figure of 90 % heap usage by Weld was observed on Glassfish 3.0.1, running on a 32-bit JVM on Windows XP.

This week, I retested the same application with Glassfish 3.1-b26 (using a newer version of Weld), running on a 64-bit JVM on Ubuntu 10.04. The result was not as drastic as before, but still about 50 % of my heap memory was consumed by Weld.

Alternatives

This article is mainly about problems with the CDI reference implementation Weld, not about CDI as such. But these implementation details have a ripple effect:

Glassfish 3.x still is the only certified Open Source Java EE 6 server. It uses a newer version of Weld in the current 3.1 promoted builds, but I can see no indication of the Weld issues being solved in time for the 3.1 release.
JBoss AS 6 is likely to be the next in line for full Java EE 6 compliance. However, JBoss also includes Weld, so you're bound to run into the same kind of problems on JBoss.
There are alternative CDI implementations, CanDI from Caucho and OpenWebBeans from Apache. Unfortunately, unlike JPA, there is no pluggable CDI provider interface that would allow you to simply replace Weld by some other CDI implementation on Glassfish or any other server.
I tried running my application on Resin 4.0.8 to see if CanDI was any better than Weld. However, Resin had bugs in the JPA and EJB areas and I was unable to deploy my application, let alone run and monitor it. (Resin only aims at supporting the Java EE 6 Web Profile, but that would have been sufficient for my case.)

Conclusion

All in all, if you want to work with Java EE 6, there is currently no way around Weld. And until Weld reaches production quality, the only way to build production quality applications on Java EE 6 is to avoid using CDI.

Free and Open Source Software is all about choice - so I would really love to see a pluggable CDI provider interface that would allow users to work with Glassfish and some CDI implementation other than Weld.

The second best solution might be to join forces from different CDI project to make Weld stable enough for production usage.

The third option would be another fully Java EE 6 compliant Open Source server (maybe Geronimo?) not including Weld.

No idea if any of these options is realistic, but this is on my wishlist for 2011.

OpenJPA and OSGi

2010-09-19T16:47:00.004+02:00

In my previous review of OSGi support in JPA 2.0 persistence providers, I promised to explain my setup for using OpenJPA in an OSGi environment.

The solution I'm going to outline is by no means final, and in the meantime, my first shot has been greatly simplified using some valuable input from the Apache Aries User mailing list.

In the past, brute force solutions like DynamicImport-Package were more or less the only way to let a JPA provider bundle access all the required classes for a persistence unit from application bundles. Since Release 4.2, the OSGi Enterprise Specification standardizes a whole lot of OSGi service wrappers for well-known Java EE APIs, including JPA, JDBC, JTA and JNDI, so this should be the preferred way of integrating JPA providers into an OSGi environment.

The Apache Aries project provides (partial) implementations for some of the OSGi enterprise services and a whole lot of other things beyond the scope of this article. Fortunately, all their service implementations are nicely modular, so it is quite easy to use just the JPA and JNDI Services from Aries to make OpenJPA work on OSGi, following the OSGi Enterprise spec rather closely (though not fully).

Here is an overview of the required steps:

Get OSGi bundles for OpenJPA and all its dependencies, plus the required Aries bundles.
Publish the datasource to be used by your persistence unit as an OSGi service.
Make sure that your persistence.xml satisfies a number of restrictions.
Run the OpenJPA enhancer and package all classes and metadata of your persistence unit into a bundle with a special manifest header.
Start your OSGi framework. The JPA Service will publish an EntityManagerFactory service for each persistence unit.
In your client bundles, import your persistence bundle, OpenJPA and javax.persistence, get the appropriate EntityManagerFactory from the service registry and use it just like in an unmanaged Java SE environment.

Each step is explained in more detail in the following sections. I'm using Java 1.6.0, Maven 2.2.1 for collecting the dependencies, Eclipse 3.5.2 for doing most of the work, and the corresponding Equinox version as OSGi runtime.

Get the dependencies

The main JAR in the OpenJPA binary distribution is an OSGi bundle, but most of the included dependencies are plain old JARs, so you need to go shopping for osgified versions of all required libraries. All the required bundles for this example can be found in the SpringSource Enterprise Bundle Repository.

The following is the best part of a Maven POM for copying all the dependencies to a local directory target/platform/plugins.

You need to add the OSGi framework, javax.persistence (I copied both of them from Eclipse 3.5.2) and a JDBC driver bundle for your database. Then copy the whole lot to a new directory to be used as target platform for your Eclipse workspace.

<project>
  <properties>
    <aries.version>0.2-incubating</aries.version>
    <output.dir>${project.build.directory}/platform/plugins</output.dir>
  </properties>
  <dependencies>
    <dependency>  
      <groupId>javax.transaction</groupId>  
      <artifactId>com.springsource.javax.transaction</artifactId>  
      <version>1.1.0</version> 
    </dependency>    
    <dependency>
      <groupId>net.sourceforge.serp</groupId>
      <artifactId>com.springsource.serp</artifactId>
      <version>1.13.1</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries</groupId>
      <artifactId>org.apache.aries.util</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jndi</groupId>
      <artifactId>org.apache.aries.jndi.api</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jndi</groupId>
      <artifactId>org.apache.aries.jndi.core</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jndi</groupId>
      <artifactId>org.apache.aries.jndi.url</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jpa</groupId>
      <artifactId>org.apache.aries.jpa.api</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jpa</groupId>
      <artifactId>org.apache.aries.jpa.container</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>com.springsource.org.apache.commons.collections</artifactId>
      <version>3.2.1</version>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>com.springsource.org.apache.commons.lang</artifactId>
      <version>2.2.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>com.springsource.org.apache.commons.pool</artifactId>
      <version>1.5.3</version>
    </dependency>
   <dependency>
      <groupId>org.apache.openjpa</groupId>
      <artifactId>openjpa-all</artifactId>
      <version>2.0.1</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>com.springsource.slf4j.api</artifactId>
      <version>1.5.0</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>com.springsource.slf4j.log4j</artifactId>
      <version>1.5.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.log4j</groupId>
      <artifactId>com.springsource.org.apache.log4j</artifactId>
      <version>1.2.15</version>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-dependency-plugin</artifactId>
        <executions>
          <execution>
            <id>copy-bundles</id>
            <phase>package</phase>
            <goals>
              <goal>copy-dependencies</goal>
            </goals>
            <configuration>
              <outputDirectory>${output.dir}</outputDirectory>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
  <repositories>
    <repository>
      <id>com.springsource.repository.bundles.external</id>
      <name>SpringSource Enterprise Bundle Repository - External Bundle Releases</name>
      <url>http://repository.springsource.com/maven/bundles/external</url>
    </repository>
  </repositories>

</project>

Register a data source service

A persistence unit requires a data source, either from a javax.sql.DataSource or from a JDBC driver URL. This may be a matter of taste - in this article, I'm focusing on the DataSource.

Create a DataSource for your database using any suitable driver and register it as an OSGi service with the javax.sql.DataSource interface and optionally some additional identifying properties of your choice, e.g. name=myds, in case you have more than one DataSource in your system.

It is useful to do this from a separate bundle, so that this bundle will be the only one in your system with a direct dependency on the JDBC driver.

Adapt your persistence.xml

If you have a persistence.xml which works on Java SE oder Java EE, you will probably have to modify it to make it work with OSGi and Aries. Here is an example:

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
  <persistence-unit name="myapp" transaction-type="RESOURCE_LOCAL">
  <non-jta-data-source>osgi:service/javax.sql.DataSource</non-jta-data-source>   
    <mapping-file>META-INF/orm.xml</mapping-file>
    <exclude-unlisted-classes>true</exclude-unlisted-classes>
  </persistence-unit>
</persistence>

First, you need to specify the RESOURCE_LOCAL transaction type explicitly, or else OpenJPA will complain about a missing transaction manager. This is because Aries creates a container EntityManagerFactory and not a stand-alone one.

Second, your data source has to be a non-jta-data-source. Note the special syntax for accessing your data source as an OSGi service via JNDI. To make this work, you need to include the Aries JNDI Service bundles. To select a specific data source by properties, you can append an LDAP filter to the JNDI URL, e.g. osgi:service/javax.sql.DataSource/(name=myds).

Third, you need to prevent the persistence provider from scanning your persistence unit for entity classes, since this may lead to classloader problems. To do so, either list all your classes explicitly if you use JPA annotations, or include all mapping files if you use XML metadata and set exclude-unlisted-classes to true to prevent the provider from scanning your bundle.

If you get funny exceptions like

org.apache.openjpa.persistence.PersistenceException: Error extracting class information from "bundleresource://23.fwk1421571929/".
Caused by: java.io.FileNotFoundException: /myapp/plugins/test.openjpa/bin (Is a directory)

then check if you have set this option. (This problem may be specific to Eclipse, but I'm not sure about that.)

Package your persistence unit bundle

To turn your persistence unit into a persistence bundle, make sure you have all mapping data resources and all entity classes included. In addition to the usual OSGi headers, insert the following one into your manifest:

Meta-Persistence: META-INF/persistence.xml

This is the hook for Aries to discover your persistence bundle using the extender pattern. If all goes well, an EntityManagerFactory will be registered as an OSGi service.

In theory, your persistence bundle should only depend on javax.persistence and not on org.apache.openjpa. In practice, you currently need to include this unwanted import, or else you will get exceptions of the following kind:

java.lang.NoClassDefFoundError: org/apache/openjpa/enhance/PersistenceCapable

Apparently, OpenJPA tries to load one of its own classes via your persistence bundle class loader, which is incorrect.

Use the EntityManagerFactory service in your clients

Finally, to work with your persistence unit from client bundles, just get hold of the appropriate EntityManagerFactory service any way you like, using the filter (osgi.unit.name=myapp).

Update 26 Sep 2010: I have submitted some sample code for inclusion into the OpenJPA test suite. The sample is attached to OPENJPA-1815.

OSGi Support in JPA 2.0 Persistence Providers

2010-09-15T23:25:00.000+02:00

About three years ago, I started working with Hibernate in an OSGi environment, and at that time, I had to roll my own solution for resolving all sorts of classloader issues.

Since then, with OSGi 4.2 and JPA 2.0, there have been major specification releases, followed by implementations; OSGi is growing ever more popular, and the current release of the JPA standard has integrated a lot of functionality which used to be available in proprietary API extensions only.

The OSGi 4.2 Enterprise Specification includes JPA, JTA and JDBC services, so in theory, using JPA in an OSGi environment should stop being an issue, as soon as this specification is supported by all OSGi and JPA implementations.

In practice, the specifications are not yet fully covered by the available implementations, and the degree of OSGi support varies a lot.

In the past few months, I have worked with the current versions of the three certified JPA 2.0 persistence providers, and in my opinion, the ranking for OSGi support is quite clear:

Eclipselink
OpenJPA
Hibernate

Some more details on each provider:

Eclipselink

Eclipselink has a fully osgified distribution containing OSGi bundles of Eclipselink itself and all dependencies. To make your entity classes visible to Eclipselink, all you need to do is add a JPA-PersistenceUnits manifest header to each bundle containing one or more persistence units. This is enough for Eclipselink to load all your XML metadata and entity classes.

No ugly hacks, no buddy policies or fragments, it just works out of the box.

So for a quick start with JPA under OSGi, Eclipslink is definitely the best choice, even though the OSGi 4.2 JPA Service Specification is not yet fully supported (for instance, the Meta-Persistence header does not work), but this will be of little or no concern to most users.

Leaving OSGi aside, I have been bitten by a number of bugs in Eclipselink, discussed in earlier articles. If you do not use any of these features, you may be very happy with Eclipselink. For me, this set of bugs is currently a no-go, so I have migrated my OSGi applications from Eclipselink to OpenJPA.

OpenJPA

The silver medal goes to OpenJPA: OSGi does not appear to be on the top of the agenda of the OpenJPA core developers, but since OpenJPA is used under OSGi by Aries, another Apache project, there is at least a certain level of OSGi support.

In the binary distribution, the OpenJPA aggregate JAR is an OSGi bundle, but most of the dependencies are plain old JARs, so you need to find osgified versions of the dependencies on your own. The same goes for the Maven artifacts.

Once you have downloaded the appropriate OSGi bundles, you still have to do some extra work to ensure that OpenJPA finds your persistence units and your entity classes, and there are a couple of minor issues with the enhancer and with user-defined value handlers.

I will explain the details of my setup and some of the problems that should be addressed in OpenJPA in my next post.

Hibernate

Sadly, regarding OSGi support, Hibernate 3.5.x is not much different from the 3.2.x release I discussed in my earlier articles. The distribution contains plain old JARs, no OSGi bundles. The latest osgified version available from the SpringSource Enterprise Bundle Repository is 3.4.0, and the SpringSource versions never used to work for me, so I expect that the steps described in my 2008 article are still required to make Hibernate work on OSGi.

Disclaimer: I did not try to make Hibernate 3.5 work on OSGi, due to its lack of support of a number of JPA 2.0 features used heavily by my applications. For this reason, I have stopped using Hibernate, both on OSGi and on Java EE.

JPA 2.0: Ordered Collections

2010-09-12T17:23:00.000+02:00

In earlier posts, I wrote about problems with persistent maps, a new feature introduced in JPA 2.0. Ordered collections with explicit order columns are another JPA 2.0 feature which, again, is not yet fully robust in all JPA 2.0 compliant and TCK tested implementations.

I'm currently working with a simple JPA model for OpenStreetMap (OSM). There are ways and nodes, and each way has a sequence of nodes. Of course the order of the nodes is important, and a node may be used more than once for a given way.

Thus, the nodes collection has to be a List, not a Set, and we need an explicit order column specifying the sequence number of the nodes along the way. Sorting the nodes by ID would not make any sense, obviously.

Here is a snippet from the model:

@Entity
@Table(name = "ways")
public class OsmWay
{
    @Id
    private long id;
    
    @ManyToMany(cascade = CascadeType.ALL)
    @OrderColumn(name = "sequence_id")
    @JoinTable(name = "way_nodes", 
               joinColumns = @JoinColumn(name = "id"), 
               inverseJoinColumns = @JoinColumn(name = "node_id"))
    private List<OsmNode> nodes = new ArrayList<OsmNode>();
}

This entity is represented by two tables:

CREATE TABLE ways
(
  id bigint NOT NULL
);

CREATE TABLE way_nodes
(
  id bigint NOT NULL,
  node_id bigint NOT NULL,
  sequence_id integer,
);

I've tested this scenario on Hibernate 3.5.3, Eclipselink 2.1.1 and OpenJPA 2.0.1: Hibernate and OpenJPA pass, Eclipselink fails with 2 issues.

First, the generated DDL is incorrect: There is a PRIMARY KEY (id, node_id) for way_nodes. This should be (id, sequence_id).

Second, the order of the list items is not maintained in all contexts. It is ok when the collection is lazily loaded, e.g.

OsmWay way = em.find(OsmWay.class, wayId);
List<OsmNode> nodes = way.getNodes();

but it is broken when using a fetch join:

String jpql = "select distinct w from OsmWay w join fetch w.nodes";
TypedQuery<OsmWay> query = em.createQuery(jpql, OsmWay.class);
List<OsmWay> ways = query.getResultList();
OsmWay way = ways.get(0);
List<OsmNode> nodes = way.getNodes();

The funny thing is, the nodes are neither ordered by sequence_id nor by node_id. In my test case, the way has 5 nodes, one of which appears twice in different positions.

So here is some more evidence for my claim that the JPA 2.0 TCK is insufficient.

Ubuntu VPN Client Setup

2010-08-08T17:48:00.000+02:00

Not a Java topic this time: Setting up an IPSec VPN client on my Linux box was a complete nightmare. The final solution is rather easy, but I wasted hours and hours trying different clients with different configurations which simply did not work...

This is the environment:

Linux client machine running Kubuntu 10.04.
Network interface eth0 on client configured via DHCP from a Netgear Router.
The router acts as DNS server.
The home network is 192.168.0.0/24.
The company network has a FortiGate firewall with an IPSec VPN.
The IPSec tunnel uses a pre-shared key and XAUTH username/password authentication.
The company network is 172.16.0.0/16, VPN clients get a virtual IP adress from 172.19.0.0/24 via DHCP-over-IPSec.

The problem:

There is no official Linux VPN client for FortiGate. (This is rather ironic, given that Fortinet products include GPLed Linux code, which became public after a GPL violation injunction in 2005.)
Ubuntu has just too many too buggy VPN clients and too little readable documentation on VPN setup. (I tried KNetworkManager, kvpnc, ike and openswan, mostly getting only error messages I could not make sense of. ike did work ocasionally, but most of the time, the incoming packets from the tunnel would not reach my client port.)

My solution:

strongswan for the IPSec tunnel
dnsmasq for merging the public and private DNS zones
eth0 now uses manual configuration instead of DHCP

Now for the details: The XAUTH credentials and the pre-shared key are stored in /etc/ipsec.secrets:

: XAUTH myusername "MyPassword"
: PSK "MyPreSharedKey"

The general and connection specific settings for IPSec are in /etc/ipsec.conf:

config setup
     nat_traversal=yes
                                                             
conn office
     left=%defaultroute
     leftsourceip=172.19.0.99
     right=firewall.example.com
     rightsubnet=172.16.0.0/16
     auto=add
     authby=xauthpsk

The only flaw is the explicit virtual IP address: I tried using leftsourceip=%config to let the client use an address assigned by the server, but this did not work as FortiGate does not support the ModeCfg protocol (at least not in the firmware version we are currently using), and I could not find out how to set up DHCP-over-IPSec with strongswan, so I ended up using an explicit address not part of the DHCP pool.

After editing these files and starting the tunnel via

ipsec restart
ipsec up office

I could successfully ping addresses from the office network.

In contrast both to the Windows FortiClient and the Shrew VPN Client (ikea), strongswan does not require a virtual network device and does not change the routing table. You can see the IPSec tunnel and the corresponding policies using the commands

ip xfrm state
ip xfrm policy

The log output from the strongswan IKEv1 daemon named pluto can be found in /etc/auth.log.

Now for DNS: Of course I don't want all my DNS requests for things like schweinebildchen.com to hit the office DNS server while the tunnel is up, so I'm using dnsmasq as a simple DNS proxy which redirects all DNS requests from my local machine either to my home router or to the office DNS server via the tunnel, depending on the domain.

The dnsmasq configuration in /etc/dnsmasq.conf is as simple as this:

server=/example.com/172.16.0.2
server=192.168.0.1

Finally, I changed the configuration of my network interface eth0 from auto to manual, using the IP address 192.168.0.5 and set this as nameserver in /etc/resolv.conf.

This means that all DNS requests are first handled by dnsmasq on my local machine, requests for example.com get forwarded to the office DNS server (via the IPSec tunnel), and everything else gets routed to my Netgear router (and from there to my ISP).

A Fish in the Clouds

2010-07-27T21:16:00.000+02:00

How long does it take to find a hosting company and get an enterprise web application up and running from scratch? Could be weeks if there are enough lawyers and pointy-haired bosses involved. And even if you can directly talk to the providers, it usually takes a couple of days to request quotes and compare them.

That's the way we started for our current project, but now we've decided to have a go at cloud computing. (So at last this adds another buzzword to my CV...)

I vaguely remembered Arun Gupta's blog about Glassfish on Amazon EC2, and using that together with the Ubuntu EC2 Starter's Guide, it just took me about half an hour to set up a virtual machine in the Amazon Cloud with Ubuntu Server 10.04, JDK 1.6.0_20 and Glassfish 3.0.1, start the Glassfish domain and access the admin console from my local web browser.

And downloading the 78 M Glassfish zip file on the EC2 instance from Oracle's server took less than 2 seconds...

Not bad for a start. I would have expected to spend about a day to get as far as that.

I was also using Ubuntu 10.04 on my local machine, working with the command line interface from the
ec2-api-tools Ubuntu package most of the time.

Of course it will take some more time to set up the database server, deploy our web app and to secure the system. And it's too early to tell if we really require all the elasticity of EC2, which is likely to be more expensive in the long run than a conventional dedicated server cluster.

At any rate, installing a site in virtually no time and running it for less than 10 cents per hour is rather impressive. Thumbs up for Amazon Web Services!

Building a Java EE 6 Web Application with Eclipse Helios, Maven and Glassfish

2010-07-26T23:25:00.000+02:00

This is a short tutorial showing how to use Eclipse 3.6 (Helios) with Glassfish and Maven for building and running a Java EE 6 web application from a multi-module source tree.

We are going to import and run the Wicket gmap2-examples, demonstrating the use of Google Maps via Apache Wicket, but this is purely incidental - even if you prefer mainstream JSF or another web framework to Wicket, you may find this tutorial useful. There are absolutely no Wicket specifics involved; gmap2 was just a handy small but non-trivial example.

If you haven't worked with Maven before, you will get an idea how Maven can save you a lot of work managing the third-party dependencies of your project automatically.

This is what you'll see in the end:

There's quite a few things on your shopping list before you can start. If you are reading this, you probably already have the following on your disk:

You'll need to install these features into Eclipse:

Maven Integration for Eclipse (also known as m2eclipse)
Maven Integration for Eclipse (Extras)
Glassfish Java EE Application Server Plugin for Eclipse

Eclipse 3.6 has a new way of installing plugins via Help | Eclipse Marketplace. This is an integrated web client which lets you search for plugins by free text, so you no longer have to copy and paste update site URLs into the Update Manager - but you can still do so if you prefer.

The m2eclipse book has step-by-step instructions and screenshots explaining the installation process: start here with the m2eclipse installation. When you get to the Extras, the only ones required for the rest of this tutorial are Maven Integration for WTP and Maven SCM Integration. The WTP integration will let Eclipse recognize your Maven projects with war packaging as Dynamic Web Projects.

The Maven SCM Integration lets you fetch Maven projects directly from source code repositories like Subversion, Mercurial or others. Note that this integration simply invokes the corresponding command line clients like /usr/bin/svn.

Having installed m2clipse via the Eclipse Marketplace, use the same procedure to install the Glassfish Plugin.

After restarting Eclipse, set your Glassfish preferences via Window | Preferences | Glassfish Preferences. For this tutorial, you should check Start the Glassfish Enterprise Server in verbose mode and uncheck the other items.

Next, define a server runtime environment via Window | Preferences | Server | Runtime Environment | Add... Select the server type Glassfish | Glassfish Server Open Source Edition 3 (Java EE 6), check Create a new local server and click Next. Select your Glassfish installation directory - this is required to be the parent of the modules and domains directories, e.g. /home/hwellmann/glassfish-3.0.1/glassfishv3/glassfish. Click Next and fill in the domain name and the administrator credentials. If you did not change the Glassfish defaults, you can probably just click on Finish.

The Servers view has opened automatically. You can select your server and click on the Run button to launch Glassfish from Eclipse. The Eclipse plugin launches Glassfish indirectly via an asadmin subprocess.

You will see some Glassfish log messages in the console. To stop Glassfish, do not hit the stop button in the Console view. This will just kill the asadmin process, but not Glassfish itself, and Eclipse and Glassfish will get terribly out of sync. Make sure to select the Servers view and hit the stop button there to avoid trouble.

Now for the interesting part: Let us import and build the gmap2 example project.

This project has a parent project with two subprojects, or modules in Maven terms. Maven requires you to store the module subprojects of a parent project in subdirectories of the parent directory. Eclipse normally cannot handle overlapping or nested directory structures for projects in the same workspace. Fortunately, m2eclipse works some magic to flatten the project structure, making Eclipse happy.

To import the example sources directly from the Subversion repository, select File | Import... | Maven | Check out Maven Projects from SCM and click Next. Fill in the SCM URL

https://wicket-stuff.svn.sourceforge.net/svnroot/wicket-stuff/tags/wicketstuff-core-1.4.9.2/gmap2-parent

and select the SCM provider svn, then click Next and Finish.

After a while, you will see three projects in your workspace:

gmap2
gmap2-examples
gmap2-parent

The parent project gmap2-parent has two subfolders gmap2 and gmap2-examples which are also represented as top-level Eclipse projects - this is a special feature of the m2eclipse integration.

The gmap2-examples subproject is a web application, as indicated by the Maven packaging type war. Using this and other information specified in the Maven POM, m2eclipse was able to turn this Eclipse project into a Dynamic Web Modules project - have a look at the Project Facets in the project properties to convince yourself.

Opening the project, you will notice two classpath containers Maven Dependencies and Web App Libraries. The latter contains just one item gmap2 with an open folder icon, indicating a reference to another subproject in our workspace that will go into WEB-INF/lib. The Maven Dependencies container displays all JARs required by our project, e.g. wicket-1.4.9.jar and slf4j-api-1.5.8.jar. All these were downloaded automatically by Maven into your local Maven repository on your hard drive, and m2eclipse makes Eclipse reference them from that location.

We are now ready to launch the web app: Select the gmap2-examples project folder and then Run As... | Run on Server from the context menu. Select the existing Glassfish server you created before and click Finish. After a while, you see the gmap2-examples welcome page in a web browser window.

Click on one of the links, e.g. marker listener, to see Google Maps in Firefox in Eclipse via Wicket on Glassfish on Java on Linux. (OK, maybe it's not Firefox or Linux on your machine...)

Note that the Java compilation and the WAR assembly and deployment were done by Eclipse, not by Maven.

However, you can also run a Maven build from Eclipse. To build our entire project hierarchy, select gmap2-parent and open Run As | Maven build... from the context menu. Enter the goals clean install and click Run. You will see the Maven messages in the console window. When Maven has finished, click Refresh (F5) on gmap2-parent, and open gmap2-examples/target to see the WAR file compiled by Maven.

JPA 2.0: Querying a Map

2010-07-20T21:40:00.000+02:00

Welcome back to more merriment with Maps in JPA 2.0!

After watching 3 out of 4 persistence providers choke on a model with a map in the previous post, let us now continue our experiments and see how our guinea pigs can handle JPQL queries for maps.

Recall that the JPQL query language has three special operators for building map queries: KEY(), VALUE() and ENTRY().

Now let us try and run the following query on a slightly modified model, compared to the previous post.

select m.text from MultilingualString s join s.map m where KEY(m) = 'de'

The corresponding model is:

@Embeddable
public class LocalizedString {

    private String language;

    private String text;

}

@Entity
@Table(schema = "jpa", name = "multilingual_string")
public class MultilingualString {

    @Id
    @GeneratedValue(strategy = GenerationType.SEQUENCE)
    @Column(name = "string_id")
    private long id;

    @ElementCollection(fetch=FetchType.EAGER)
    @MapKeyColumn(name = "language_key")
    @CollectionTable(schema = "jpa", name = "multilingual_string_map", 
                     joinColumns = @JoinColumn(name = "string_id"))
    private Map<String, LocalizedString> map = new HashMap<String, LocalizedString>();
}

This time I've changed the model so that the map key is stored in its own column, which gives Hibernate and Eclipselink at least a chance to digest the model and proceed to the query. OpenJPA is fine with either version of the model.

DataNucleus is out of the game by now. I even tried replacing the @Embeddable by an @Entity and a few other things to cheat it into accepting my model, but in the end I gave up.

Now, Ladies and Gentleman, the winner and sole survivor is: OpenJPA again!

Both Hibernate and Eclipselink fail, merrily throwing exceptions. Hibernate only seems to have stubbed out the KEY() and VALUE() operators in their parser code (see HHH-5396 for the gory details and elaborate stack traces).

And Eclipselink's famous last words are:

Error compiling the query [select m.text from MultilingualString s join s.map m where KEY(m) = 'de'], 
line 1, column 9: unknown state or association field [text] of class [LocalizedString].

Not sure what the poor soul is trying to tell me.

To sum up: Should you ever consider working with persistent maps à la JPA 2.0, beware! Here be dragons...

JPA 2.0: Mapping a Map

2010-07-17T15:24:00.001+02:00

JPA 2.0 has added support for persistent maps where keys and values may be any combination of basic types, embeddables or entities.

Let's start with a use case:

The Use Case

In an internationalized application, working with plain old Strings is not enough, sometimes you also need to know the language of a string, and given a string in English, you may need to find an equivalent string in German.

So you come up with a LocalizedString, which is nothing but a plain old String together with a language code, and then you build a MultilingualString as a map of language codes to LocalizedStrings. Since you want to reuse LocalizedStrings in other contexts, and you don't need to address them individually, you model them as an embeddable class, not as an entity.

The special thing about this map is that the keys are part of the value. The map contents look like

'de' -> ('de', 'Hallo')
'en' -> ('en', 'Hello')

The Model

This is the resulting model:

[Update 20 July 2010: There is a slight misconception in my model as pointed out by Mike Keith in his first comment on this post. Editing the post in-place would turn the comments meaningless, so I think I'd better leave the original text unchanged and insert a few Editor's Notes. The @MapKey annotation below should be replaced by @MapKeyColumn(name = "language", insertable = false, updatable = false) to make the model JPA 2.0 compliant.]

@Embeddable
public class LocalizedString {

    private String language;

    private String text;

    public LocalizedString() {}

    public LocalizedString(String language, String text) {
        this.language = language;
        this.text = text;
    }
    
    // autogenerated getters and setters, hashCode(), equals()
}

@Entity
@Table(schema = "jpa", name = "multilingual_string")
public class MultilingualString {

    @Id
    @GeneratedValue(strategy = GenerationType.SEQUENCE)
    @Column(name = "string_id")
    private long id;

    @ElementCollection(fetch=FetchType.EAGER)
    @MapKey(name = "language")
    @CollectionTable(schema = "jpa", name = "multilingual_string_map", 
                     joinColumns = @JoinColumn(name = "string_id"))
    private Map<String, LocalizedString> map = new HashMap<String, LocalizedString>();

    public MultilingualString() {}
    
    public MultilingualString(String lang, String text) {
        addText(lang, text);
    }
    
    public void addText(String lang, String text) {
        map.put(lang, new LocalizedString(lang, text));
    }

    public String getText(String lang) {
        if (map.containsKey(lang)) {
            return map.get(lang).getText();
        }
        return null;
    }
    
    // autogenerated getters and setters, hashCode(), equals()
}

The SQL statements for creating the corresponding tables:

CREATE TABLE jpa.multilingual_string
(
  string_id bigint NOT NULL,
  CONSTRAINT multilingual_string_pkey PRIMARY KEY (string_id)
)

CREATE TABLE jpa.multilingual_string_map
(
  string_id bigint,
  language character varying(255) NOT NULL,
  text character varying(255)
)

The Specification

The most important and most difficult annotation in this example is @MapKey. According to JSR-317, section 2.1.7 Map Keys:

If the map key type is a basic type, the MapKeyColumn annotation can be used to specify the column mapping for the map key. [...]
The MapKey annotation is used to specify the special case where the map key is itself the primary key or a persistent field or property of the entity that is the value of the map.

Unfortunately, in our case it is not quite clear whether we should use @MapKey or @MapKeyColumn to define the table column for our map key. Our map key is a basic type and our map value is not an entity, so this seems to imply we should use @MapKeyColumn.

On the other hand, our key is a persistent field of the map value, and I think the whole point of the @MapKey annotation is to indicate the fact that we simply reuse a property of the map value as the map key, so we do not need to provide an extra table column, as the given property is already mapped to a column.

The way I see it, replacing @MapKey by @MapKeyColumn(name = "language_key") - note the _key suffix! - is also legal, but then we get a different table model and different semantics: The table jpa.multilingual_string_map would have a fourth column language_key, this language_key would not necessarily have to be equal to the language of the map value.

Another open question: Is it legal to write @MapKeyColumn(name = "language")? If so, this should indicate that the language column is to be used as the map key, so this would be equivalent to the @MapKey annotation. On the other hand, you might say that this annotation indicates that the application is free to use map keys that are independent of the map values, so this contract would be violated if the column name indicated by the annotation is already mapped.

The Persistence Providers

I've tried implementing this example with the current versions of Hibernate, Eclipselink, OpenJPA and DataNucleus. I did not succeed with any of them. Only OpenJPA provided a workable solution using @MapKeyColumn, but as I said, I'm not sure if this usage is really intended by the specification.

[Update 20 July 2010: With the corrected model, the updated verdict is: Only OpenJPA passes the test, the other three bail out for various reasons.]

Let's look at the contestants in turn:

Hibernate

Using the mapping defined above, Hibernate 3.5.3-Final complains:

org.hibernate.AnnotationException: Associated class not found: LocalizedString

Apparently Hibernate is expecting the map value to be an entity not an embeddable.

Using @MapKeyColumn(name = "language"), the exception is

org.hibernate.MappingException: Repeated column in mapping for collection: MultilingualString.map column: language

Finally, with @MapKeyColumn(name = "language_key"), Hibernate no longer complains about duplicate columns, but I end up with a redundant table column in my database which I was trying to avoid.

Another problem with Hibernate is different behaviour when working with XML mapping data instead of annotations (which is what I prefer for various reasons, but that's a topic for another post).

Using XML metadata for this example, Hibernate happily ignores the table names from the metadata and simply uses the default names. I filed a bug report in April 2010 (HHH-5136), with no reaction ever since.

Eclipselink

Using Eclipselink 2.1.0, I simply get a rather cryptic exception

java.lang.NullPointerException
 at org.eclipse.persistence.internal.queries.MapContainerPolicy.compareKeys(MapContainerPolicy.java:234)

With @MapKeyColumn=(name = "language"), Eclipselink also complains about a duplicate column, and changing the name to language_key, my test finally passes, at the expense of a redundant column, as with Hibernate.

OpenJPA

With OpenJPA 2.0.0, the message is

org.apache.openjpa.persistence.ArgumentException: Map field "MultilingualString.map" is attempting to use a map table, 
but its key is mapped by another field.  Use an inverse key or join table mapping.

which I can't make sense of. Switching to @MapKeyColumn=(name = "language"), the new message is

org.apache.openjpa.persistence.ArgumentException: 
"LocalizedString.text" declares a column that is not compatible with the expected type "varchar".

Its seems OpenJPA is confused by the column name text which sounds like a column data type. After adding @Column(name = "_text") to LocalizedString.text, my test case works and my database table only has three columns.

DataNucleus

DataNucleus 2.1.1 complains

javax.persistence.PersistenceException: Persistent class "LocalizedString" has no table in the database, 
but the operation requires it. Please check the specification of the MetaData for this class.

I'm getting the same message with all three variants of the annotation, so it appears that DataNucleus simply cannot handle embeddable map value and expects them to be entities.

Conclusion

Mapping maps with JPA is much harder than you would think, both for the user and for the implementor. Hibernate, Eclipselink and OpenJPA have all passed the JPA TCK. DataNucleus would have liked to do so, but they have not yet been granted access to the TCK.

All four implementors failed this simple map example to various degrees, which implies that there are features in the JPA 2.0 specification which are not sufficiently covered by the TCK.

An Open Source TCK for JPA would help in detecting and eliminating such gaps instead of leaving that to the initiative of individuals.

JPA 2.0 Frustration

2010-07-16T22:34:00.000+02:00

JPA 2.0 is one of the areas where Java EE 6 can make your life a lot easier compared to Java EE 5 - at least if your life is somehow connected to Java software development.

So much for the marketing blurb. In practice, working with JPA 2.0 means

trying to understand a specification (JSR 317) which may be quite challenging to read for the average developer and occasionally somewhat too vague even for experts
making sense of obscure stack traces from the JPA provider of your choice
discovering numerous bugs and omissions in implementations claiming to be JPA 2.0 compliant.

If you think this sounds fun, then read on...

Actually, this post is just an introductory note to a series of articles on specific use cases that seem to be particularly hard to get right.

I'm not unhappy about the spec as such - Object Relational Mapping (ORM) is a challenging topic and not exactly the stuff you expect first year computer science students to understand. JPA 2.0 narrows the gap between the JPA standard and vendor-specific extensions or native ORM features.

Still, there are some areas not covered by the standard where you have to fall back to vendor extensions or even write your own code: for instance, I would really like to see more flexible support for enum types, a standard for user-defined column types and an addition for spatial objects and queries, based on OGC standards.

The main source of frustration with JPA 2.0 is simply the lack of specification compliance of the available implementations. Implementing any but the most trivial persistence mappings and queries at application level can require hours of trial and error to get the expected results. Yes, most of the time the problem may be in your application code. But with JPA 2.0, chances are really high that your persistence provider has a bug.

The situation is not helped by the policy of Sun/Oracle/the JCP (I'm not really sure who's in charge of that) to keep the JPA TCK (Technology Compliance Kit) under a non-disclosure agreement - see this blog post from DataNucleus, which was linked in a related thread in the Glassfish forums recently. (DataNucleus is a JPA implementor not beloging to the inner circle of the JCP.)

So far, I've looked at

Hibernate
Eclipselink
OpenJPA
DataNucleus,

and sadly, I've had problems with all of them. I'm going to provide specific examples in the following posts.

Early Adopter

2010-06-26T22:55:00.000+02:00

My mum keeps telling me I used to be gifted child. Of course I don't remember everything I did at the age of three, but I seem to have signed up for the java.net forums at that time. Quite impressive!

Java XML Binding with Property Change Support

2010-05-14T19:44:00.000+02:00

In an earlier post, I was talking about generating Java classes with xjc with property change support which is not enabled by default. This post explains the details, an in fact the Maven build explained below is a lot a easier than my original approch with Ant which required some downloads to set things up.

The general idea is same as described by Kohsuke Kawaguchi in a post of 2007, but his POM is broken with the current stage of the Maven repositories. In addition, there is a tweak with the PropertyListener customization.

So here is a little example schema:

<?xml version="1.0" encoding="UTF-8"?>
<schema xmlns="http://www.w3.org/2001/XMLSchema" xmlns:tns="http://www.example.org/person/" targetNamespace="http://www.example.org/person/">
    <complexType name="Person">
      <sequence>
        <element name="firstName" type="string"></element>
        <element name="lastName" type="string"></element>
        <element name="address" type="tns:Address"></element>
      </sequence>
    </complexType>

    <complexType name="Address">
      <sequence>
        <element name="street" type="string"></element>
        <element name="houseNumber" type="string"></element>
        <element name="city" type="string"></element>
        <element name="postalCode" type="string"></element>
        <element name="country" type="string"></element>
      </sequence>
    </complexType>
</schema>

We use the following file bindings.xjb to customize the Java classes generated by xjc. We make all classes serializable and include support for PropertyChangeListeners.

<?xml version="1.0" encoding ="UTF-8"?>
<jaxb:bindings
    schemaLocation="person.xsd"
    version="2.1"
    xmlns:li="http://jaxb.dev.java.net/plugin/listener-injector"
    xmlns:jaxb="http://java.sun.com/xml/ns/jaxb"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <jaxb:bindings node="/xs:schema">
    <jaxb:globalBindings>
      <jaxb:serializable uid="1" />
    </jaxb:globalBindings>
    <li:listener>java.beans.PropertyChangeListener</li:listener>    
  </jaxb:bindings>
</jaxb:bindings>

The following POM takes care of the code generation:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>jaxb-properties</groupId>
  <artifactId>jaxb-properties</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <name>JAXB PropertyListener Demo</name>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>2.3</version>
        <configuration>
          <source>1.6</source>
          <target>1.6</target>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.jvnet.jaxb2.maven2</groupId>
        <artifactId>maven-jaxb2-plugin</artifactId>
        <version>0.7.3</version>
        <executions>
          <execution>
            <goals>
              <goal>generate</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <extension>true</extension>
          <generatePackage>com.example.jaxb</generatePackage>
          <schemaIncludes>
            <schemaInclude>person.xsd</schemaInclude>
          </schemaIncludes>
          <bindingIncludes>
            <bindingInclude>bindings.xjb</bindingInclude>
          </bindingIncludes>
          <args>
            <arg>-Xinject-listener-code</arg>
          </args>
        </configuration>
        <dependencies>
          <dependency>
            <groupId>com.sun.xml.bind</groupId>
            <artifactId>jaxb-xjc</artifactId>
            <version>2.1.13</version>
          </dependency>
          <dependency>
            <groupId>org.jvnet.jaxb2-commons</groupId>
            <artifactId>property-listener-injector</artifactId>
            <version>1.1-SNAPSHOT</version>
            <exclusions>
              <exclusion>
                <groupId>com.sun.xml.bind</groupId>
                <artifactId>jaxb-xjc</artifactId>
              </exclusion>
            </exclusions>
          </dependency>
        </dependencies>
      </plugin>

    </plugins>
  </build>
</project>

You do need to use the 1.1-SNAPSHOT version of the property-listener-injector, as the 1.0 release only supports VetoableChangeListeners and not the PropertyChangeListeners we want to use. The <li:listener> element in the bindings file is used to define the listener class.

It is important to exclude the jaxb-xjc dependency from the property-listener-injector, or else it will try to download an non-existing snapshot version. Looks like something is broken in the java.net repository, but then again, working with snapshot releases is always dangerous....

Glassfish Logging with slf4j

2010-05-13T18:38:00.002+02:00

Update 18 Dec 2010: There is an extended version of this article.

It can be annoying to have each third-party library or framework in your application logging to a different logfile. slf4j with its various adapters enables you to collect logging events from various APIs (org.slf4j.api, java.util.logging, org.apache.commons.logging) and redirect them to a logging backend of your choice.

I've always preferred log4j or, more recently, logback over java.util.logging, so after working with Glassfish v3 for a while, I tried to tweak it to use logback over slf4j and the jul-to-slf4j bridge.

To redirect java.util.logging used by Glassfish v3, you put the following libs on your classpath:

jul-to-slf4j.jar
slf4j-api.jar
logback-classic.jar
logback-core.jar

(You can replace the logback libs by any other supported backend like log4j, see the slf4j docs for more details.)
The problem with Glassfish is:

You need to take care of its class loader hierarchy and make sure that the logging jars get picked up early enough.
Glassfish does some all-too-clever logger manipulation in its LogManagerService which will get in your way if you don't like the Glassfish defaults: It redirects all System.out messages to a logger.

I did not manage to fully supersede Glassfish logging by slf4j and logback, but at least I can copy all Glassfish log events to my logback application log like this:

Put the logging libs in [install-root]/lib/endorsed.
Build a JAR containing your logback.xml configuration and put it in the same place.
Edit an entry in [instance-root]/config/logging.properties, setting
handlers = org.slf4j.bridge.SLF4JBridgeHandler

Now if you omit a ConsoleAppender from your logging configuration, the Glassfish server.log will no longer contain any log output from your application. It will still contain the log messages from Glassfish itself, but they will also be included in your own logback logfile.

jeeunit: In-Container Integration Testing for Java EE 6

2010-05-09T22:05:00.002+02:00

Recently, I've started working with Java EE 6 in general and Glassfish v3 in particular. Some software engineering best practices do not really depend on the framework or even the language you work with, so I was looking for a convenient way of doing automatic integration tests.

I've been in the habit of using JUnit not just for unit tests, but also for integration or system tests, so this was a natural starting point.

Traditionally, Java EE containers were heavy-weight machinery so that people preferred writing their tests to run out-of-container, paying the price of emulating or mocking some of the container functionality.

Maybe I've been looking in the wrong places, but most of the out-of-container testing approaches like ejb3unit seem to carry more baggage than the container itself, at least when it comes to working with Glassfish v3.

Anyway, as I could not find a ready-to-go solution, I wrote a little JUnit extension called jeeunit together with an example project showing how to set things up with Glassfish v3 and Maven.

The jeeunit project is available from Google Code under an Apache License.

The Ultimate Software Metric

2010-03-03T19:30:00.000+01:00

Misconceptions about Java Internationalization

2010-02-28T20:19:00.001+01:00

Let me start with a joke:

What do you call someone who speaks three languages?
Trilingual.
What do you call someone who speaks two languages?
Bilingual.
What do you call someone who speaks one language?
American.

To be fair on Americans, even most of us multilingual Europeans tend to be biased when it comes to internationalization, tacitly assuming that text is written left-to-right and can be sorted from A to Z.

Most Java developers are familiar with resource bundles backed by properties files. The basics can be found in the Internationalization Trail of the Java Tutorial. Multilingual Java applications often come with a set of properties files, e.g.

MyApp_de_AT.properties
MyApp_de.properties
MyApp_es.properties
MyApp.properties

where MyApp.properties contains the "default" message resources in English, MyApp_de.properties and MyApp_es.properties contain the German and Spanish resources, respectively, and MyApp_de_AT.properties contains some country specific variants for the Austrian flavour of German. Usually, files for country specific variants are sparsely populated, containing only those properties that actually differ from the mainstream language version, like Jänner (de_AT) vs. Januar (de) vs. January (en).

However, you may be surprised in this case to end up with a German string even when you requested a resource for an English locale.

Assume nothing is a sound principle for robust software development, and you should not assume that English is the default or fallback language. In fact, the fallback for resources from an unsupported locale is the system default locale, which is based on the host environment.

See the documentation for ResourceBundle.getBundle() and Locale.getDefault() for more details.

So when the default locale of your system is de_DE and you request a resource for locale en_US, the lookup order for the properties files is

MyApp_en_US.properties
MyApp_en.properties
MyApp_de_DE.properties
MyApp_de.properties
MyApp.properties

Hence, ResourceBundle.getString() will return a German string from MyApp_de.properties, since the first three files do not exist and the English resources are preceded by the German ones in this sequence.

There are two solutions:

As a user, set your default locale to en when launching the application.
As a developer, make sure to provide a properties file for locale en (which may be empty).

The method for changing the default locale depends on your Java VM and your operating system. Setting the system property user.language may work on some platforms, but not with the Sun JDK 1.6.0 under Linux. Instead, you need to set the environment variable LANG before launching the Java VM.

The preferred solution is the second one, of course. Even when MyApp_en.properties is empty, it will be picked up as entry point for resource lookup. If a given key cannot be found in this file, the parent file MyApp.properties will be used as fallback, which is just the desired behaviour.

Editing Resource Bundles in Eclipse

2010-02-24T22:18:00.003+01:00

Playing around with the Apache Roller blog engine, I noticed that some of the localized German text messages were missing or broken. Roller uses plain old Java resource bundles instead of the NLS mechanisms offered by Eclipse. Editing resource bundles for multiple languages in parallel is rather a pain with a plain text editor, so I was looking for an Eclipse plugin to do this job.

(Just to avoid any confusion, even though I've been writing a lot about OSGi bundles, the term bundle is only used in the sense of a resource bundle, or properties file, in this article.)

I found two solutions, both of which have minor bugs and lack some documentation but are very helpful nevertheless. And it turned out that the second solution uses code from the first one:

Resource Bundle Editor from Sourceforge
Messages Editor from the Eclipse Babel project

At first, I tried the Resource Bundle Editor. The latest version is from 2007, and some people have reported conflicts with newer Eclipse versions. I installed the Bundle in my Eclipse 3.5.1, and did not notice any version conflicts at all.

However, the Resource Bundle Editor does not parse the properties files correctly. It does not recognize exclamation marks as comment signs. For comment lines of the form

!some.key = some value

the editor will display a bogus key !some.key.

Looking at the sources, I found the the PropertiesParser class only recognizes a subset of the valid properties file syntax.

After that, I had a look at the Eclipse Babel editor. Unfortunately, the Babel project does not yet provide binary downloads, so you have to build the two plugins from source.

As it turned out, parts of the Babel sources are derived from the Resource Bundle Editor sources, and the same incomplete parser code is also used in the Eclipse project in class PropertiesDeserializer.

I changed a regular expression in the source to fix the "!"-problem. You can get the binary plugins including my patch from here:

After installing the plugins, go to Window | Preferences | Messages Editor and deselect the option Setup validation builder on Java projects automatically, or else you may get lots of error markers on other properties files which are not used as message bundles at all. I also set the Reports severities to Ignore and the Displayed Locales to de to narrow the Editor display to the language I'm actually working on.

To edit a resource bundle, select the properties file in the Package Explorer and open it with the Messages Editor via the context menu.

Here is a screenshot of the Messages Editor in action:

With the additional toolbar buttons, you can limit the view to missing or unused translations.

Setting up Eclipse for Roller

2010-02-23T19:38:00.003+01:00

There is an Eclipse plug-in for almost any task, and most of them do their job rather nicely. On the other hand, even some of the more or less official ones may give you a hard time if you try to use them in combination.

Recently, I've been playing around with Apache Roller, a Java blog engine, much like Blogger or Wordpress. This project is currently in beta for the next major release 5.0, so I checked out the sources from trunk, ran the Maven build, created a PostgreSQL database and got my own blog engine up and running within minutes.

Some minor things did not quite work as expected, so I thought I'd just create an Eclipse workspace for Roller and build and run it from there. As it turned out, the combination of Maven, Subversion, and a Web Application was a rather fatal mix, and it took me a day to figure out what was going wrong.

Most of this was not a Roller issue at all: Eclipse Web application tooling (WTP) and Maven Integration (m2eclipse) just make too many implicit and conflicting assumptions which make it hard to set things up correctly, so this article is really about working with a mavenized web application in Eclipse, and Roller is just an example.

There are a couple of threads on the Roller developer mailing list dealing with Eclipse setups, but none of them really provides a working solution, so maybe this post can fill gap.

Step 1: Get Eclipse and all required plug-ins

To avoid any conflicts with other plug-ins or features not required for this project, I used a separate Eclipse installation consisting of

Eclipse for Java EE Developers 3.5.1
Subversive SVN Team Provider 0.7.8
Subversive SVN Connectors 2.2.1
Maven Integration for Eclipse 0.10.0
Maven Integration for WTP 0.10.0

Even if you do not intend to commit any changes from Eclipse, you will need the Subversion integration to deploy your web application from a Subversion working copy, otherwise Eclipse will try to pack the hidden .svn directories into the JARs and WARs and complain about duplicate paths.

Step 2: Set up m2eclipse

m2eclipse has a built-in pre-release version of Maven 3.0.0 which is not compatible with most existing projects based on Maven 2.x. Get a local installation of Maven 2.1.0 and define it as default for m2eclipse in Window | Preferences | Maven | Installations.

Step 3: Set up Tomcat

Download and install Tomcat 6.0.24 to a local directory. Create a Tomcat server instance for Eclipse via Window | Preferences | Server | Runtime Environments pointing to your Tomcat installation directory.

Install the additional prerequisites of Roller in the Tomcat lib directory:

mail.jar
activation.jar
your JDBC driver

Step 4: Get Roller into your Eclipse workspace

Create a new empty workspace and switch to the SVN Repository Exploring perspective. There is supposed to be an integration of m2eclipse and Subversive, which I never managed to get to work, so this is why I use the following somewhat clumsy procedure to populate my workspace:

Switch to the SVN Repository Exploring perspective and define a new repository location for https://svn.apache.org/repos/asf.
Check out Roller from roller/trunk. This will create a new project roller-project in your workspace.
Unfortunately, the Maven modules of this project do not yet appear as separate Eclipse projects. To change this, delete the project from your workspace and use File | Import | Maven | Existing Maven Projects. Select the workspace folder from your initial checkout.
After this, you should have six Maven projects in your workspace, all shared via Subversive.

Step 5: Apply some fixes in the workspace

Go to roller-weblogger-business and delete src/test/resources/org/apache/roller/weblogger/business/package.html, since this file would cause a clash with another copy from src/main/resources.
Open /roller-weblogger-web/src/main/webapp/WEB-INF/security.xml and replace spring-security-2.0.1-openidfix.xsd by spring-security-2.0.4.xsd.
Copy your roller-custom.properties to /roller-weblogger-web/src/main/resources.

Step 6: Configure your web application

Open the project properties of roller-weblogger-web.
Select the Java EE Module Dependencies and activate roller-planet-business, roller-core and roller-weblogger-business.
Make sure that the resources from all dependent projects will get copied into the web application by modifying the build path settings of roller-planet-business, roller-weblogger-business and roller-weblogger-web. Select Java Build Path from the project properties and remove the Excluded: ** entry from src/main/resources for each of these projects.

Step 7: Run a Maven build

Select roller-project/pom.xml. From the context menu, select Run As | Maven build...
In the launcher dialog, fill in the goals clean install and (optionally) check Skip Tests to save some time during each build.
When the build has completed, select all projects and press F5 so that Eclipse will see all the resources created by Maven.
This step is required, since the Maven build generates some additional resources and runs the OpenJPA Enhancer. These two steps would not be handled by the Eclipse automatic build.

Step 8: Make sure that Eclipse picks up the generated resources

Create a folder /roller-weblogger-web/src/main/sql and turn it into a source folder.
Copy /roller-weblogger-business/target/dbscripts into this folder.

Step 9: Get Rolling!

Select roller-weblogger-web and invoke Run As | Run on Server from the context menu.
Select the Tomcat instance created in Step 3 and activate it as default if you like and click Finish.

Troubleshooting

If you get stuck, clean the Tomcat instance. Open the Servers view and select Tomcat. From the context menu, invoke Clean...
To check the web application assembled by Eclipse, have a look into <Eclipse workspace>/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/roller-weblogger-web/

All of this should have been a lot easier. m2eclipse will have to mature, and Eclipse should be more flexible about resource locations...

Running parameterized JUnit tests in parallel

2009-12-17T21:42:00.000+01:00

We use JUnit 4 for Anaconda, not just for unit tests, but also for integration or system tests. Typically, we iterate over all features of a given class in a database and validate each feature.

Note: I'm using the term feature in the sense of map feature, not in the sense of application feature or implemented requirement.

A simple pattern for such tests is

public class FeatureTest
{ 
    @Test
    public void testAllFeatures()
    {
        for (Feature feature : findAllFeatures())
        {
            testOneFeature(feature);
        }
    }

    private void testOneFeature(Feature feature)
    {
        // some logic with one or more JUnit assertions
    }
}

Obviously, this naive approach has the following drawbacks:

The test fails and terminates on the first incorrect feature. The remaining features will not be tested.
All features get tested sequentially. This may take awfully long for a large database.

JUnit 4 has a specialized runner Parameterized for running all tests in a given class with different parameters from a given list of parameter sets. An instance of the test class is created for each parameter set, and the parameters are passed to the constructor via reflection:

@RunWith(Parameterized.class)
public class FeatureTest
{
    // This is the parameter for each instance of the test.
    private Feature feature;

    public FeatureTest(Feature feature)
    {
         this.feature = feature;
    }

    @Parameters
    public static Collection<Object[]> getParameters()     
    {         
        List<Feature> features = findAllFeatures();
        List<Object> parameters = new ArrayList<Object[]>(features.size());
        for (Feature feature : features)         
        {
             parameters.add(new Object[] { feature };
        }
        return parameters;
    }
 
    @Test
    public void testOneFeature()     
    {
        // assertions acting on the feature member
    }
}

This solves the first problem: Each feature gets tested in its own test instance. Now if there is a large number of features or if each individual test is very expensive, we would like to run the test instances in parallel, using a thread pool, or maybe even a grid of multiple computers.

Browsing through the JUnit sources, I found a surprisingly easy way of parallelizing the tests with a thread pool, simply by using a custom runner:

@RunWith(Parallelized.class)
public class FeatureTest
{
   // same class body as above
}

All you need is a simple extension of the Parameterized runner:

public class Parallelized extends Parameterized
{
    
    private static class ThreadPoolScheduler implements RunnerScheduler
    {
        private ExecutorService executor; 
        
        public ThreadPoolScheduler()
        {
            String threads = System.getProperty("junit.parallel.threads", "16");
            int numThreads = Integer.parseInt(threads);
            executor = Executors.newFixedThreadPool(numThreads);
        }
        
        @Override
        public void finished()
        {
            executor.shutdown();
            try
            {
                executor.awaitTermination(10, TimeUnit.MINUTES);
            }
            catch (InterruptedException exc)
            {
                throw new RuntimeException(exc);
            }
        }

        @Override
        public void schedule(Runnable childStatement)
        {
            executor.submit(childStatement);
        }
    }

    public Parallelized(Class klass) throws Throwable
    {
        super(klass);
        setScheduler(new ThreadPoolScheduler());
    }
}

The RunnerScheduler interface is fairly new in JUnit and marked as experimental. I discovered it in the current version JUnit 4.8.1 and found it missing in JUnit 4.4.0 which we have been using so far. RunnerScheduler is also available in JUnit 4.7.0, but I did not check if this is the earliest version.

Eclipse Forms and Data Binding

2009-10-04T20:50:00.000+02:00

Our map compiler Anaconda reads all parameters and settings from a configuration file, which over time has evolved from a simple Java properties file to a not-so-simple XML file which is validated by an XML schema.

To access the configuration at runtime, we use Java XML Bindings generated from our schema by xjc in a straightforward manner, without any fancy customizations.

Using the XML editing support in Eclipse, it is very easy to edit and validate a configuration file, at least from a developer perspective. However, our customers will not be too happy about editing large XML files by hand, so the idea is to develop a form-based configuration editor for our RCP application Anaconda Workbench, similar to the manifest editor of Eclipse PDE or the POM editor of m2eclipse.

These editors are based on Eclipse UI forms, another layer on top of SWT and JFace, which is obviously powerful enough for complex tasks, as demonstrated by the above examples. Less obviously, it is rather poorly documented. The Eclipse online help has just 10 brief pages about UI Forms and the Javadocs which, as usual, are not very useful for getting started.

The Eclipe Rich Client Platform book also has not more than one page on UI Forms and a link to an online article from 2004. On the Eclipse site, there are two more recent articles

Looking for futher tutorials, I came across Marco van Meegen's critical review Eclipse Forms im Härtetest. I decided to make up my own mind, but after implementing a few examples with Eclipse Forms, I largely agree to his criticism: the API forces you to write lots of repetitive code and it is not easy to figure out how to wire up the different classes to do your job.

To alleviate the shortcomings of Eclipse Forms, Marco created yet another layer called RCPForms. So I gave it a try, and I found it a lot easier to use than working with UI Forms directly. I had to use the sources from the Subversion trunk at Sourceforge, the older tagged or released versions do not seem to work with Eclipse 3.5.

Eclipse forms are usually wrapped by a ManagedForm which manages the state of the form parts and the underlying data models.

The form parts and the models can vary independently, and one model can be shared by multiple form parts.

A managed form is dirty, when one of its parts is more recent than the underlying model. Conversely, it is stale, when a change in the underlying model is not yet reflected on the UI.

To handle this form lifecycle, RCPForms expects the model to support PropertyChangeListeners, which again requires you to add some boilerplate code to your model beans. For my example with a JAXB model, I managed to tweak xjc to generate the required listeners, which is to be discussed in detail in a separate article.

Here is a screenshot of a simple example:

The warnings result from validators on missing mandatory fields.

Eclipse RCP Application Branding

2009-09-13T21:35:00.018+02:00

A while ago, I figured out how to use the branding features in Eclipse to make the About dialog in my Anaconda Workbench show all the included features with the appropriate icons and other information. This was based on Eclipse 3.4, and here is a screenshot of the results:

Clicking on Feature Details, you get a list of all branded features included in the product:

Not every feature is branded by default. You have to associate a branding plug-in to a given feature and include an about.ini file and an optional icon in the branding plug-in. I won't go into details for this Eclipse 3.4 example, seeing that things have changed for Eclipse 3.5, and the rest of this post relates to Eclipse 3.5.

What's new in 3.5

The Feature, Plug-ins and Configuration Details buttons have gone, now there is just a single Installation Details button. To see the installed features, click on one of the feature icons to open a dialog listing all features branded with the same icon.

(Actually, I'm not quite sure whether or not the icon name is the grouping criterion, but I guess so from my experiments.)

Selecting a feature from the list and clicking on Plug-in Details, you can display all plug-ins contained in the feature.

Things to note

Eclipse Help has just two rather brief topics on Product Branding and Customizing a product. Branding a product is straightforward if you use the Product Configuration Editor as explained in the first topic. Things get more complicated when your product contains multiple features or even third-party products.

For each feature to branded, you need a branding plug-in containing the following resources:

about.ini
about.properties (optional)
a 32*32 icon

Make sure to include these resources in you build.properties so they will be exported to your plug-in JAR.

In about.ini, set the following two properties:


aboutText=Anaconda Workbench for Car Navigation Databases
featureImage=icons/anaconda32.png

You can use the optional about.properties to work with language dependent variables in about.ini. Set featureImage to the relative path of your icon resource.

By default, Eclipse assumes that the branding plugin for a feature com.acme.foo is also named com.acme.foo. If you want to assign a different name, you have to explicitly declare the branding plug-in in the feature editor.

I ran into a conflict between feature branding and product branding: My top-level product Anaconda Workbench contains another product udigLite, which in turn contains Geotools, Eclipse EMF and Eclipse RCP.

udigLite is a modified subset of uDig. uDig has a top-level feature named net.refractions.udig which a contains a branding plug-in of the same name with a product definition.

udigLite originally had a top-level feature net.refractions.udig.lite containing just a subset of the uDig plug-ins, including the net.refractions.udig plug-in which cannot be left out as it contains more than just branding information.

With this configuration, I ended up with two uDig features in the feature list: net.refractions.udig.lite as expected, and net.refractions.udig, although no such feature exists in my product.

It seems that a branding plug-in containing a product definition is automatically treated as a feature of the same name. I wonder if that's a bug or a feature (pun intended).

Anyway, to remove this duplicate, I dropped my net.refractions.udig.lite feature and changed the original net.refractions.udig feature to include just the required subset of plug-ins.

Another problem I ran into is bug 280186 or bug 289300, causing each feature to be listed twice in the exported product. To supress that, you can drop the org.eclipse.equinox.ds plug-in from your product, provided your application does not depend on Declarative Services.

Arrows with Styled Layer Descriptor

2009-04-07T16:09:00.004+02:00

In our uDig plug-ins for the Anaconda Workbench, we use Styled Layer Descriptors (SLD) for the map styles.

One of the features we are currently working on is the direction attributes for one-way streets. Of course you want to visualize them on the map, but SLD does not directly support arrow decorations for lines.

I asked for advice on the uDig mailing list and got the answer in no time. You can either use an arrow character from a suitable font in a Text Symbolizer, or a clever combination of Line Symbolizers with different styles of dashed lines to emulate arrows.

Using an SLD snippet from the Geoserver blog, I got the following result:

[Sorry, screenshot deleted to avoid potential licence issues.]

Getting Started with OSGi Declarative Services

2009-04-02T10:10:00.002+02:00

OSGi Declarative Services have been on my mental agenda for a while, and now I've started using them. I'm not going to write yet another tutorial on the subject, but there are some bits and pieces I could not find anywhere else that may be worth mentioning.

To get an overview of Declarative Services, have a look at Neil Bartlett's tutorial on EclipseZone, or at his more recent EclipseCon presentation. The former is based on Eclipse 3.2.2, the latter on the latest Eclipse 3.5 milestones, so there are some differences to the current release Eclipse 3.4.2. In particular, Eclipse 3.5 supports a newer version of the Declarative Services Specification (1.1 vs. 1.0 in Eclipse 3.4).

All of the following refers to Eclipse 3.4.2. For using Declarative Services with Equinox, you need the following bundles:

org.eclipse.osgi
org.eclipse.osgi.services
org.eclipse.equinox.ds
org.eclipse.equinox.util

The last two are not included in standard Eclipse distributions. Download them from the Equinox download area, or use the Eclipse update manager to install Equinox Bundles.

Follow the tutorials to implement a simple example. If you cannot figure out why your component does not start, enable console logging for the Service Component Runtime in your OSGi launcher by setting the following system properties.

equinox.ds.debug=true
equinox.ds.print=true

Make sure to include the OSGI-INF folder with your component descriptors into the binary build on the Build tab of the Eclipe manifest editor. This amounts to adding the folder to the bin.includes property in the build.properties.

Add a header Service-Component: OSGI-INF/components.xml to your manifest. (Choose any other filename if you like.)

Do not forget to specify the correct XML namespace http://www.osgi.org/xmlns/scr/v1.0.0 in your XML documents, or else your component definitions will not be recognized.

Seeing that Eclipse 3.4 has no graphical editors for the component descriptors, it is useful to install the Eclipse XML editor features and to work with XML validation on the component descriptors.

If your bundle contains more than one component, you can have one XML document per component and list them all in the manifest header, or you can have a single XML document containing all component descriptions, which is what I prefer.

To enable XML validation on a multi-component descriptor, write your own XML schema where the root element contains a sequence of 1..n component descriptors.

Here is an XML schema for this purpose:


<?xml version="1.0" encoding="UTF-8"?>
<schema xmlns="http://www.w3.org/2001/XMLSchema"
        xmlns:scr="http://www.osgi.org/xmlns/scr/v1.0.0" 
        targetNamespace="http://www.harmanbecker.com/anaconda/components" 
        xmlns:tns="http://www.example.org/components" 
        elementFormDefault="qualified">

   <import namespace="http://www.osgi.org/xmlns/scr/v1.0.0"/>

    <element name="components">
      <complexType>
        <sequence>        
          <element ref="scr:component" maxOccurs="unbounded"/>
        </sequence>
      </complexType>
    </element>
</schema>

Using this schema, a component descriptor for two components might look like this:


<?xml version="1.0" encoding="UTF-8"?>
<c:components xmlns:scr="http://www.osgi.org/xmlns/scr/v1.0.0"
      xmlns:c="http://www.harmanbecker.com/anaconda/components">


 <scr:component name="foo">              
   <implementation class="com.acme.foo.FooImpl"/>
   <service>
     <provide interface="com.acme.foo.Foo"/>
   </service>
 </scr:component>

 <scr:component name="bar">              
   <implementation class="com.acme.foo.BarImpl"/>
   <service>
     <provide interface="com.acme.bar.Bar"/>
   </service>
 </scr:component>

</c:components>

JDBC Drivers in OSGi

2009-04-01T16:50:00.005+02:00

How do you create a JDBC connection for a given JDBC URL in an OSGi application? In particular, how do you avoid an explicit dependency of your application code on a given driver?

In a plain old classpath context, you would invoke DriverManager.getConnection(), maybe after loading the driver class using Class.forName(), if your driver does not support the Service Provider mechanism which is mandatory for JDBC 4.0.

The trouble is, even if your driver does provide the META-INF/services metadata, this does not work in OSGi, since DriverManager creates a number of class loader problems by using Class.forName() internally.

I'm working with a variant of the Zentus driver for SQLite, which I've modified slightly to build an OSGi bundle under Java 1.6. I added the service metadata, and indeed my application code now works without loading the driver via Class.forName() - as long as the code is running outside of OSGi.

There are the following issues:

The context class loader of the current thread is used to scan for META-INF/services resources and to load the referenced classes. This happens during initialization of DriverManager, so it will not catch resources from bundles which are not visible to the current context.

Additional drivers can register with DriverManager, they should do so in static initalization code, so that the driver gets registered when someone calls Class.forName("my.own.Driver").

DriverManager.getConnection() iterates over the registered drivers until it finds one that can process the given URL. Unfortunately, it does a fatal double check to see if the class of that driver is the same that would be obtained by the caller. This amounts to calling Class.forName("some.matching.Driver", ccl) where ccl is the caller's class loader.

(To get some information on what is happening, it is useful to call DriverManager.setLogStream(System.out).)

The third point means that DriverManager.getConnection() will always fail if the driver class is not visible to your bundle, so there really is no way to avoid a dependency.

Here is an outline of a partial solution:

Create an OSGi bundle for each JDBC driver you want to use. Add a bundle activator that calls Class.forName("some.jdbc.Driver"). This will register the driver with DriverManager when the bundle is started.
For each bundle that needs to create connections using DriverManager.getConnection(), add optional dependencies on the JDBC drivers packages for all drivers that you are planning to use.

This is rather ugly, because you need to edit your client bundle manifest whenever you want to add support for an additional driver. But at least you can deploy your system with any subset of the defined drivers, maybe with just a single driver.

If you do not need to support legacy code, do not bother using DriverManager and the Service Provider mechanism. After all, this is just a poor man's service registry, and you are much better off using the OSGi Service Registry.

I'll give an example in one of my next posts.

Maven Plugin for Eclipse Source Bundles

2009-03-29T19:46:00.007+02:00

The Maven Source Plugin creates a JAR containing the sources of a project and attaches it to the main artifact of the project.

It would be nice if you could directly use this JAR as source attachment for a bundle in your Eclipse target platform, but Eclipse 3.4 expects a special Source Bundle format with specific headers in the manifest. There is a feature request to support this directly in the Maven Source Plugin.

To fill this gap, I had a look at the sources of the Maven Source and Archiver plugins and wrote my own maven-sourcebundle-plugin. This plugin is available for public use as a subproject of the DataScript project hosted at BerliOS.

There is no public Maven repository for this project yet, but it is contained in the latest binary release rds-bin-0.30, or you can get the sources from Subversion.

3D Rendering in uDig

2009-03-29T17:22:00.003+02:00

I've started experimenting with Java3D rendering in uDig. The goal is to use Java3D as a rendering engine first for plain old 2D data, and then for real 3D data in the next step.

More details can be found in the uDig wiki.

Java3D and Eclipse

2009-03-25T20:49:00.006+01:00

The world is not flat, and 3D maps is one of the latest hypes in navigation systems. (Is anybody using them for real, I wonder....)

So there is a need for me to deal with 3D city and terrain models in my applications, and of course the first thing you want is a 3D viewer. I started experimenting with Java3D a few months ago, and it felt such a relief compared to OpenGL programming in C. The only thing that gave me a hard time is the lack of documentation in some areas, but in the end I could answer most of my questions by studying the source code or by trial and error.

At first, I wrote a couple of stand-alone viewers with a Canvas3D embedded in an AWT Frame, but what I really need is Java3D embedded in an Eclipse view or editor.

I found a couple of postings on Java3D with SWT which sounded rather discouraging, but in the end it was rather straightforward. The java3d-eclipse project on Sourceforge gave me some useful hints, but I cannot recommend using it, since the way that it re-bundles the Java3D JARs is definitely not OSGi-compliant.

Now here is my own recipe:

Get the sources for Java3D 1.5.2 from https://java3d.dev.java.net. There is no source package, you have to use CVS. You need three subprojects: j3d-core, j3d-core-utils and vecmath.
Compile vecmath into a stand-alone OSGi bundle.
Follow the Java3D build instructions to build j3d-core and j3d-utils in one go, including some native library compilation and Java code generation.
j3d-core and j3d-core-utils mutually depend on each other and cannot be compiled separately, so I really don't understand why Sun created two libraries instead of one. Anyway, to be friendly to Maven and OSGi, I copied the original sources, the generated sources and the native library binaries to just one Maven project, created a POM and built my Java3D OSGi bundle using the maven-bundle-plugin.
I did this both for Windows and Linux and managed to launch the HelloUniverse demo from Equinox.
For cross-platform use, move the platform specific code (both the native libs and a handful of Java classes) to separate projects and build OSGi fragments for these, similar to SWT.

After these preparations, embedding a Canvas3D into an Eclipse view or editor is as easy as this:


public void createPartControl(Composite parent)
{
   awtContainer = new Composite(parent, SWT.EMBEDDED);

   Frame frame = SWT_AWT.new_Frame(awtContainer);
   frame.setLayout(new BorderLayout());

   GraphicsConfiguration config = SimpleUniverse
       .getPreferredConfiguration();

   Canvas3D canvas = new Canvas3D(config);

   createUniverse(canvas);
   frame.add(canvas, BorderLayout.CENTER);
}

Anaconda: A New Architecture for Compiling Navigation Databases

2009-03-19T19:57:00.002+01:00

In my previous post, I presented the Anaconda Workbench, without explaining about Anaconda. This is the code name for my current project, a Map Compiler for processing map data into a compact binary database for car navigation systems of Harman/Becker.

I gave a talk about Anaconda at the Eclipse Demo Camp in Hamburg in November last year. The slides of this talk explain the purpose of a Map Compiler and our usage of OSGi and Eclipse and lots of other fabulous Open Source components.

Anaconda Workbench: A uDig Application

2009-03-18T11:01:00.010+01:00

Now here is a glimpse at what we are using uDig for. I think all the digging in the internals of several Open Source projects is now beginning to pay off.

The Anaconda Workbench is an application for viewing and testing car navigation databases in the forthcoming Navigation Data Standard format (NDS). (This is a closed source project of Harman/Becker, so I cannot offer more than some general information here.)

We implemented a Geotools extension for the NDS format, and a corresponding uDig catalog plug-in. This was enough to build an NDS map viewer. The map has multiple levels of detail in separate layers. We use SLD for styling the layers and for activating the appropriate layer depending on the current map scale.

On top of the map viewer functionality, there is a name browser which allows you to select road names and display the road on the map or use it as a start or destination for a route.

There is also a route calculator plug-in which finds the shortest route between two points and highlights the route in a separate layer.

More functionality will be added step by step. In the end, the Anaconda Workbench shall become a front-end for our map compilation process and not just a viewer for the compiled map databases.

udigLite: An OSGi-friendly subset of uDig

2009-03-18T10:49:00.006+01:00

At long last, my team and I have succeeded in creating a subset of uDig which suits our environment. The baby is called udigLite, and the first binary releases can be downloaded from BerliOS, the site also hosting our Mercurial clones of the original Subversion source repositories.

Read the whole story in the uDig wiki.

The Digital World

2008-11-08T11:55:00.003+01:00

Digital maps and geographical databases have come to be a commonplace commodity. To locate a friend's address, you simple open Google Maps, when you go to their place by car, your nav system will guide you, and if you feel challenged by the patent folding of your Falk city map, you can download a map on your mobile phone to cover the last mile to your friend's home.

The raw geographical data for such services are supplied by companies like NAVTEQ and Tele Atlas, but it takes a lot of additional data processing to build user-friendly services based on these data.

My daytime work is dedicated to the design and development of a map compiler for Harman/Becker, which will produce the map databases for the next generation of car navigation systems to hit the market within a few years from now.

Doing this job in Java does not sound natural to everyone, but for us, the choice of Java has been the key to building our work on top of a whole lot of ready-to-use components and libraries, both for general purpose and domain specific tasks. So it is really Java and all these excellent open source components that help us getting around the world.

Let me just mention some of the Java libraries or applications that are useful for dealing with geographical data. I will cover some of them in more detail in future posts.

The best thing about geo data is that you can have a look at them - if you have some viewer tool that renders a map for you. A couple of years ago, I was searching for a Java-based alternative to the home-grown tools we use at work, and I came across JUMP. This gave me a jump start into geo data processing in Java...

JUMP is a map viewer and editor for a number of open standard data formats, and it has "plug-in" interfaces for supporting alternative data formats, which was a key requirement for me. I put the word "plug-in" in quotes, because JUMP has absolutely nothing to do with Eclipse plug-ins or extension points or OSGi bundles. JUMP does its own bit of class loader magic to use extension classes from JAR files you drop into a given folder.

I found JUMP easy to use and fairly easy to extend. Some people complained to me about its bland and somewhat out-dated GUI, but that has never really bothered me. JUMP is a solid and mature piece of work, and the major downside I see is the fact that it is no longer under active development. There are now more flexible extension mechanisms, and some of the features I would like to extend or modify are simply not laid open, so I would have to modify the sources.

Actually, Mama JUMP may have retired, but her offspring is alive and kicking: There is a whole family of JUMP spin-off projects.

JUMP introduced me to the JTS Topology Suite which it uses for all operations on two-dimensional geometrical shapes. I found JTS to be extremely useful and well-designed, it is based on solid mathematical background and suceeds in hiding the most of the maths from the average user who just needs to get his job done, and it has a rather interesting history.

JTS has an LPGL license, whereas JUMP is under GPL. This may be too restrictive for some commercial applications, and when I wrote to Vivid Solutions, the company behind both JUMP and JTS, about this licensing issue, they pointed me to uDig as an LGPLed alternative.

That must have been some time in 2005. My first encounter with uDig was not very successful: I did see the power of its architecture, built on top of Eclipse RCP, but at that time, I was simply overwhelmed by all the prerequisites you had to understand to implement as much as a Hello World plug-in in uDig. And back then, I knew little or nothing about Eclipse plug-in development, so I concluded that uDig was not (yet) for me, but I kept watching its progress.

In the meantime, uDig has grown a lot in terms of functionality, stability and documentation, whereas my own knowledge of OSGi and Eclipse technologies has increased. Getting started with uDig extensions is still rather a challenge, but my efforts have started paying off. With support from the developer mailing list, my uDig plug-ins have grown to a level of functionality almost equivalent to my JUMP extensions, so I am planning to stop working with JUMP and use uDig as a more flexible, powerful and (hopefully) future-proof platform.

The abstractions of cartographic features and datastores used by uDig are based on Geotools, another open source project providing a set of Java libraries for processing a vast number of geospatial data formats, including plain files, databases or web services. Both Geotools and uDig use JTS for geometrical operations.

All these projects directly or indirectly rely on OpenGIS standards and specifications, in particular, the Simple Features Specification. Geotools provides implementations of the OpenGIS Java interfaces published by the GeoAPI project.

To sum up, for geospatial data processing, there is a wealth of open source Java libraries based on open standards. For any Java-based development in this area, one of these projects may save you some work.

Hibernate and OSGi: An elaborate solution

2008-11-06T22:10:00.004+01:00

A much cleaner and flexible way of osgifying a plain old JAR with lots of dependencies is adding the required OSGi headers to each JAR, so you will replace the megabundle with a bunch of small bundles, one per JAR, which you can reuse in other contexts.

In an ideal OSGi world, every JAR would be an OSGi bundle, so you would have nothing to worry about. Unfortunately, most Java libraries still come as plain old JARs and you have to add the headers on your own.

If you are lucky, someone has done the job for you already. There are a number of third-party repositories offering osgified versions of popular Java libraries, e.g. the OSGi bundle repository or the SpringSource Enterprise Bundle Repository.

SpringSource even has an OSGi bundle of Hibernate itself. However, their version does not work, at any rate not in my setup, so I had to build my own Hibernate bundle.

At least, I was able to use the SpringSource OSGi version for each dependency of Hibernate.

These are the problems I had with the SpringSource version of Hibernate (3.2.6.ga):

The javax.transaction package does not resolve. It is imported with a specific version range. However, in Java 1.6.0, this package is contained in the JRE and comes from there without a version. I had to drop the version directive to make Hibernate use the javax.transaction package from the JRE.
I got an exception on running a HQL query, since ANTLR could not load the Hibernate token class.
I had a very mysterious ClassNotFoundException when accessing some of my model classes. As it turned out, the reason was my usage of lazy loading, where Hibernate injects CGLIB proxy classes into my model classes. The exception was due to the fact that my model bundle did not have access to the CGLIB classes. Rather than declaring a dependency on CGLIB for each of my model bundles, I added the following header to my Hibernate manifest:

Require-Bundle: com.springsource.net.sf.cglib;visibility:=reexport

This means that every bundle with a dependency on Hibernate automatically inherits the dependency on CGLIB. Even St. Peter the Evangelist who usually preaches about using Import-Package instead of Require-Bundle admits this to be one of the rare cases where the latter has its merits.

So far, with this approach, we have not used any Equinox buddy policies, but we still need to deal with the application model classes and resources.

For a while, I thought a fragment bundle would be the definitive solution, which would work not only on Equinox.

I created a bundle com.acme.myapp.hibernate.fragment, containing no Java classes, but only a manifest and some resources, i.e. the Hibernate configuration file and all my Hibernate mapping files. I added all model packages and all relevant JDBC driver packages to the Import-Package header (using resolution:= optional for the JDBC packages.) The host for this fragment is the org.hibernate.osgi bundle, of course.

This worked perfectly when launching my application from within the Eclipse IDE. However, I had an unpleasant surprise when running our batch builds which are based on the Eclipse PDE Ant runner.

PDE kept complaining about cyclic dependencies in my bundles. And not even in my own bundles, also between some of the third-party libraries used by Hibernate, in particular between jaxen and dom4j.

(I had a look at the sources of these two libraries to understand what was going on here: each of them contains some helper classes for the other one, and each must have been compiled against an older version of its friend - really scary...)

I had already spent half a day in working around this problem by repackaging each cycle of libraries in one bundle, and I was going to file an Eclipse bug report. When searching for similar issues, I found bug 208011 which not only describes the problem, but also offers a partial solution:

allowBinaryCycles = true

is a property you can set in your top-level build.properties file as of Eclipse 3.4. (Not a word about this in the Eclipse Online Help, and not even in the comments of the batch build template files!)

According to Chris Aniszczyk, this option will be accessible from the IDE UI in Eclipse 3.5M3.

allowBinaryCycles did suppress the error message regarding the dom4j-jaxen cycle.

But alas, now I had another error complaining about a cycle between Hibernate, my hibernate fragment and my model bundles. Apparently, Eclipse regards the fragment as part of its host (which is ok), and since the fragment depends on an application bundle, and the application bundle depends on Hibernate, so the PDE batch build now complains about a cycle between Hibernate and my application bundle.

So for now, I left all the resources and the JDBC dependencies in the Hibernate fragment, but reverted to using buddy declarations in Hibernate and my model bundles to break the dependency cycle.

I also created a fragment com.acme.myapp.antlr.fragment which imports org.hibernate.hql.ast.HqlToken and thus make the custom token class of Hibernate visible to ANTLR. (See the SpringSource bug report.)

All in all, this looks 95 % clean to me, so I think I'm going to leave it at that for a while...

Hibernate and OSGi: A pragmatic solution

2008-11-06T21:59:00.005+01:00

The simplest thing you can do to turn a plain old JAR with a number of external dependencies into an OSGi bundle is wrapping this JAR and all its dependencies in another JAR and adding an OSGi manifest with all dependencies on the bundle classpath.

For example:

Bundle-SymbolicName: org.hibernate.osgi
Bundle-ClassPath: hibernate3.jar,
lib/antlr.jar,
lib/cglib.jar,
lib/commons-logging.jar,
...
Export-Package: org.hibernate,
org.hibernate.configuration,
...

With this megabundle, you can trivially solve all classpath or visibility issues between Hibernate and its dependencies.

You still have to do something to make the model classes of your application and the mapping files accessible to Hibernate.

Eclipse Equinox has an extension of the OSGi standard called buddy policies, which enables you to define some kind of classloader callback.

Your application depends on Hibernate, but you do not want Hibernate to depend on your application. Even if you wrap your own Hibernate bundle, you want to be able to use it in more than one of your applications.

There are several flavours of buddy policies, I will just mention one of them: Adding the header

Eclipse-BuddyPolicy: registered

to your Hibernate bundle manifest will tell Hibernate to ask all its buddies for classes it cannot load using its own classloader.

In each bundle of your application containing some Hibernate model classes, you then add a dependency on Hibernate and a header telling Hibernate that your bundle is a buddy:

Eclipse-RegisterBuddy: org.hibernate.osgi
Require-Bundle: org.hibernate.osgi

(The buddy thing does not work unless there is an actual dependency by Require-Bundle or Import-Package.) This will enable Hibernate to load classes and resources from your bundle, so it will also be able to locate your Hibernate mapping files if you put them into your bundle next to the model classes. Make sure to define your mappings in terms of resources, not files.

You can use the same approach for your JDBC drivers. More likely than not, your JDBC JAR does not come with an OSGi manifest, so you have to wrap it in a bundle anyway. Add a dependency on Hibernate and make your JDBC bundle a buddy of Hibernate. Logically, this is upside-down, but it has the advantage that you can exchange your JDBC drivers without changing the Hibernate bundle.

Alternatively, you can add optional dependencies to the Hibernate manifest on any JDBC driver you are planning to use, e.g.

Require-Bundle: com.microsoft.sqljdbc;resolution:=optional,
org.postgresql;resolution:=optional

Of course, this solution is a nightmare to any OSGi purist, but it does work and it is easy to set up.

There are the following major drawbacks:

You are bound to Equinox. Other OSGi implementations do not have a buddy policy equivalent. To my best knowledge, you would have to resort to DynamicImport-Package which opens the gates much wider.
The megabundle approach does not scale. It will be okay if Hibernate is the only component in your system you treat this way. If you have three or four of such heavyweight components you will end up wrapping general purpose JARs like commons-logging.jar over and over again.
And you will be in real trouble if some of the wrapped third-party dependencies occur in more than one megabundle and are used on the API of your components. With multiple copies of the same JAR in different bundles, and each bundle having its own classloader, you will end up with class cast or assignment exceptions, because classes loaded via different class loaders are always incompatible, even if they have the same fully qualified name.

Hibernate and OSGi: The problem

2008-11-05T20:40:00.005+01:00

Making Hibernate work in an OSGi environment is not trivial. Over the last couple of months, I have been experimenting with various approaches, and I'm not yet fully satisfied with the solutions I have found.

The funny thing is, there must be a whole bunch of people facing the same problem, and you do find a couple of postings or example code with the help of your favourite search engine, however, none of the articles or examples seem to match my environment or my requirements.

Now here is an outline of my approaches, which may or may not work for you. Take a look and give me some feedback.

Before talking about the solutions, we all know the answer is 42, but we haven't really worked out the question yet, so let me first list the issues you have to deal with:

Hibernate has a lot of third-party dependencies.
When you use an XML configuration file for your session factory, Hibernate must be able to load ist.
Hibernate needs to load your JDBC driver classes.
When using Hibernate mapping files for your entity classes, Hibernate needs to be able to load them as resources.
While processing the mapping files, Hibernate reads the names of your entity classes and loads them using Class.forName().
Hibernate internally uses ANTLR to parse HQL queries, and it passes a custom token class to ANTLR. Unfortunately, ANTLR requires you to specify this class by name and uses Class.forName() load the custom token class.
When using lazy loading with the CGLIB proxy generator, Hibernate injects a dependency on CGLIB into your model classes. Thus, any bundle using you model classes may have to load CGLIB classes.

There may be more issues, depending on the way you work with Hibernate. In my project environment, some issues mentioned by other people simply do not occur, while others lead to new problems.

We do not use Hibernate annotations.
Our OSGi configuration is static. All bundles are installed and started during the framework startup phase, registering all services. We do not have to worry about adding new classes to the session factory when a bundle gets installed at run-time. (For this reason, the extender pattern approach is a bit too heavyweight for my taste.)
We do not use a web container, or Spring, or anything else, just naked OSGi - well, Eclipse Equinox, in fact.
Our IDE is Eclipse and we want to be able to launch our OSGi application including Hibernate from Eclipse.
We also use Eclipse PDE batch builds.

Having described to problem in some detail, I will explain a pragmatic and a more elaborate solution in separate articles.

Why I use OSGi

2008-11-03T22:00:00.003+01:00

I want modules and controlled dependencies in my Java applications. OSGi gives me both, that's why I use it.

It's as simple as that. Really.

Yes, OSGi gives you a lot more into the bargain, most notably services, which I also use, but the unique selling point for me is modules.

When you look at the sources and binaries of any software system (not only in Java), preferably one you haven't written yourself, you see a lot of files in a directory structure. Usually, the directory structure reflects the system architecture, subdirectories corresponding to subsystems or components.

Some of these directories correspond to binary artifacts (JARs, DLLs, whatever) which you may want to reuse independently or in another system.

Now when you start pulling out this library, you have to satisfy all its runtime dependencies. So you need to take a bunch of other libraries for this library to work. Identifying the dependencies may be a tedious trial-and-error process.

There may be additional compile-time dependencies you cannot recognize by looking at the binaries only. In C/C++, there is often some global.h directly or indirectly included by every source file in your system.

Even if there was some clever architect who designed the system in terms of components and allowed dependencies, there is usually no guarantee that this architecture will be adhered to, because nothing ever prevents a developer from including or importing something they are not supposed to use.

Java has packages and classes with visibility levels, but classes and even packages are rather too fine-grained from an architect's perspective. I want my modules to be larger than packages, and I want to use them as configuration units. The modules should be in one-to-one correspondence to libraries or JARs.

This is just what OSGi provides. A plain old JAR is turned into an OSGi bundle (that's what modules are called in OSGi-land) simply by adding a couple of special headers to its manifest.

The manifest headers declare the import and export relations between modules. A bundle B may have a package with public classes, but at runtime, no other bundle C can use these classes unless bundle B exports them.

When working with Eclipse, its Plugin Development Environment (PDE) already controls bundle imports and exports at compile time. You get a warning when you import a class that was not exported.

Of course this does not prevent you from adding import or export directives to your bundle manifests, thus diluting the original architecture. But Eclipse offers tools for the architect (or yourself) to inspect the bundle hierarchy and to easily detect any unintentional dependencies.

This is just one aspect of OSGi, but it's the one I find most valuable.

Joining the Blogosphere

2008-11-02T22:45:00.002+01:00

To blog, or not to blog - that never used to be the question, but now it somehow makes sense to start.

Blogo ergo sum?

Well, recently I found myself reading a growing number of blog postings from other people working on related topics, which gave me some help on problems I was dealing with or showed me some new tools or technologies I found useful, so I think it's time to reciprocate.

Topics that are likely to show up here include Software Development in general, with a special focus on Java, Eclipse, OSGi and all sorts of tools and tricks that help you solve problems you never had without them.