Java EE and Spring: Why I couldn't care less

 Java EE or Spring - for an independent software architect, the question is just as relevant as Catholic or Protestant to an atheist. The question has recently been discussed in a number of articles mostly taking one side or the other:

• Moving from Spring to Java EE 6: The Age of Frameworks is Over
• Spring vs. Java EE and Why I Don't Care
• Java EE, Spring, and why I care
• Java EE 6 und das Ende von Spring

In this post, I'll try to point out why I'm not fully happy with either of the two.

If there's one thing that matters to me, it's modularity. To put my cards on the table, I'm a strong OSGi supporter, but I'm not even religious about that - in fact, Living without OSGi (which is what I've been doing for nine months now) would be a nice headline for one of my next posts.

Seen from outside, Spring and Java EE share a number of shortcomings:

Preaching to the Converted

When did you last visit the official homepages of Java EE or Spring? Take a moment to click on the links and imagine you've never heard about either. After visiting the homepages, you are none the wiser, all you get is the usual marketing blurb and a couple of newsbites completely meaningless to the uninitiated.

Lack of focus

Java EE and Spring both are animals of the rare species Sus ovipara lactifera velluta unknown in English speaking countries which we call eierlegende Wollmilchsau in German. In plain English, they both try to be everything to everyone and end up doing nothing in particular.

In contrast, OSGi has a clear mission statement The Dynamic Module System for Java and provides a link to its key benefits right on its homepage.

Bloat

Ever heard about lean development? Let's have a look at the official Javadocs:

• Java EE 6: 1597 classes
• Spring 3.0.5: 2312 classes

Legacy

Backward compatibility is a good thing for veteran users of a framework. Users don't want to change all of their application code just for upgrading to a new framework version.

On the other hand, backward compatibility can be extremely confusing to new users: There's two or three solutions for the same kind of problem, the legacy one is not marked as legacy or deprecated clearly enough, tutorials and example code still use the old style, and you can only resort to your favourite search engine or to trial and error to make things work consistently.

Love Thy User

Don't just list what you can do for your users, ask them what they want you to do for them. Looking at the Java EE and Spring reference documentation, I can see that both frameworks solve lots of problems I've never had in my life.

On the other hand, looking for a solution for some specific standard kind of problem, it is often surprisingly hard to find a comprehensive code example.

User documentation should be structured by user stories, not by technology.

Enough Rope

Both Spring and Java EE give you enough rope to hang yourself. It's easy to write messy and ugly code on top of both frameworks, and I've seen enough of that.

You can write loosely coupled and elegant applications on top of either, but it takes a good architect to set up guidelines and restrict the number of choices.

Development Process Integration

As an application developer, you don't just write code to run on a given framework. You want to run integration tests on your applications, you want to deploy and debug them directly in your favourite IDE, and you want your batch build and continuous integration to be able to deploy your applications.

When choosing a framework, always consider the entire process chain - some lack of tooling for your favourite framework may be reason enough to pick the second best instead.

Modularity

My ideal framework/container/app server - pick any name you like - would allow me to write truly modular applications, enforcing modularity at design time, at compile time, and at run time. The framework itself would be modular, allowing me to pick out those and only those modules actually required by my application.

It would actively support OSGi as the most mature module system for Java, and maybe a few others on top.

I expect some further degree of convergence between Java EE, Spring and OSGi within the next two years or so. There has been a lot slideware and a number of proof-of-concept or early adopter projects, but my feeling is that none of this has reached production quality yet.

To sum up, frameworks in general are alive and kicking, but the age of monolithic all-in-one frameworks is definitely over.

Java EE 6 Integration Testing with Spring

Half a year ago, I created and published the jeeunit project to fill in some gaps for running integration tests on Java EE 6 components in an embedded EJB container, mainly targeted at Embedded Glassfish 3.x. One of the alternatives I considered and dismissed before writing my own solution was the Spring Test Context.

At the time, I was left with the impression you had to pollute your Java EE components with Spring annotations like @Transactional to make declarative transactions work in the Spring container, and I did not (and still do not) like the idea of writing XML just to create a couple of Java beans and wire them up.

After taking a closer look at Spring 3.0.x these days, I realized a number of facts which are not quite obvious from the Spring reference manual:

• You can do most of your Spring configuration (but sadly not all of it!) in Java instead of XML, which makes your application context type-safe and easier to read, and gives you refactoring support from your IDE.
• Spring supports some (but not all) of the Java EE 6 annotations as alternatives to its own flavours.
• You can run Stateless Session Beans and JPA entities in a Spring container with declarative transactions without introducing dependencies on Spring APIs or annotations.

jeeunit now includes an example project jeeunit-example-test-spring demonstrating this approach. The details are documented in the jeeunit Wiki page Testing on Spring.

This solution is not intended to replace the original in-container-testing approach of jeeunit, because transactional Spring beans and EJBs have different run-time semantics. However, for many integration tests, these differences are irrelevant.

Using the Spring Test Context instead of Embedded Glassfish and jeeunit has two main advantages:

• Spring starts noticeably faster than Embedded Glassfish.
• Launching a single test from your suite is much easier than with jeeunit. E.g. in Eclipse, you can simply use Run as JUnit Test just as with plain old unit tests.

CDI - A Major Risk Factor in Java EE 6

Update 18 Dec 2010: This post mainly relates to Glassfish 3.0.1. As of 3.1-b33, the situation has improved a lot.

Evaluating platforms and architectures for a new enterprise web application earlier this year, around March/April, scheduled to be released in 2011, I was bold enough to settle on Java EE 6.

In retrospective, I do not regret this decision, but I had to spend a significant amount of time analyzing and debugging platform problems. Based on this experience with Java EE 6, which may not be representative of course, the only advice I can give is: When Java EE 7 is released, wait a year or two before using it for production, unless you have the time and resources to act as beta tester.

The Java EE 6 specifications were released in December 2009, together with Glassfish v3, the first "production-quality release" of a Java EE 6 server. "Marketing release" would have been more appropriate, since going by the number of bugs in Glassfish 3.0, the main driving factor for the release date was quite obviously not a quality benchmark, but simply the need for Sun as the Java EE champion to release a working server simultaneously with the specifications.

The Problem

Java EE 6 was advertised as the lightest Java Enterprise Edition ever, with Contexts and Dependency Injection (CDI, JSR-299) as one of the most prominent innovations. However, for my project, CDI in the guise of Weld, its reference implementation, turned out to be the largest single source of problems. In addition to a number of functional bugs where injection would not work in certain scenarios, there are at least two severe memory issues:

1. Weld causes memory leaks on application redeployment.
2. Even in the first deployment of an application, Weld uses enormous amounts of memory.

The first issue had plagued us for months, and my team had grown accustomed to kill all Glassfish and Eclipse processes after three or four redeployments and to re-initialize their entire workspace, which was rather a pain.
Investigating the memory leaks using the Eclipse Memory Analyzer, I realized that Weld claimed almost 90 % of the heap memory of my application. Most of this was due to instances of org.jboss.weld.introspector.weld.jlr.WeldClassImpl, which appears to be a reflective model of a class inspected by the CDI BeanManager. Some instances of this class were as large as 1 MB.

I should add that our application uses Wicket for the web front end, injecting EJBs into custom Wicket components. Custom components inherit from Wicket base classes with a fairly deep inheritance hierarchy and a large number of methods. This accounts for the size of the reflective models.

Our usage of CDI was fairly basic, so it was just an hour's work to replace all @Inject annotations by Java EE 5 style @EJB annotations. After that, the heap usage of our application was reduced by 90 %, we no longer had any memory leaks, and we finally had a hassle-free edit-save-debug cycle in our development environment.

The figure of 90 % heap usage by Weld was observed on Glassfish 3.0.1, running on a 32-bit JVM on Windows XP.

This week, I retested the same application with Glassfish 3.1-b26 (using a newer version of Weld), running on a 64-bit JVM on Ubuntu 10.04. The result was not as drastic as before, but still about 50 % of my heap memory was consumed by Weld.

Alternatives

This article is mainly about problems with the CDI reference implementation Weld, not about CDI as such. But these implementation details have a ripple effect:

• Glassfish 3.x still is the only certified Open Source Java EE 6 server. It uses a newer version of Weld in the current 3.1 promoted builds, but I can see no indication of the Weld issues being solved in time for the 3.1 release.
• JBoss AS 6 is likely to be the next in line for full Java EE 6 compliance. However, JBoss also includes Weld, so you're bound to run into the same kind of problems on JBoss.
• There are alternative CDI implementations, CanDI from Caucho and OpenWebBeans from Apache. Unfortunately, unlike JPA, there is no pluggable CDI provider interface that would allow you to simply replace Weld by some other CDI implementation on Glassfish or any other server.
• I tried running my application on Resin 4.0.8 to see if CanDI was any better than Weld. However, Resin had bugs in the JPA and EJB areas and I was unable to deploy my application, let alone run and monitor it. (Resin only aims at supporting the Java EE 6 Web Profile, but that would have been sufficient for my case.)

Conclusion

All in all, if you want to work with Java EE 6, there is currently no way around Weld. And until Weld reaches production quality, the only way to build production quality applications on Java EE 6 is to avoid using CDI.

Free and Open Source Software is all about choice - so I would really love to see a pluggable CDI provider interface that would allow users to work with Glassfish and some CDI implementation other than Weld.

The second best solution might be to join forces from different CDI project to make Weld stable enough for production usage.

The third option would be another fully Java EE 6 compliant Open Source server (maybe Geronimo?) not including Weld.

No idea if any of these options is realistic, but this is on my wishlist for 2011.

OpenJPA and OSGi

In my previous review of OSGi support in JPA 2.0 persistence providers, I promised to explain my setup for using OpenJPA in an OSGi environment.

The solution I'm going to outline is by no means final, and in the meantime, my first shot has been greatly simplified using some valuable input from the Apache Aries User mailing list.

In the past, brute force solutions like DynamicImport-Package were more or less the only way to let a JPA provider bundle access all the required classes for a persistence unit from application bundles. Since Release 4.2, the OSGi Enterprise Specification standardizes a whole lot of OSGi service wrappers for well-known Java EE APIs, including JPA, JDBC, JTA and JNDI, so this should be the preferred way of integrating  JPA providers into an OSGi environment.

The Apache Aries project provides (partial) implementations for some of the OSGi enterprise services and a whole lot of other things beyond the scope of this article. Fortunately, all their service implementations are nicely modular, so it is quite easy to use just the JPA and JNDI Services from Aries to make OpenJPA work on OSGi, following the OSGi Enterprise spec rather closely (though not fully).

Here is an overview of the required steps:

• Get OSGi bundles for OpenJPA and all its dependencies, plus the required Aries bundles.
• Publish the datasource to be used by your persistence unit as an OSGi service.
• Make sure that your persistence.xml satisfies a number of restrictions.
• Run the OpenJPA enhancer and package all classes and metadata of your persistence unit into a bundle with a special manifest header.
• Start your OSGi framework. The JPA Service will publish an EntityManagerFactory service for each persistence unit.
• In your client bundles, import your persistence bundle, OpenJPA and javax.persistence, get the appropriate EntityManagerFactory from the service registry and use it just like in an unmanaged Java SE environment.

Each step is explained in more detail in the following sections. I'm using Java 1.6.0, Maven 2.2.1 for collecting the dependencies, Eclipse 3.5.2 for doing most of the work, and the corresponding Equinox version as OSGi runtime.

Get the dependencies

The main JAR in the OpenJPA binary distribution is an OSGi bundle, but most of the included dependencies are plain old JARs, so you need to go shopping for osgified versions of all required libraries. All the required bundles for this example can be found in the SpringSource Enterprise Bundle Repository.

The following is the best part of a Maven POM for copying all the dependencies to a local directory target/platform/plugins.

You need to add the OSGi framework, javax.persistence (I copied both of them from Eclipse 3.5.2) and a JDBC driver bundle for your database. Then copy the whole lot to a new directory to be used as target platform for your Eclipse workspace.

<project>
  <properties>
    <aries.version>0.2-incubating</aries.version>
    <output.dir>${project.build.directory}/platform/plugins</output.dir>
  </properties>
  <dependencies>
    <dependency>  
      <groupId>javax.transaction</groupId>  
      <artifactId>com.springsource.javax.transaction</artifactId>  
      <version>1.1.0</version> 
    </dependency>    
    <dependency>
      <groupId>net.sourceforge.serp</groupId>
      <artifactId>com.springsource.serp</artifactId>
      <version>1.13.1</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries</groupId>
      <artifactId>org.apache.aries.util</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jndi</groupId>
      <artifactId>org.apache.aries.jndi.api</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jndi</groupId>
      <artifactId>org.apache.aries.jndi.core</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jndi</groupId>
      <artifactId>org.apache.aries.jndi.url</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jpa</groupId>
      <artifactId>org.apache.aries.jpa.api</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jpa</groupId>
      <artifactId>org.apache.aries.jpa.container</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>com.springsource.org.apache.commons.collections</artifactId>
      <version>3.2.1</version>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>com.springsource.org.apache.commons.lang</artifactId>
      <version>2.2.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>com.springsource.org.apache.commons.pool</artifactId>
      <version>1.5.3</version>
    </dependency>
   <dependency>
      <groupId>org.apache.openjpa</groupId>
      <artifactId>openjpa-all</artifactId>
      <version>2.0.1</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>com.springsource.slf4j.api</artifactId>
      <version>1.5.0</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>com.springsource.slf4j.log4j</artifactId>
      <version>1.5.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.log4j</groupId>
      <artifactId>com.springsource.org.apache.log4j</artifactId>
      <version>1.2.15</version>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-dependency-plugin</artifactId>
        <executions>
          <execution>
            <id>copy-bundles</id>
            <phase>package</phase>
            <goals>
              <goal>copy-dependencies</goal>
            </goals>
            <configuration>
              <outputDirectory>${output.dir}</outputDirectory>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
  <repositories>
    <repository>
      <id>com.springsource.repository.bundles.external</id>
      <name>SpringSource Enterprise Bundle Repository - External Bundle Releases</name>
      <url>http://repository.springsource.com/maven/bundles/external</url>
    </repository>
  </repositories>

</project>

Register a data source service

A persistence unit requires a data source, either from a javax.sql.DataSource or from a JDBC driver URL. This may be a matter of taste - in this article, I'm focusing on the DataSource.

Create a DataSource for your database using any suitable driver and register it as an OSGi service with the javax.sql.DataSource interface and optionally some additional identifying properties of your choice, e.g. name=myds, in case you have more than one DataSource in your system.

It is useful to do this from a separate bundle, so that this bundle will be the only one in your system with a direct dependency on the JDBC driver.

Adapt your persistence.xml

If you have a persistence.xml which works on Java SE oder Java EE, you will probably have to modify it  to make it work with OSGi and Aries. Here is an example:

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
  <persistence-unit name="myapp" transaction-type="RESOURCE_LOCAL">
  <non-jta-data-source>osgi:service/javax.sql.DataSource</non-jta-data-source>   
    <mapping-file>META-INF/orm.xml</mapping-file>
    <exclude-unlisted-classes>true</exclude-unlisted-classes>
  </persistence-unit>
</persistence>

First, you need to specify the RESOURCE_LOCAL transaction type explicitly, or else OpenJPA will complain about a missing transaction manager. This is because Aries creates a container EntityManagerFactory and not a stand-alone one.

Second, your data source has to be a non-jta-data-source. Note the special syntax for accessing your data source as an OSGi service via JNDI. To make this work, you need to include the Aries JNDI Service bundles. To select a specific data source by properties, you can append an LDAP filter to the JNDI URL, e.g. osgi:service/javax.sql.DataSource/(name=myds).

Third, you need to prevent the persistence provider from scanning your persistence unit for entity classes, since this may lead to classloader problems. To do so, either list all your classes explicitly if you use JPA annotations, or include all mapping files if you use XML metadata and set exclude-unlisted-classes to true to prevent the provider from scanning your bundle.

If you get funny exceptions like

org.apache.openjpa.persistence.PersistenceException: Error extracting class information from "bundleresource://23.fwk1421571929/".
Caused by: java.io.FileNotFoundException: /myapp/plugins/test.openjpa/bin (Is a directory)

then check if you have set this option. (This problem may be specific to Eclipse, but I'm not sure about that.)

Package your persistence unit bundle

To turn your persistence unit into a persistence bundle, make sure you have all mapping data resources and all entity classes included. In addition to the usual OSGi headers, insert the following one into your manifest:

Meta-Persistence: META-INF/persistence.xml

This is the hook for Aries to discover your persistence bundle using the extender pattern. If all goes well, an EntityManagerFactory will be registered as an OSGi service.

In theory, your persistence bundle should only depend on javax.persistence and not on org.apache.openjpa. In practice, you currently need to include this unwanted import, or else you will get exceptions of the following kind:

java.lang.NoClassDefFoundError: org/apache/openjpa/enhance/PersistenceCapable

Apparently, OpenJPA tries to load one of its own classes via your persistence bundle class loader, which is incorrect.

Use the EntityManagerFactory service in your clients

Finally, to work with your persistence unit from client bundles, just get hold of the appropriate EntityManagerFactory service any way you like, using the filter (osgi.unit.name=myapp).


Update 26 Sep 2010: I have submitted some sample code for inclusion into the OpenJPA test suite. The sample is attached to OPENJPA-1815.

OSGi Support in JPA 2.0 Persistence Providers

About three years ago, I started working with Hibernate in an OSGi environment, and at that time, I had to roll my own solution for resolving all sorts of classloader issues.

Since then, with OSGi 4.2 and JPA 2.0, there have been major specification releases, followed by implementations; OSGi is growing ever more popular, and the current release of the JPA standard has integrated a lot of functionality which used to be available in proprietary API extensions only.

The OSGi 4.2 Enterprise Specification includes JPA, JTA and JDBC services, so in theory, using JPA in an OSGi environment should stop being an issue, as soon as this specification is supported by all OSGi and JPA implementations.

In practice, the specifications are not yet fully covered by the available implementations, and the degree of OSGi support varies a lot.

In the past few months, I have worked with the current versions of the three certified JPA 2.0 persistence providers, and in my opinion, the ranking for OSGi support is quite clear:

1. Eclipselink
2. OpenJPA
3. Hibernate

Some more details on each provider:

Eclipselink

Eclipselink has a fully osgified distribution containing OSGi bundles of Eclipselink itself and all dependencies. To make your entity classes visible to Eclipselink, all you need to do is add a JPA-PersistenceUnits manifest header to each bundle containing one or more persistence units. This is enough for Eclipselink to load all your XML metadata and entity classes.

No ugly hacks, no buddy policies or fragments, it just works out of the box.

So for a quick start with JPA under OSGi, Eclipslink is definitely the best choice, even though the OSGi 4.2 JPA Service Specification is not yet fully supported (for instance, the Meta-Persistence header does not work), but this will be of little or no concern to most users.

Leaving OSGi aside, I have been bitten by a number of bugs in Eclipselink, discussed in earlier articles. If you do not use any of these features, you may be very happy with Eclipselink. For me, this set of bugs is currently a no-go, so I have migrated my OSGi applications from Eclipselink to OpenJPA.

OpenJPA

The silver medal goes to OpenJPA: OSGi does not appear to be on the top of the agenda of the OpenJPA core developers, but since OpenJPA is used under OSGi by Aries, another Apache project, there is at least a certain level of OSGi support.

In the binary distribution, the OpenJPA aggregate JAR is an OSGi bundle, but most of the dependencies are plain old JARs, so you need to find osgified versions of the dependencies on your own. The same goes for the Maven artifacts.

Once you have downloaded the appropriate OSGi bundles, you still have to do some extra work to ensure that OpenJPA finds your persistence units and your entity classes, and there are a couple of minor issues with the enhancer and with user-defined value handlers.

I will explain the details of my setup and some of the problems that should be addressed in OpenJPA in my next post.

Hibernate

Sadly, regarding OSGi support, Hibernate 3.5.x is not much different from the 3.2.x release I discussed in my earlier articles. The distribution contains plain old JARs, no OSGi bundles. The latest osgified version available from the SpringSource Enterprise Bundle Repository is 3.4.0, and the SpringSource versions never used to work for me, so I expect that the steps described in my 2008 article are still required to make Hibernate work on OSGi.

Disclaimer: I did not try to make Hibernate 3.5 work on OSGi, due to its lack of support of a number of JPA 2.0 features used heavily by my applications. For this reason, I have stopped using Hibernate, both on OSGi and on Java EE.

JPA 2.0: Ordered Collections

In earlier posts, I wrote about problems with persistent maps, a new feature introduced in JPA 2.0. Ordered collections with explicit order columns are another JPA 2.0 feature which, again, is not yet fully robust in all JPA 2.0 compliant and TCK tested implementations.

I'm currently working with a simple JPA model for OpenStreetMap (OSM). There are ways and nodes, and each way has a sequence of nodes. Of course the order of the nodes is important, and a node may be used more than once for a given way.

Thus, the nodes collection has to be a List, not a Set, and we need an explicit order column specifying the sequence number of the nodes along the way. Sorting the nodes by ID would not make any sense, obviously.

Here is a snippet from the model:

@Entity
@Table(name = "ways")
public class OsmWay
{
    @Id
    private long id;
    
    @ManyToMany(cascade = CascadeType.ALL)
    @OrderColumn(name = "sequence_id")
    @JoinTable(name = "way_nodes", 
               joinColumns = @JoinColumn(name = "id"), 
               inverseJoinColumns = @JoinColumn(name = "node_id"))
    private List<OsmNode> nodes = new ArrayList<OsmNode>();
}    

This entity is represented by two tables:

CREATE TABLE ways
(
  id bigint NOT NULL
);

CREATE TABLE way_nodes
(
  id bigint NOT NULL,
  node_id bigint NOT NULL,
  sequence_id integer,
);

I've tested this scenario on Hibernate 3.5.3, Eclipselink 2.1.1 and OpenJPA 2.0.1: Hibernate and OpenJPA pass, Eclipselink fails with 2 issues.

First, the generated DDL is incorrect: There is a PRIMARY KEY (id, node_id) for way_nodes. This should be (id, sequence_id).

Second, the order of the list items is not maintained in all contexts. It is ok when the collection is lazily loaded, e.g.

OsmWay way = em.find(OsmWay.class, wayId);
List<OsmNode> nodes = way.getNodes();

but it is broken when using a fetch join:

String jpql = "select distinct w from OsmWay w join fetch w.nodes";
TypedQuery<OsmWay> query = em.createQuery(jpql, OsmWay.class);
List<OsmWay> ways = query.getResultList();
OsmWay way = ways.get(0);
List<OsmNode> nodes = way.getNodes();

The funny thing is, the nodes are neither ordered by sequence_id nor by node_id. In my test case, the way has 5 nodes, one of which appears twice in different positions.

So here is some more evidence for my claim that the JPA 2.0 TCK is insufficient.

Ubuntu VPN Client Setup

Not a Java topic this time: Setting up an IPSec VPN client on my Linux box was a complete nightmare. The final solution is rather easy, but I wasted hours and hours trying different clients with different configurations which simply did not work...


This is the environment:

• Linux client machine running Kubuntu 10.04.
• Network interface eth0 on client configured via DHCP from a Netgear Router.
• The router acts as DNS server.
• The home network is 192.168.0.0/24.
• The company network has a FortiGate firewall with an IPSec VPN.
• The IPSec tunnel uses a pre-shared key and XAUTH username/password authentication.
• The company network is 172.16.0.0/16, VPN clients get a virtual IP adress from 172.19.0.0/24 via DHCP-over-IPSec.

The problem:

• There is no official Linux VPN client for FortiGate. (This is rather ironic, given that Fortinet products include GPLed Linux code, which became public after a GPL violation injunction in 2005.)
• Ubuntu has just too many too buggy VPN clients and too little readable documentation on VPN setup. (I tried KNetworkManager, kvpnc, ike and openswan, mostly getting only error messages I could not make sense of. ike did work ocasionally, but most of the time, the incoming packets from the tunnel would not reach my client port.)

My solution:

• strongswan for the IPSec tunnel
• dnsmasq for merging the public and private DNS zones
• eth0 now uses manual configuration instead of DHCP

Now for the details: The XAUTH credentials and the pre-shared key are stored in /etc/ipsec.secrets:

: XAUTH myusername "MyPassword"
: PSK "MyPreSharedKey"

The general and connection specific settings for IPSec are in /etc/ipsec.conf:

config setup
     nat_traversal=yes
                                                             
conn office
     left=%defaultroute
     leftsourceip=172.19.0.99
     right=firewall.example.com
     rightsubnet=172.16.0.0/16
     auto=add
     authby=xauthpsk

The only flaw is the explicit virtual IP address: I tried using leftsourceip=%config to let the client use an address assigned by the server, but this did not work as FortiGate does not support the ModeCfg protocol (at least not in the firmware version we are currently using), and I could not find out how to set up DHCP-over-IPSec with strongswan, so I ended up using an explicit address not part of the DHCP pool.

After editing these files and starting the tunnel via

ipsec restart
ipsec up office

I could successfully ping addresses from the office network.

In contrast both to the Windows FortiClient and the Shrew VPN Client (ikea), strongswan does not require a virtual network device and does not change the routing table. You can see the IPSec tunnel and the corresponding policies using the commands

ip xfrm state
ip xfrm policy

The log output from the strongswan IKEv1 daemon named pluto can be found in /etc/auth.log.

Now for DNS: Of course I don't want all my DNS requests for things like schweinebildchen.com to hit the office DNS server while the tunnel is up, so I'm using dnsmasq as a simple DNS proxy which redirects all DNS requests from my local machine either to my home router or to the office DNS server via the tunnel, depending on the domain.

The dnsmasq configuration in /etc/dnsmasq.conf is as simple as this:

server=/example.com/172.16.0.2
server=192.168.0.1

Finally, I changed the configuration of my network interface eth0 from auto to manual, using the IP address 192.168.0.5 and set this as nameserver in /etc/resolv.conf.

This means that all DNS requests are first handled by dnsmasq on my local machine, requests for example.com get forwarded to the office DNS server (via the IPSec tunnel), and everything else gets routed to my Netgear router (and from there to my ISP).

A Fish in the Clouds

How long does it take to find a hosting company and get an enterprise web application up and running from scratch? Could be weeks if there are enough lawyers and pointy-haired bosses involved. And even if you can directly talk to the providers, it usually takes a couple of days to request quotes and compare them.

That's the way we started for our current project, but now we've decided to have a go at cloud computing. (So at last this adds another buzzword to my CV...)

I vaguely remembered Arun Gupta's blog about Glassfish on Amazon EC2, and using that together with the Ubuntu EC2 Starter's Guide, it just took me about half an hour to set up a virtual machine in the Amazon Cloud with Ubuntu Server 10.04, JDK 1.6.0_20 and Glassfish 3.0.1, start the Glassfish domain and access the admin console from my local web browser.

And downloading the 78 M Glassfish zip file on the EC2 instance from Oracle's server took less than 2 seconds...

Not bad for a start. I would have expected to spend about a day to get as far as that.

I was also using Ubuntu 10.04 on my local machine, working with the command line interface from the
ec2-api-tools Ubuntu package most of the time.

Of course it will take some more time to set up the database server, deploy our web app and to secure the system. And it's too early to tell if we really require all the elasticity of EC2, which is likely to be more expensive in the long run than a conventional dedicated server cluster.

At any rate, installing a site in virtually no time and running it for less than 10 cents per hour is rather impressive. Thumbs up for Amazon Web Services!

Building a Java EE 6 Web Application with Eclipse Helios, Maven and Glassfish

This is a short tutorial showing how to use Eclipse 3.6 (Helios) with Glassfish and Maven for building and running a Java EE 6 web application from a multi-module source tree.

We are going to import and run the Wicket gmap2-examples, demonstrating the use of Google Maps via Apache Wicket, but this is purely incidental - even if you prefer mainstream JSF or another web framework to Wicket, you may find this tutorial useful. There are absolutely no Wicket specifics involved; gmap2 was just a handy small but non-trivial example.

If you haven't worked with Maven before, you will get an idea how Maven can save you a lot of work managing the third-party dependencies of your project automatically.

This is what you'll see in the end:

There's quite a few things on your shopping list before you can start. If you are reading this, you probably already have the following on your disk:

• Eclipse 3.6 for Java EE Developers
• Glassfish 3.0.1

You'll need to install these features into Eclipse:

• Maven Integration for Eclipse (also known as m2eclipse)
• Maven Integration for Eclipse (Extras)
• Glassfish Java EE Application Server Plugin for Eclipse

Eclipse 3.6 has a new way of installing plugins via Help | Eclipse Marketplace. This is an integrated web client which lets you search for plugins by free text, so you no longer have to copy and paste update site URLs into the Update Manager - but you can still do so if you prefer.

The m2eclipse book has step-by-step instructions and screenshots explaining the installation process: start here with the m2eclipse installation. When you get to the Extras, the only ones required for the rest of this tutorial are Maven Integration for WTP and Maven SCM Integration. The WTP integration will let Eclipse recognize your Maven projects with war packaging as Dynamic Web Projects.

The Maven SCM Integration lets you fetch Maven projects directly from source code repositories like Subversion, Mercurial or others. Note that this integration simply invokes the corresponding command line clients like /usr/bin/svn.

Having installed m2clipse via the Eclipse Marketplace, use the same procedure to install the Glassfish Plugin.

After restarting Eclipse, set your Glassfish preferences via Window | Preferences | Glassfish Preferences. For this tutorial, you should check Start the Glassfish Enterprise Server in verbose mode and uncheck the other items.

Next, define a server runtime environment via Window | Preferences | Server | Runtime Environment | Add... Select the server type Glassfish | Glassfish Server Open Source Edition 3 (Java EE 6), check Create a new local server and click Next. Select your Glassfish installation directory - this is required to be the parent of the modules and domains directories, e.g. /home/hwellmann/glassfish-3.0.1/glassfishv3/glassfish. Click Next and fill in the domain name and the administrator credentials. If you did not change the Glassfish defaults, you can probably just click on Finish.

The Servers view has opened automatically. You can select your server and click on the Run button to  launch Glassfish from Eclipse. The Eclipse plugin launches Glassfish indirectly via an asadmin subprocess.

You will see some Glassfish log messages in the console. To stop Glassfish, do not hit the stop button in the Console view. This will just kill the asadmin process, but not Glassfish itself, and Eclipse and Glassfish will get terribly out of sync. Make sure to select the Servers view and hit the stop button there to avoid trouble.

Now for the interesting part: Let us import and build the gmap2 example project.

This project has a parent project with two subprojects, or modules in Maven terms. Maven requires you to store the module subprojects of a parent project in subdirectories of the parent directory. Eclipse normally cannot handle overlapping or nested directory structures for projects in the same workspace. Fortunately, m2eclipse works some magic to flatten the project structure, making Eclipse happy.

To import the example sources directly from the Subversion repository, select File | Import... | Maven | Check out Maven Projects from SCM and click Next. Fill in the SCM URL

https://wicket-stuff.svn.sourceforge.net/svnroot/wicket-stuff/tags/wicketstuff-core-1.4.9.2/gmap2-parent

and select the SCM provider svn, then click Next and Finish.

After a while, you will see three projects in your workspace:

• gmap2
• gmap2-examples
• gmap2-parent

The parent project gmap2-parent has two subfolders gmap2 and gmap2-examples which are also represented as top-level Eclipse projects - this is a special feature of the m2eclipse integration.

The gmap2-examples subproject is a web application, as indicated by the Maven packaging type war. Using this and other information specified in the Maven POM, m2eclipse was able to turn this Eclipse project into a Dynamic Web Modules project - have a look at the Project Facets in the project properties to convince yourself.

Opening the project, you will notice two classpath containers Maven Dependencies and Web App Libraries. The latter contains just one item gmap2 with an open folder icon, indicating a reference to another subproject in our workspace that will go into WEB-INF/lib. The Maven Dependencies container displays all JARs required by our project, e.g. wicket-1.4.9.jar and slf4j-api-1.5.8.jar. All these were downloaded automatically by Maven into your local Maven repository on your hard drive, and m2eclipse makes Eclipse reference them from that location.

We are now ready to launch the web app: Select the gmap2-examples project folder and then Run As... | Run on Server from the context menu. Select the existing Glassfish server you created before and click Finish. After a while, you see the gmap2-examples welcome page in a web browser window.

Click on one of the links, e.g. marker listener, to see Google Maps in Firefox in Eclipse via Wicket on Glassfish on Java on Linux. (OK, maybe it's not Firefox or Linux on your machine...)

Note that the Java compilation and the WAR assembly and deployment were done by Eclipse, not by Maven.

However, you can also run a Maven build from Eclipse. To build our entire project hierarchy, select gmap2-parent and open Run As | Maven build... from the context menu. Enter the goals clean install and click Run. You will see the Maven messages in the console window. When Maven has finished, click Refresh (F5) on gmap2-parent, and open gmap2-examples/target to see the WAR file compiled by Maven.

JPA 2.0: Querying a Map

Welcome back to more merriment with Maps in JPA 2.0!

After watching 3 out of 4 persistence providers choke on a model with a map in the previous post, let us now continue our experiments and see how our guinea pigs can handle JPQL queries for maps.

Recall that the JPQL query language has three special operators for building map queries: KEY(), VALUE() and ENTRY().

Now let us try and run the following query on a slightly modified model, compared to the previous post.

select m.text from MultilingualString s join s.map m where KEY(m) = 'de'

The corresponding model is:

@Embeddable
public class LocalizedString {

    private String language;

    private String text;

} 

@Entity
@Table(schema = "jpa", name = "multilingual_string")
public class MultilingualString {

    @Id
    @GeneratedValue(strategy = GenerationType.SEQUENCE)
    @Column(name = "string_id")
    private long id;

    @ElementCollection(fetch=FetchType.EAGER)
    @MapKeyColumn(name = "language_key")
    @CollectionTable(schema = "jpa", name = "multilingual_string_map", 
                     joinColumns = @JoinColumn(name = "string_id"))
    private Map<String, LocalizedString> map = new HashMap<String, LocalizedString>();
}

This time I've changed the model so that the map key is stored in its own column, which gives Hibernate and Eclipselink at least a chance to digest the model and proceed to the query. OpenJPA is fine with either version of the model.

DataNucleus is out of the game by now. I even tried replacing the @Embeddable by an @Entity and a few other things to cheat it into accepting my model, but in the end I gave up.

Now, Ladies and Gentleman, the winner and sole survivor is: OpenJPA again!

Both Hibernate and Eclipselink fail, merrily throwing exceptions. Hibernate only seems to have stubbed out the KEY() and VALUE() operators in their parser code (see HHH-5396 for the gory details and elaborate stack traces).

And Eclipselink's famous last words are:

Error compiling the query [select m.text from MultilingualString s join s.map m where KEY(m) = 'de'], 
line 1, column 9: unknown state or association field [text] of class [LocalizedString]. 

Not sure what the poor soul is trying to tell me.

To sum up: Should you ever consider working with persistent maps à la JPA 2.0, beware! Here be dragons...