18 December 2010

Glassfish Logging with slf4j (Part 2)

My previous post on the same subject is one of the most popular articles in this blog, and since Glassfish still does not officially support slf4j, this update may be helpful to anyone hitting my blog via Google trying to improve their logging experience with Glassfish.

To redirect (almost) all log messages from Glassfish and your applications to the same log file and/or console, follow this recipe:
  • Copy these JARs to [install-root]/lib/endorsed:
    • jul-to-slf4j.jar
    • slf4j-api.jar
    • logback-classic.jar
    • logback-core.jar
  • Build a JAR containing your logback.xml configuration at root level and put it in the same place. [Update 26 Jul 2011: Actually, this is no longer required: Simply create a logback configuration file somewhere in your local file system and add the system property -Dlogback.configurationFile=file:/path/to/my/logback.xml to your Glassfish configuration.]
  • Define a new system property in the jvm-options section of your domain.xml:
    -Djava.util.logging.config.file=${com.sun.aas.instanceRoot}/config/my_logging.properties
  • Create the corresponding file my_logging.properties with the following contents:
  • handlers = org.slf4j.bridge.SLF4JBridgeHandler
    com.sun.enterprise.server.logging.GFFileHandler.flushFrequency=1
    com.sun.enterprise.server.logging.GFFileHandler.file=${com.sun.aas.instanceRoot}/logs/server.log
    com.sun.enterprise.server.logging.GFFileHandler.rotationTimelimitInMinutes=0
    com.sun.enterprise.server.logging.GFFileHandler.logtoConsole=false
    com.sun.enterprise.server.logging.GFFileHandler.rotationLimitInBytes=2000000
    com.sun.enterprise.server.logging.GFFileHandler.alarms=false
    com.sun.enterprise.server.logging.GFFileHandler.formatter=com.sun.enterprise.server.logging.UniformLogFormatter
    com.sun.enterprise.server.logging.GFFileHandler.retainErrorsStasticsForHours=0
    
When you restart Glassfish, you should only see two log messages in the old java.util.logging format, all the rest goes via slf4j and logback to the appenders configured in your logback.xml. The two remaining log messages are issued before Glassfish reads the domain.xml configuration with the adapted configuration, so there is no easy way of avoiding them.

I have been using this approach on various Glassfish versions from 3.0.1 to 3.1-b33. It is rather ugly, but nonetheless effective.

All this may sound rather mysterious, so I'd better add some explanations:

jul-to-slf4j.jar is a bridge for redirecting java.util.logging via the slf4j API to any logging backend of your choice (logback in this case). To make the redirection work, the bridge has to be visible to the classloader evaluating the logging configuration. Since Glassfish starts logging very early in its bootstrapping phase, the lib/endorsed folder is the only place where the bridge has the desired effect.

Unfortunately, class file folders seem to be unsupported for endorsed libraries. Of course, it would be easier to simply put the logback.xml file in lib/endorsed/classes where you can directly edit it. But for a quick change to your logging configuration, you can always use an editor which supports editing ZIP/JAR file contents in place.

I created the contents of my_logging.properties by trial and error, starting with a one-liner containing only the handler. This caused a couple of exceptions as Glassfish seemed to be missing the GFFileHandler settings, so I copied them over from the original logging.properties.

The same procedure should also work for other slf4j backends, so you could replace the logback JARs by a log4j.jar.

An Update on Memory Issues in Glassfish and Weld

Using the most recent Glassfish milestone build 3.1-b33, I took another look at the memory issues that had plagued my project on Glassfish 3.0.1. The last build I tried was b30, which did not even let me deploy my application.

Just in time for Christmas, there are Good Tidings - Weld has improved a lot, and I can no longer see any suspicious memory usage related to Weld. I took my Wicket+EJB+OpenJPA+PostgreSQL application packaged in a single WAR and deployed it to several different Glassfish versions from the command line. Then I used the Eclipse Memory Analyzer to a create a heap dump of the running Glassfish instance.

The Total Heap column in the following table is the total size of the heap as reported by the Memory Analyzer. The Shallow, Retained, and Percentage Columns refer to the amount of heap memory consumed by instances of org.jboss.weld.introspector.weld.jlr.WeldClassImpl

Glassfish Version Total Heap Shallow Retained Percentage
3.0.1 412.1 MB 84.680 378.215.056 87.52 %
3.1-b26 335.3 MB 85.144 205.527.048 58.38 %
3.1-b33 149.6 MB 84.216 7.041.704 4.49 %

The retained heap of an object is the set of objects that will be garbage collected when the given parent object is garbage collected.

So in b33, Weld still holds 5 % of the total heap - whether or not this is still too much or just reasonable is surely debatable. But I'm really happy to see that Weld has improved a lot, so I would no longer consider it as a no-go.

The version of my application I used for these measurements is the last one that actually used Weld, before I decided in August to stop using CDI to get rid of most memory issues.

I took some more measurements with the successor version in which CDI was eliminated essentially just by replacing @Inject by @EJB throughout the sources. This is really the only difference between the two versions, and here are the total heap sizes:

Glassfish Version with @Inject with @EJB
3.0.1 412.1 MB 44.4 MB
3.1-b33 149.6 MB 138.3 MB

The illustrates, as claimed in my previous post, that I was able to save 90 % of heap memory by not using CDI on Glassfish 3.0.1. Doing the same on Glassfish 3.1-b33, I can still save about 9 %.

But I was really surprised to see the heap usage of my application increase by a factor of 3 between Glassfish 3.0.1 and 3.1-b33.

The Memory Analyzer reveals the culprit to be org.glassfish.hk2.classmodel.reflect.impl.TypesCtr with a retained heap of about 80 MB.

I have no clue what this class is doing, but it smells like another case of an oversized reflective model of the application, which was just the problem with earlier Weld versions.

GLASSFISH-15266 is the related issue in Glassfish JIRA.

10 December 2010

The Interface Antipattern

Keeping interfaces and implementations separate is a useful design pattern for building modular systems. If your clients only depend on a service interface, you can switch service implementations without your clients even noticing.

Now there can be too much of a good thing, and the quality of a software system is certainly not measured by the ratio of interfaces to classes.

When you are planning to implement a FrobnicatorService, think twice before creating an IFrobnicatorService interface and a FrobnicatorServiceImpl. How many different Frobnicator implementations are there going to be? If you do need at least two implementations with distinct behaviour, then go ahead and create the interface. If there's only one implementation, then don't bother with the interface.

Resist the temptation of speculative generalization: "Oh, there might be a performance bottleneck, and in that case an alternative CacheingFrobnicatorServiceImpl might help, so I really should start with a service interface now." Think twice: you ain't gonna need it.

And if you do need the interface, you can still pull it out when you need it. Most IDEs have an automatic refactoring Extract Interface. (Unfortunately, none of the major Java IDEs seems to have the opposite refactoring Merge Interface and Implementation.)

Even in Java EE, the times of interface inflation are gone with EJB 3.1, thanks to the no-interface local view. For a stateless session bean, it is enough to implement

@Stateless
public class FrobnicatorService {
    
    public void frobnicate() {
        ...
    }
}

Every public method of this class will be part of the implicit local business interface.

In OSGi, a service does not need to have a separate interface, you can register any class as a service. Even when using Declarative Services, don't get fooled by the XML syntax of the Service Component Descriptor:

<service>
  <provide interface="com.example.FrobnicatorService"/>
</service>


The provide element has an interface attribute, but the attribute value can be any old class.

The same is true in Spring: There is no rule forcing you to inject Spring beans only via their interface. Any old class will do, even when working with automatic transaction proxies (just make sure that CGLIB is on your classpath).

So remember the KISS principle and kill some of the interfaces you don't really need!

01 December 2010

Glassfish and Weld: New builds, new bugs

My article about memory issues with CDI/Weld seems to have caused rather a wave. Someone (not me) posted a link on DZone, which triggered an unprecedented amount of traffic on this blog, there were discussions both in the Glassfish and Weld communities, and I was very happy to see that Weld appears to have made significant progress in terms of resource usage.

So the good news is: First, things are moving in the right direction with Weld, and second, blogging helps - at least in this case, it seems to have been a lot more effective than my previous bug reports and forum messages.

Unfortunately, there is also some bad news. Several users have already reported regressions with Glassfish 3.1-b29 and b30 related to Weld. My application fails to deploy on b30 with a stack trace which looks like a case of GLASSFISH-13131. So I'd better wait and try again with Glassfish 3.1 milestone 8.

29 November 2010

Java EE and Spring: Why I couldn't care less

 Java EE or Spring - for an independent software architect, the question is just as relevant as Catholic or Protestant to an atheist. The question has recently been discussed in a number of articles mostly taking one side or the other:

In this post, I'll try to point out why I'm not fully happy with either of the two.

If there's one thing that matters to me, it's modularity. To put my cards on the table, I'm a strong OSGi supporter, but I'm not even religious about that - in fact, Living without OSGi (which is what I've been doing for nine months now) would be a nice headline for one of my next posts.

Seen from outside, Spring and Java EE share a number of shortcomings:

Preaching to the Converted

When did you last visit the official homepages of Java EE or Spring? Take a moment to click on the links and imagine you've never heard about either. After visiting the homepages, you are none the wiser, all you get is the usual marketing blurb and a couple of newsbites completely meaningless to the uninitiated.

Lack of focus

Java EE and Spring both are animals of the rare species Sus ovipara lactifera velluta unknown in English speaking countries which we call eierlegende Wollmilchsau in German. In plain English, they both try to be everything to everyone and end up doing nothing in particular.

In contrast, OSGi has a clear mission statement The Dynamic Module System for Java and provides a link to its key benefits right on its homepage.

Bloat

Ever heard about lean development? Let's have a look at the official Javadocs:
  • Java EE 6: 1597 classes
  • Spring 3.0.5: 2312 classes

Legacy

Backward compatibility is a good thing for veteran users of a framework. Users don't want to change all of their application code just for upgrading to a new framework version.

On the other hand, backward compatibility can be extremely confusing to new users: There's two or three solutions for the same kind of problem, the legacy one is not marked as legacy or deprecated clearly enough, tutorials and example code still use the old style, and you can only resort to your favourite search engine or to trial and error to make things work consistently.

Love Thy User

Don't just list what you can do for your users, ask them what they want you to do for them. Looking at the Java EE and Spring reference documentation, I can see that both frameworks solve lots of problems I've never had in my life.

On the other hand, looking for a solution for some specific standard kind of problem, it is often surprisingly hard to find a comprehensive code example.

User documentation should be structured by user stories, not by technology.

Enough Rope

Both Spring and Java EE give you enough rope to hang yourself. It's easy to write messy and ugly code on top of both frameworks, and I've seen enough of that.

You can write loosely coupled and elegant applications on top of either, but it takes a good architect to set up guidelines and restrict the number of choices.

Development Process Integration

As an application developer, you don't just write code to run on a given framework. You want to run integration tests on your applications, you want to deploy and debug them directly in your favourite IDE, and you want your batch build and continuous integration to be able to deploy your applications.

When choosing a framework, always consider the entire process chain - some lack of tooling for your favourite framework may be reason enough to pick the second best instead.

Modularity

My ideal framework/container/app server - pick any name you like - would allow me to write truly modular applications, enforcing modularity at design time, at compile time, and at run time. The framework itself would be modular, allowing me to pick out those and only those modules actually required by my application.

It would actively support OSGi as the most mature module system for Java, and maybe a few others on top.

I expect some further degree of convergence between Java EE, Spring and OSGi within the next two years or so. There has been a lot slideware and a number of proof-of-concept or early adopter projects, but my feeling is that none of this has reached production quality yet.

To sum up, frameworks in general are alive and kicking, but the age of monolithic all-in-one frameworks is definitely over.

28 November 2010

Java EE 6 Integration Testing with Spring

Half a year ago, I created and published the jeeunit project to fill in some gaps for running integration tests on Java EE 6 components in an embedded EJB container, mainly targeted at Embedded Glassfish 3.x. One of the alternatives I considered and dismissed before writing my own solution was the Spring Test Context.

At the time, I was left with the impression you had to pollute your Java EE components with Spring annotations like @Transactional to make declarative transactions work in the Spring container, and I did not (and still do not) like the idea of writing XML just to create a couple of Java beans and wire them up.

After taking a closer look at Spring 3.0.x these days, I realized a number of facts which are not quite obvious from the Spring reference manual:
  • You can do most of your Spring configuration (but sadly not all of it!) in Java instead of XML, which makes your application context type-safe and easier to read, and gives you refactoring support from your IDE.
  • Spring supports some (but not all) of the Java EE 6 annotations as alternatives to its own flavours.
  • You can run Stateless Session Beans and JPA entities in a Spring container with declarative transactions without introducing dependencies on Spring APIs or annotations.
jeeunit now includes an example project jeeunit-example-test-spring demonstrating this approach. The details are documented in the jeeunit Wiki page Testing on Spring.

This solution is not intended to replace the original in-container-testing approach of jeeunit, because transactional Spring beans and EJBs have different run-time semantics. However, for many integration tests, these differences are irrelevant.

Using the Spring Test Context instead of Embedded Glassfish and jeeunit has two main advantages:
  • Spring starts noticeably faster than Embedded Glassfish.
  • Launching a single test from your suite is much easier than with jeeunit. E.g. in Eclipse, you can simply use Run as JUnit Test just as with plain old unit tests.

04 November 2010

CDI - A Major Risk Factor in Java EE 6

Update 18 Dec 2010: This post mainly relates to Glassfish 3.0.1. As of 3.1-b33, the situation has improved a lot.

Evaluating platforms and architectures for a new enterprise web application earlier this year, around March/April, scheduled to be released in 2011, I was bold enough to settle on Java EE 6.

In retrospective, I do not regret this decision, but I had to spend a significant amount of time analyzing and debugging platform problems. Based on this experience with Java EE 6, which may not be representative of course, the only advice I can give is: When Java EE 7 is released, wait a year or two before using it for production, unless you have the time and resources to act as beta tester.

The Java EE 6 specifications were released in December 2009, together with Glassfish v3, the first "production-quality release" of a Java EE 6 server. "Marketing release" would have been more appropriate, since going by the number of bugs in Glassfish 3.0, the main driving factor for the release date was quite obviously not a quality benchmark, but simply the need for Sun as the Java EE champion to release a working server simultaneously with the specifications.

The Problem


Java EE 6 was advertised as the lightest Java Enterprise Edition ever, with Contexts and Dependency Injection (CDI, JSR-299) as one of the most prominent innovations. However, for my project, CDI in the guise of Weld, its reference implementation, turned out to be the largest single source of problems. In addition to a number of functional bugs where injection would not work in certain scenarios, there are at least two severe memory issues:
  1. Weld causes memory leaks on application redeployment.
  2. Even in the first deployment of an application, Weld uses enormous amounts of memory.
The first issue had plagued us for months, and my team had grown accustomed to kill all Glassfish and Eclipse processes after three or four redeployments and to re-initialize their entire workspace, which was rather a pain.
Investigating the memory leaks using the Eclipse Memory Analyzer, I realized that Weld claimed almost 90 % of the heap memory of my application. Most of this was due to instances of org.jboss.weld.introspector.weld.jlr.WeldClassImpl, which appears to be a reflective model of a class inspected by the CDI BeanManager. Some instances of this class were as large as 1 MB.

I should add that our application uses Wicket for the web front end, injecting EJBs into custom Wicket components. Custom components inherit from Wicket base classes with a fairly deep inheritance hierarchy and a large number of methods. This accounts for the size of the reflective models.

Our usage of CDI was fairly basic, so it was just an hour's work to replace all @Inject annotations by Java EE 5 style @EJB annotations. After that, the heap usage of our application was reduced by 90 %, we no longer had any memory leaks, and we finally had a hassle-free edit-save-debug cycle in our development environment.

The figure of 90 % heap usage by Weld was observed on Glassfish 3.0.1, running on a 32-bit JVM on Windows XP.

This week, I retested the same application with Glassfish 3.1-b26 (using a newer version of Weld), running on a 64-bit JVM on Ubuntu 10.04. The result was not as drastic as before, but still about 50 % of my heap memory was consumed by Weld.

Alternatives


This article is mainly about problems with the CDI reference implementation Weld, not about CDI as such. But these implementation details have a ripple effect:
  • Glassfish 3.x still is the only certified Open Source Java EE 6 server. It uses a newer version of Weld in the current 3.1 promoted builds, but I can see no indication of the Weld issues being solved in time for the 3.1 release.
  • JBoss AS 6 is likely to be the next in line for full Java EE 6 compliance. However, JBoss also includes Weld, so you're bound to run into the same kind of problems on JBoss.
  • There are alternative CDI implementations, CanDI from Caucho and OpenWebBeans from Apache. Unfortunately, unlike JPA, there is no pluggable CDI provider interface that would allow you to simply replace Weld by some other CDI implementation on Glassfish or any other server.
  • I tried running my application on Resin 4.0.8 to see if CanDI was any better than Weld. However, Resin had bugs in the JPA and EJB areas and I was unable to deploy my application, let alone run and monitor it. (Resin only aims at supporting the Java EE 6 Web Profile, but that would have been sufficient for my case.)

Conclusion


All in all, if you want to work with Java EE 6, there is currently no way around Weld. And until Weld reaches production quality, the only way to build production quality applications on Java EE 6 is to avoid using CDI.

Free and Open Source Software is all about choice - so I would really love to see a pluggable CDI provider interface that would allow users to work with Glassfish and some CDI implementation other than Weld.

The second best solution might be to join forces from different CDI project to make Weld stable enough for production usage.

The third option would be another fully Java EE 6 compliant Open Source server (maybe Geronimo?) not including Weld.

No idea if any of these options is realistic, but this is on my wishlist for 2011.

19 September 2010

OpenJPA and OSGi

In my previous review of OSGi support in JPA 2.0 persistence providers, I promised to explain my setup for using OpenJPA in an OSGi environment.

The solution I'm going to outline is by no means final, and in the meantime, my first shot has been greatly simplified using some valuable input from the Apache Aries User mailing list.

In the past, brute force solutions like DynamicImport-Package were more or less the only way to let a JPA provider bundle access all the required classes for a persistence unit from application bundles. Since Release 4.2, the OSGi Enterprise Specification standardizes a whole lot of OSGi service wrappers for well-known Java EE APIs, including JPA, JDBC, JTA and JNDI, so this should be the preferred way of integrating  JPA providers into an OSGi environment.

The Apache Aries project provides (partial) implementations for some of the OSGi enterprise services and a whole lot of other things beyond the scope of this article. Fortunately, all their service implementations are nicely modular, so it is quite easy to use just the JPA and JNDI Services from Aries to make OpenJPA work on OSGi, following the OSGi Enterprise spec rather closely (though not fully).

Here is an overview of the required steps:
  • Get OSGi bundles for OpenJPA and all its dependencies, plus the required Aries bundles.
  • Publish the datasource to be used by your persistence unit as an OSGi service.
  • Make sure that your persistence.xml satisfies a number of restrictions.
  • Run the OpenJPA enhancer and package all classes and metadata of your persistence unit into a bundle with a special manifest header.
  • Start your OSGi framework. The JPA Service will publish an EntityManagerFactory service for each persistence unit.
  • In your client bundles, import your persistence bundle, OpenJPA and javax.persistence, get the appropriate EntityManagerFactory from the service registry and use it just like in an unmanaged Java SE environment.
Each step is explained in more detail in the following sections. I'm using Java 1.6.0, Maven 2.2.1 for collecting the dependencies, Eclipse 3.5.2 for doing most of the work, and the corresponding Equinox version as OSGi runtime.

Get the dependencies


The main JAR in the OpenJPA binary distribution is an OSGi bundle, but most of the included dependencies are plain old JARs, so you need to go shopping for osgified versions of all required libraries. All the required bundles for this example can be found in the SpringSource Enterprise Bundle Repository.

The following is the best part of a Maven POM for copying all the dependencies to a local directory target/platform/plugins.

You need to add the OSGi framework, javax.persistence (I copied both of them from Eclipse 3.5.2) and a JDBC driver bundle for your database. Then copy the whole lot to a new directory to be used as target platform for your Eclipse workspace.


<project>
  <properties>
    <aries.version>0.2-incubating</aries.version>
    <output.dir>${project.build.directory}/platform/plugins</output.dir>
  </properties>
  <dependencies>
    <dependency>  
      <groupId>javax.transaction</groupId>  
      <artifactId>com.springsource.javax.transaction</artifactId>  
      <version>1.1.0</version> 
    </dependency>    
    <dependency>
      <groupId>net.sourceforge.serp</groupId>
      <artifactId>com.springsource.serp</artifactId>
      <version>1.13.1</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries</groupId>
      <artifactId>org.apache.aries.util</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jndi</groupId>
      <artifactId>org.apache.aries.jndi.api</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jndi</groupId>
      <artifactId>org.apache.aries.jndi.core</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jndi</groupId>
      <artifactId>org.apache.aries.jndi.url</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jpa</groupId>
      <artifactId>org.apache.aries.jpa.api</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jpa</groupId>
      <artifactId>org.apache.aries.jpa.container</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>com.springsource.org.apache.commons.collections</artifactId>
      <version>3.2.1</version>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>com.springsource.org.apache.commons.lang</artifactId>
      <version>2.2.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>com.springsource.org.apache.commons.pool</artifactId>
      <version>1.5.3</version>
    </dependency>
   <dependency>
      <groupId>org.apache.openjpa</groupId>
      <artifactId>openjpa-all</artifactId>
      <version>2.0.1</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>com.springsource.slf4j.api</artifactId>
      <version>1.5.0</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>com.springsource.slf4j.log4j</artifactId>
      <version>1.5.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.log4j</groupId>
      <artifactId>com.springsource.org.apache.log4j</artifactId>
      <version>1.2.15</version>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-dependency-plugin</artifactId>
        <executions>
          <execution>
            <id>copy-bundles</id>
            <phase>package</phase>
            <goals>
              <goal>copy-dependencies</goal>
            </goals>
            <configuration>
              <outputDirectory>${output.dir}</outputDirectory>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
  <repositories>
    <repository>
      <id>com.springsource.repository.bundles.external</id>
      <name>SpringSource Enterprise Bundle Repository - External Bundle Releases</name>
      <url>http://repository.springsource.com/maven/bundles/external</url>
    </repository>
  </repositories>

</project>



Register a data source service


A persistence unit requires a data source, either from a javax.sql.DataSource or from a JDBC driver URL. This may be a matter of taste - in this article, I'm focusing on the DataSource.

Create a DataSource for your database using any suitable driver and register it as an OSGi service with the javax.sql.DataSource interface and optionally some additional identifying properties of your choice, e.g. name=myds, in case you have more than one DataSource in your system.

It is useful to do this from a separate bundle, so that this bundle will be the only one in your system with a direct dependency on the JDBC driver.


Adapt your persistence.xml


If you have a persistence.xml which works on Java SE oder Java EE, you will probably have to modify it  to make it work with OSGi and Aries. Here is an example:

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
  <persistence-unit name="myapp" transaction-type="RESOURCE_LOCAL">
  <non-jta-data-source>osgi:service/javax.sql.DataSource</non-jta-data-source>   
    <mapping-file>META-INF/orm.xml</mapping-file>
    <exclude-unlisted-classes>true</exclude-unlisted-classes>
  </persistence-unit>
</persistence>


First, you need to specify the RESOURCE_LOCAL transaction type explicitly, or else OpenJPA will complain about a missing transaction manager. This is because Aries creates a container EntityManagerFactory and not a stand-alone one.

Second, your data source has to be a non-jta-data-source. Note the special syntax for accessing your data source as an OSGi service via JNDI. To make this work, you need to include the Aries JNDI Service bundles. To select a specific data source by properties, you can append an LDAP filter to the JNDI URL, e.g. osgi:service/javax.sql.DataSource/(name=myds).

Third, you need to prevent the persistence provider from scanning your persistence unit for entity classes, since this may lead to classloader problems. To do so, either list all your classes explicitly if you use JPA annotations, or include all mapping files if you use XML metadata and set exclude-unlisted-classes to true to prevent the provider from scanning your bundle.

If you get funny exceptions like

org.apache.openjpa.persistence.PersistenceException: Error extracting class information from "bundleresource://23.fwk1421571929/".
Caused by: java.io.FileNotFoundException: /myapp/plugins/test.openjpa/bin (Is a directory)

then check if you have set this option. (This problem may be specific to Eclipse, but I'm not sure about that.)

Package your persistence unit bundle


To turn your persistence unit into a persistence bundle, make sure you have all mapping data resources and all entity classes included. In addition to the usual OSGi headers, insert the following one into your manifest:

Meta-Persistence: META-INF/persistence.xml

This is the hook for Aries to discover your persistence bundle using the extender pattern. If all goes well, an EntityManagerFactory will be registered as an OSGi service.

In theory, your persistence bundle should only depend on javax.persistence and not on org.apache.openjpa. In practice, you currently need to include this unwanted import, or else you will get exceptions of the following kind:

java.lang.NoClassDefFoundError: org/apache/openjpa/enhance/PersistenceCapable

Apparently, OpenJPA tries to load one of its own classes via your persistence bundle class loader, which is incorrect.

Use the EntityManagerFactory service in your clients


Finally, to work with your persistence unit from client bundles, just get hold of the appropriate EntityManagerFactory service any way you like, using the filter (osgi.unit.name=myapp).


Update 26 Sep 2010: I have submitted some sample code for inclusion into the OpenJPA test suite. The sample is attached to OPENJPA-1815.

15 September 2010

OSGi Support in JPA 2.0 Persistence Providers

About three years ago, I started working with Hibernate in an OSGi environment, and at that time, I had to roll my own solution for resolving all sorts of classloader issues.

Since then, with OSGi 4.2 and JPA 2.0, there have been major specification releases, followed by implementations; OSGi is growing ever more popular, and the current release of the JPA standard has integrated a lot of functionality which used to be available in proprietary API extensions only.

The OSGi 4.2 Enterprise Specification includes JPA, JTA and JDBC services, so in theory, using JPA in an OSGi environment should stop being an issue, as soon as this specification is supported by all OSGi and JPA implementations.

In practice, the specifications are not yet fully covered by the available implementations, and the degree of OSGi support varies a lot.

In the past few months, I have worked with the current versions of the three certified JPA 2.0 persistence providers, and in my opinion, the ranking for OSGi support is quite clear:
  1. Eclipselink
  2. OpenJPA
  3. Hibernate
Some more details on each provider:

Eclipselink


Eclipselink has a fully osgified distribution containing OSGi bundles of Eclipselink itself and all dependencies. To make your entity classes visible to Eclipselink, all you need to do is add a JPA-PersistenceUnits manifest header to each bundle containing one or more persistence units. This is enough for Eclipselink to load all your XML metadata and entity classes.

No ugly hacks, no buddy policies or fragments, it just works out of the box.

So for a quick start with JPA under OSGi, Eclipslink is definitely the best choice, even though the OSGi 4.2 JPA Service Specification is not yet fully supported (for instance, the Meta-Persistence header does not work), but this will be of little or no concern to most users.

Leaving OSGi aside, I have been bitten by a number of bugs in Eclipselink, discussed in earlier articles. If you do not use any of these features, you may be very happy with Eclipselink. For me, this set of bugs is currently a no-go, so I have migrated my OSGi applications from Eclipselink to OpenJPA.

OpenJPA


The silver medal goes to OpenJPA: OSGi does not appear to be on the top of the agenda of the OpenJPA core developers, but since OpenJPA is used under OSGi by Aries, another Apache project, there is at least a certain level of OSGi support.

In the binary distribution, the OpenJPA aggregate JAR is an OSGi bundle, but most of the dependencies are plain old JARs, so you need to find osgified versions of the dependencies on your own. The same goes for the Maven artifacts.

Once you have downloaded the appropriate OSGi bundles, you still have to do some extra work to ensure that OpenJPA finds your persistence units and your entity classes, and there are a couple of minor issues with the enhancer and with user-defined value handlers.

I will explain the details of my setup and some of the problems that should be addressed in OpenJPA in my next post.

Hibernate


Sadly, regarding OSGi support, Hibernate 3.5.x is not much different from the 3.2.x release I discussed in my earlier articles. The distribution contains plain old JARs, no OSGi bundles. The latest osgified version available from the SpringSource Enterprise Bundle Repository is 3.4.0, and the SpringSource versions never used to work for me, so I expect that the steps described in my 2008 article are still required to make Hibernate work on OSGi.

Disclaimer: I did not try to make Hibernate 3.5 work on OSGi, due to its lack of support of a number of JPA 2.0 features used heavily by my applications. For this reason, I have stopped using Hibernate, both on OSGi and on Java EE.

12 September 2010

JPA 2.0: Ordered Collections

In earlier posts, I wrote about problems with persistent maps, a new feature introduced in JPA 2.0. Ordered collections with explicit order columns are another JPA 2.0 feature which, again, is not yet fully robust in all JPA 2.0 compliant and TCK tested implementations.

I'm currently working with a simple JPA model for OpenStreetMap (OSM). There are ways and nodes, and each way has a sequence of nodes. Of course the order of the nodes is important, and a node may be used more than once for a given way.

Thus, the nodes collection has to be a List, not a Set, and we need an explicit order column specifying the sequence number of the nodes along the way. Sorting the nodes by ID would not make any sense, obviously.

Here is a snippet from the model:

@Entity
@Table(name = "ways")
public class OsmWay
{
    @Id
    private long id;
    
    @ManyToMany(cascade = CascadeType.ALL)
    @OrderColumn(name = "sequence_id")
    @JoinTable(name = "way_nodes", 
               joinColumns = @JoinColumn(name = "id"), 
               inverseJoinColumns = @JoinColumn(name = "node_id"))
    private List<OsmNode> nodes = new ArrayList<OsmNode>();
}    

This entity is represented by two tables:

CREATE TABLE ways
(
  id bigint NOT NULL
);

CREATE TABLE way_nodes
(
  id bigint NOT NULL,
  node_id bigint NOT NULL,
  sequence_id integer,
);

I've tested this scenario on Hibernate 3.5.3, Eclipselink 2.1.1 and OpenJPA 2.0.1: Hibernate and OpenJPA pass, Eclipselink fails with 2 issues.

First, the generated DDL is incorrect: There is a PRIMARY KEY (id, node_id) for way_nodes. This should be (id, sequence_id).

Second, the order of the list items is not maintained in all contexts. It is ok when the collection is lazily loaded, e.g.

OsmWay way = em.find(OsmWay.class, wayId);
List<OsmNode> nodes = way.getNodes();

but it is broken when using a fetch join:

String jpql = "select distinct w from OsmWay w join fetch w.nodes";
TypedQuery<OsmWay> query = em.createQuery(jpql, OsmWay.class);
List<OsmWay> ways = query.getResultList();
OsmWay way = ways.get(0);
List<OsmNode> nodes = way.getNodes();

The funny thing is, the nodes are neither ordered by sequence_id nor by node_id. In my test case, the way has 5 nodes, one of which appears twice in different positions.

So here is some more evidence for my claim that the JPA 2.0 TCK is insufficient.