New OSGi Console in Equinox 3.8.0

Equinox 3.8.0, the OSGi framework running Eclipse Juno, comes with a new console built on top of Apache Felix Gogo Shell.

It is debatable whether or not this is an improvement. Veteran Equinox users are greeted with the following message when trying to activate the console:

Read more »

OSGi and CDI Combined

Pax CDI is a new OPS4J community project I've started to combine the best of OSGi and CDI.

Once you've worked with CDI or with an up-to-date annotation-based flavour of Spring, the service registry API or XML-based dependency injection in the Blueprint or Declarative Services style suddenly appears rather verbose.

On the other hand, even though Java EE 6 is more lightweight than ever, its monolithic nature feels rather a burden when you're used to OSGi.

Why not have the best of both worlds? The idea is not new, it has been implemented to some extent both in weld-osgi and in the FighterFish subproject of GlassFish.

Unlike these two projects, Pax CDI is independent of any given CDI, Java EE or OSGi implementation. The first proof-of-concept release is based on Apache OpenWebBeans and Equinox. In contrast to weld-osgi, which required some modifications in Weld itself, Pax CDI works with an unmodified version of OpenWebBeans.

Trying to do the same with Weld is one of the next goals for Pax CDI.

An alpha release Pax CDI 0.1.0 is available from Maven Central.

Pax JDBC: An OSGi JDBC Service

Pax JDBC is a new project under the OPS4J umbrella which simplifies the use of JDBC drivers in OSGi applications by providing a generic driver extender and various native driver adapters for your favourite database.

This project aims at compliance with the OSGi Enterprise Release 5.0 specification.

Pax JDBC 0.1.0 is available from Maven Central, with native adapters for Apache Derby and PostgreSQL. Since this first release, support for H2 and MySQL has been added, and further contributions are always welcome.

OSGi, Java EE and CDI Integration Testing with Pax Exam 3

In-container testing is an approach shared by many testing frameworks like Pax Exam, jeeunit, Arquillian and Spring's test context.

Pax Exam 2.x is a mature solution for testing OSGi bundles and applications, jeeunit was created to simplify Java EE 6 integration testing and has been extended to handle stand-alone CDI applications and non-CDI setups based on Tomcat and Spring.

Taking the best of both worlds, the next major release line Pax Exam 3.x is designed to provide a unified testing experience by incorporating additional test containers for Java EE and CDI from jeeunit while staying backward compatible with Pax Exam 2.x for OSGi.

The first public milestone release Pax Exam 3.0.0.M1 is now available from The Central Repository (aka Maven Central), including test containers for GlassFish, OpenWebBeans and Weld.

While jeeunit works with Embedded GlassFish which lacks all OSGi capabilities of the GlassFish server, the Pax Exam GlassFish Container launches GlassFish 3.1.2 on top of an OSGi framework, enabling users to deploy or provision both WAR modules and OSGi bundles, thus providing a test environment for hybrid Java EE/OSGi applications.

A plain old JUnit test class can be turned into an in-container integration test simply by adding a @RunWith(PaxExam.class) annoation and some configuration data in a properties file.

See the Pax Exam 3.0.0.M1 Release Notes for more details and use the OPS4J mailing list (general@lists.ops4j.org) for feedback and support.

Integration Testing for JBoss AS7, Tomcat and Weld SE

Initially, the jeeunit Integration Testing framework was exclusively focused on Java EE 6, supporting GlassFish 3.x, which was the only Java EE 6 compliant server at that time. Support for Resin 4.x has been added in subsequent releases.

With the current release 0.9.1, jeeunit supports JBoss AS 7, as well as alternative containers and injection methods beyond the scope of Java EE 6. You can now run jeeunit tests on Tomcat 6 and 7 and Weld SE containers.

Tomcat can be combined either with Spring 3.1 or with CDI (Weld Servlet) to inject dependencies into jeeunit tests.

And there's more to come: While jeeunit will continue a life of its own for a while, I'm planning to merge it step by step into Pax Exam, the OSGi testing framework of the OPS4J community.

Pax Exam and jeeunit both implement an in-container testing approach - while Pax Exam focuses on OSGi alone, jeeunit now supports various other containers, but no OSGi at all.

The Pax Exam/jeeunit merger opens interesting perspectives for hybrid applications, i.e. enterprise applications composed of traditional WARs and OSGi bundles. GlassFish 3.x supports this hybrid application model, implementing a subset of the OSGi Enterprise specifications.

For Pax Exam, the road towards the next major release 3.0.0 will be marked by a sequence of milestone releases, each of which is to incoporate a new container adapted from jeeunit.

A Pax Exam GlassFish Test Container is the goal of the first proof-of-concept milestone 3.0.0.M1. This is work in progress on a dedicated branch exam3-milestones in the Pax Exam GitHub repository.

Stay tuned...

Wicket and OSGi

After playing around with the OSGi JPA and JTA components of Apache Aries for a while, it was only natural to try and add a web layer to my sample projects. The Aries samples all use plain old servlets and JSPs, so I wondered if Wicket would work in this context.

So far, I had only ever used Wicket in standard Java EE web applications, but I had noticed the Wicket JARs come with an OSGi manifest, so someone must have tried to use Wicket in OSGi before.

Read more »

reFit: Acceptance Testing for Java EE 6, Spring and OSGi

Even if you don't practice test-driven development, you are certainly familiar with the JUnit family of testing frameworks (including ports to other languages like cppunit, NUnit, or independent but similar approaches like TestNG).

This post is about the Fit framework family, which has a somewhat different focus and is not just another JUnit clone. In a nutshell, Fit represents expected and actual test results in tables, and you do not have to be a programmer to read or write them.

The following table has one column of inputs and two columns of outputs:

Each row of this table can be regarded as an acceptance test case. You can feed this table to a given system and compare the outputs:

It's only a couple of weeks ago I was introduced to Fit in my current project, and I must admit my first thought was "Why don't you just write a @Parameterized JUnit test?"

The answer is, JUnit is for developers, and developers only; Fit is for customers and developers.

You still need a developer to implement the skeleton logic of a test (called fixture in Fit), but anyone can write new test cases by adding rows or columns to a given Fit table.

The Fit method and the original Java implementation were created by Ward Cunningham in or around 2002, the project is hosted on Sourceforge and has been inactive since 2008.

The Fitnesse project integrates Fit with a Wiki, it includes a modified version of the original Fit implementation and is under active development. However, Fit and Fitnesse are incompatible, Fit lets you write your test tables in plain old HTML, whereas Fitnesse supports its own Wiki syntax only.

Our project has a fair amount of plain old HTML Fit tests, and our production code uses Spring, so it was a fairly natural idea to inject Spring beans from the system under test into our Fit test code.

Spring is just one (and not really my favourite) framework for dependency injection (and lots of others things). so I inevitably started thinking about how to use Fit together with Java EE or OSGi.

Thinking was followed by coding (yeah, it can be the other way round sometimes...), and this led to a project called reFit hosted on Google Code.

reFit is based on the latest Fit sources from SourceForge, it provides up-to-date Maven artifacts on Maven Central and integrations with Java EE 6, Spring and OSGi Declarative Services. The Java EE integration reuses parts of my jeeunit project.

In addition, reFit lets you run Fit tests under a JUnit wrapper or from a Maven plugin, both with or without Spring integration. reFit includes ready-to-run example code for Glassfish 3.1, Spring 3, Equinox and Weld SE.

There's more to come, I'm currently experimenting with a Web frontend and a WYSIWYG HTML editor.

For more details, check out the reFit Wiki.

The Interface Antipattern

Keeping interfaces and implementations separate is a useful design pattern for building modular systems. If your clients only depend on a service interface, you can switch service implementations without your clients even noticing.

Now there can be too much of a good thing, and the quality of a software system is certainly not measured by the ratio of interfaces to classes.

When you are planning to implement a FrobnicatorService, think twice before creating an IFrobnicatorService interface and a FrobnicatorServiceImpl. How many different Frobnicator implementations are there going to be? If you do need at least two implementations with distinct behaviour, then go ahead and create the interface. If there's only one implementation, then don't bother with the interface.

Resist the temptation of speculative generalization: "Oh, there might be a performance bottleneck, and in that case an alternative CacheingFrobnicatorServiceImpl might help, so I really should start with a service interface now." Think twice: you ain't gonna need it.

And if you do need the interface, you can still pull it out when you need it. Most IDEs have an automatic refactoring Extract Interface. (Unfortunately, none of the major Java IDEs seems to have the opposite refactoring Merge Interface and Implementation.)

Even in Java EE, the times of interface inflation are gone with EJB 3.1, thanks to the no-interface local view. For a stateless session bean, it is enough to implement

@Stateless
public class FrobnicatorService {
    
    public void frobnicate() {
        ...
    }
}

Every public method of this class will be part of the implicit local business interface.

In OSGi, a service does not need to have a separate interface, you can register any class as a service. Even when using Declarative Services, don't get fooled by the XML syntax of the Service Component Descriptor:

<service>
  <provide interface="com.example.FrobnicatorService"/>
</service>

The provide element has an interface attribute, but the attribute value can be any old class.

The same is true in Spring: There is no rule forcing you to inject Spring beans only via their interface. Any old class will do, even when working with automatic transaction proxies (just make sure that CGLIB is on your classpath).

So remember the KISS principle and kill some of the interfaces you don't really need!

Java EE and Spring: Why I couldn't care less

 Java EE or Spring - for an independent software architect, the question is just as relevant as Catholic or Protestant to an atheist. The question has recently been discussed in a number of articles mostly taking one side or the other:

• Moving from Spring to Java EE 6: The Age of Frameworks is Over
• Spring vs. Java EE and Why I Don't Care
• Java EE, Spring, and why I care
• Java EE 6 und das Ende von Spring

In this post, I'll try to point out why I'm not fully happy with either of the two.

If there's one thing that matters to me, it's modularity. To put my cards on the table, I'm a strong OSGi supporter, but I'm not even religious about that - in fact, Living without OSGi (which is what I've been doing for nine months now) would be a nice headline for one of my next posts.

Seen from outside, Spring and Java EE share a number of shortcomings:

Preaching to the Converted

When did you last visit the official homepages of Java EE or Spring? Take a moment to click on the links and imagine you've never heard about either. After visiting the homepages, you are none the wiser, all you get is the usual marketing blurb and a couple of newsbites completely meaningless to the uninitiated.

Lack of focus

Java EE and Spring both are animals of the rare species Sus ovipara lactifera velluta unknown in English speaking countries which we call eierlegende Wollmilchsau in German. In plain English, they both try to be everything to everyone and end up doing nothing in particular.

In contrast, OSGi has a clear mission statement The Dynamic Module System for Java and provides a link to its key benefits right on its homepage.

Bloat

Ever heard about lean development? Let's have a look at the official Javadocs:

• Java EE 6: 1597 classes
• Spring 3.0.5: 2312 classes

Legacy

Backward compatibility is a good thing for veteran users of a framework. Users don't want to change all of their application code just for upgrading to a new framework version.

On the other hand, backward compatibility can be extremely confusing to new users: There's two or three solutions for the same kind of problem, the legacy one is not marked as legacy or deprecated clearly enough, tutorials and example code still use the old style, and you can only resort to your favourite search engine or to trial and error to make things work consistently.

Love Thy User

Don't just list what you can do for your users, ask them what they want you to do for them. Looking at the Java EE and Spring reference documentation, I can see that both frameworks solve lots of problems I've never had in my life.

On the other hand, looking for a solution for some specific standard kind of problem, it is often surprisingly hard to find a comprehensive code example.

User documentation should be structured by user stories, not by technology.

Enough Rope

Both Spring and Java EE give you enough rope to hang yourself. It's easy to write messy and ugly code on top of both frameworks, and I've seen enough of that.

You can write loosely coupled and elegant applications on top of either, but it takes a good architect to set up guidelines and restrict the number of choices.

Development Process Integration

As an application developer, you don't just write code to run on a given framework. You want to run integration tests on your applications, you want to deploy and debug them directly in your favourite IDE, and you want your batch build and continuous integration to be able to deploy your applications.

When choosing a framework, always consider the entire process chain - some lack of tooling for your favourite framework may be reason enough to pick the second best instead.

Modularity

My ideal framework/container/app server - pick any name you like - would allow me to write truly modular applications, enforcing modularity at design time, at compile time, and at run time. The framework itself would be modular, allowing me to pick out those and only those modules actually required by my application.

It would actively support OSGi as the most mature module system for Java, and maybe a few others on top.

I expect some further degree of convergence between Java EE, Spring and OSGi within the next two years or so. There has been a lot slideware and a number of proof-of-concept or early adopter projects, but my feeling is that none of this has reached production quality yet.

To sum up, frameworks in general are alive and kicking, but the age of monolithic all-in-one frameworks is definitely over.

OpenJPA and OSGi

In my previous review of OSGi support in JPA 2.0 persistence providers, I promised to explain my setup for using OpenJPA in an OSGi environment.

The solution I'm going to outline is by no means final, and in the meantime, my first shot has been greatly simplified using some valuable input from the Apache Aries User mailing list.

In the past, brute force solutions like DynamicImport-Package were more or less the only way to let a JPA provider bundle access all the required classes for a persistence unit from application bundles. Since Release 4.2, the OSGi Enterprise Specification standardizes a whole lot of OSGi service wrappers for well-known Java EE APIs, including JPA, JDBC, JTA and JNDI, so this should be the preferred way of integrating  JPA providers into an OSGi environment.

The Apache Aries project provides (partial) implementations for some of the OSGi enterprise services and a whole lot of other things beyond the scope of this article. Fortunately, all their service implementations are nicely modular, so it is quite easy to use just the JPA and JNDI Services from Aries to make OpenJPA work on OSGi, following the OSGi Enterprise spec rather closely (though not fully).

Here is an overview of the required steps:

• Get OSGi bundles for OpenJPA and all its dependencies, plus the required Aries bundles.
• Publish the datasource to be used by your persistence unit as an OSGi service.
• Make sure that your persistence.xml satisfies a number of restrictions.
• Run the OpenJPA enhancer and package all classes and metadata of your persistence unit into a bundle with a special manifest header.
• Start your OSGi framework. The JPA Service will publish an EntityManagerFactory service for each persistence unit.
• In your client bundles, import your persistence bundle, OpenJPA and javax.persistence, get the appropriate EntityManagerFactory from the service registry and use it just like in an unmanaged Java SE environment.

Each step is explained in more detail in the following sections. I'm using Java 1.6.0, Maven 2.2.1 for collecting the dependencies, Eclipse 3.5.2 for doing most of the work, and the corresponding Equinox version as OSGi runtime.

Get the dependencies

The main JAR in the OpenJPA binary distribution is an OSGi bundle, but most of the included dependencies are plain old JARs, so you need to go shopping for osgified versions of all required libraries. All the required bundles for this example can be found in the SpringSource Enterprise Bundle Repository.

The following is the best part of a Maven POM for copying all the dependencies to a local directory target/platform/plugins.

You need to add the OSGi framework, javax.persistence (I copied both of them from Eclipse 3.5.2) and a JDBC driver bundle for your database. Then copy the whole lot to a new directory to be used as target platform for your Eclipse workspace.

<project>
  <properties>
    <aries.version>0.2-incubating</aries.version>
    <output.dir>${project.build.directory}/platform/plugins</output.dir>
  </properties>
  <dependencies>
    <dependency>  
      <groupId>javax.transaction</groupId>  
      <artifactId>com.springsource.javax.transaction</artifactId>  
      <version>1.1.0</version> 
    </dependency>    
    <dependency>
      <groupId>net.sourceforge.serp</groupId>
      <artifactId>com.springsource.serp</artifactId>
      <version>1.13.1</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries</groupId>
      <artifactId>org.apache.aries.util</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jndi</groupId>
      <artifactId>org.apache.aries.jndi.api</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jndi</groupId>
      <artifactId>org.apache.aries.jndi.core</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jndi</groupId>
      <artifactId>org.apache.aries.jndi.url</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jpa</groupId>
      <artifactId>org.apache.aries.jpa.api</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.aries.jpa</groupId>
      <artifactId>org.apache.aries.jpa.container</artifactId>
      <version>${aries.version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>com.springsource.org.apache.commons.collections</artifactId>
      <version>3.2.1</version>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>com.springsource.org.apache.commons.lang</artifactId>
      <version>2.2.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>com.springsource.org.apache.commons.pool</artifactId>
      <version>1.5.3</version>
    </dependency>
   <dependency>
      <groupId>org.apache.openjpa</groupId>
      <artifactId>openjpa-all</artifactId>
      <version>2.0.1</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>com.springsource.slf4j.api</artifactId>
      <version>1.5.0</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>com.springsource.slf4j.log4j</artifactId>
      <version>1.5.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.log4j</groupId>
      <artifactId>com.springsource.org.apache.log4j</artifactId>
      <version>1.2.15</version>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-dependency-plugin</artifactId>
        <executions>
          <execution>
            <id>copy-bundles</id>
            <phase>package</phase>
            <goals>
              <goal>copy-dependencies</goal>
            </goals>
            <configuration>
              <outputDirectory>${output.dir}</outputDirectory>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
  <repositories>
    <repository>
      <id>com.springsource.repository.bundles.external</id>
      <name>SpringSource Enterprise Bundle Repository - External Bundle Releases</name>
      <url>http://repository.springsource.com/maven/bundles/external</url>
    </repository>
  </repositories>

</project>

Register a data source service

A persistence unit requires a data source, either from a javax.sql.DataSource or from a JDBC driver URL. This may be a matter of taste - in this article, I'm focusing on the DataSource.

Create a DataSource for your database using any suitable driver and register it as an OSGi service with the javax.sql.DataSource interface and optionally some additional identifying properties of your choice, e.g. name=myds, in case you have more than one DataSource in your system.

It is useful to do this from a separate bundle, so that this bundle will be the only one in your system with a direct dependency on the JDBC driver.

Adapt your persistence.xml

If you have a persistence.xml which works on Java SE oder Java EE, you will probably have to modify it  to make it work with OSGi and Aries. Here is an example:

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
  <persistence-unit name="myapp" transaction-type="RESOURCE_LOCAL">
  <non-jta-data-source>osgi:service/javax.sql.DataSource</non-jta-data-source>   
    <mapping-file>META-INF/orm.xml</mapping-file>
    <exclude-unlisted-classes>true</exclude-unlisted-classes>
  </persistence-unit>
</persistence>

First, you need to specify the RESOURCE_LOCAL transaction type explicitly, or else OpenJPA will complain about a missing transaction manager. This is because Aries creates a container EntityManagerFactory and not a stand-alone one.

Second, your data source has to be a non-jta-data-source. Note the special syntax for accessing your data source as an OSGi service via JNDI. To make this work, you need to include the Aries JNDI Service bundles. To select a specific data source by properties, you can append an LDAP filter to the JNDI URL, e.g. osgi:service/javax.sql.DataSource/(name=myds).

Third, you need to prevent the persistence provider from scanning your persistence unit for entity classes, since this may lead to classloader problems. To do so, either list all your classes explicitly if you use JPA annotations, or include all mapping files if you use XML metadata and set exclude-unlisted-classes to true to prevent the provider from scanning your bundle.

If you get funny exceptions like

org.apache.openjpa.persistence.PersistenceException: Error extracting class information from "bundleresource://23.fwk1421571929/".
Caused by: java.io.FileNotFoundException: /myapp/plugins/test.openjpa/bin (Is a directory)

then check if you have set this option. (This problem may be specific to Eclipse, but I'm not sure about that.)

Package your persistence unit bundle

To turn your persistence unit into a persistence bundle, make sure you have all mapping data resources and all entity classes included. In addition to the usual OSGi headers, insert the following one into your manifest:

Meta-Persistence: META-INF/persistence.xml

This is the hook for Aries to discover your persistence bundle using the extender pattern. If all goes well, an EntityManagerFactory will be registered as an OSGi service.

In theory, your persistence bundle should only depend on javax.persistence and not on org.apache.openjpa. In practice, you currently need to include this unwanted import, or else you will get exceptions of the following kind:

java.lang.NoClassDefFoundError: org/apache/openjpa/enhance/PersistenceCapable

Apparently, OpenJPA tries to load one of its own classes via your persistence bundle class loader, which is incorrect.

Use the EntityManagerFactory service in your clients

Finally, to work with your persistence unit from client bundles, just get hold of the appropriate EntityManagerFactory service any way you like, using the filter (osgi.unit.name=myapp).


Update 26 Sep 2010: I have submitted some sample code for inclusion into the OpenJPA test suite. The sample is attached to OPENJPA-1815.

OSGi Support in JPA 2.0 Persistence Providers

About three years ago, I started working with Hibernate in an OSGi environment, and at that time, I had to roll my own solution for resolving all sorts of classloader issues.

Since then, with OSGi 4.2 and JPA 2.0, there have been major specification releases, followed by implementations; OSGi is growing ever more popular, and the current release of the JPA standard has integrated a lot of functionality which used to be available in proprietary API extensions only.

The OSGi 4.2 Enterprise Specification includes JPA, JTA and JDBC services, so in theory, using JPA in an OSGi environment should stop being an issue, as soon as this specification is supported by all OSGi and JPA implementations.

In practice, the specifications are not yet fully covered by the available implementations, and the degree of OSGi support varies a lot.

In the past few months, I have worked with the current versions of the three certified JPA 2.0 persistence providers, and in my opinion, the ranking for OSGi support is quite clear:

1. Eclipselink
2. OpenJPA
3. Hibernate

Some more details on each provider:

Eclipselink

Eclipselink has a fully osgified distribution containing OSGi bundles of Eclipselink itself and all dependencies. To make your entity classes visible to Eclipselink, all you need to do is add a JPA-PersistenceUnits manifest header to each bundle containing one or more persistence units. This is enough for Eclipselink to load all your XML metadata and entity classes.

No ugly hacks, no buddy policies or fragments, it just works out of the box.

So for a quick start with JPA under OSGi, Eclipslink is definitely the best choice, even though the OSGi 4.2 JPA Service Specification is not yet fully supported (for instance, the Meta-Persistence header does not work), but this will be of little or no concern to most users.

Leaving OSGi aside, I have been bitten by a number of bugs in Eclipselink, discussed in earlier articles. If you do not use any of these features, you may be very happy with Eclipselink. For me, this set of bugs is currently a no-go, so I have migrated my OSGi applications from Eclipselink to OpenJPA.

OpenJPA

The silver medal goes to OpenJPA: OSGi does not appear to be on the top of the agenda of the OpenJPA core developers, but since OpenJPA is used under OSGi by Aries, another Apache project, there is at least a certain level of OSGi support.

In the binary distribution, the OpenJPA aggregate JAR is an OSGi bundle, but most of the dependencies are plain old JARs, so you need to find osgified versions of the dependencies on your own. The same goes for the Maven artifacts.

Once you have downloaded the appropriate OSGi bundles, you still have to do some extra work to ensure that OpenJPA finds your persistence units and your entity classes, and there are a couple of minor issues with the enhancer and with user-defined value handlers.

I will explain the details of my setup and some of the problems that should be addressed in OpenJPA in my next post.

Hibernate

Sadly, regarding OSGi support, Hibernate 3.5.x is not much different from the 3.2.x release I discussed in my earlier articles. The distribution contains plain old JARs, no OSGi bundles. The latest osgified version available from the SpringSource Enterprise Bundle Repository is 3.4.0, and the SpringSource versions never used to work for me, so I expect that the steps described in my 2008 article are still required to make Hibernate work on OSGi.

Disclaimer: I did not try to make Hibernate 3.5 work on OSGi, due to its lack of support of a number of JPA 2.0 features used heavily by my applications. For this reason, I have stopped using Hibernate, both on OSGi and on Java EE.

Getting Started with OSGi Declarative Services

OSGi Declarative Services have been on my mental agenda for a while, and now I've started using them. I'm not going to write yet another tutorial on the subject, but there are some bits and pieces I could not find anywhere else that may be worth mentioning.

To get an overview of Declarative Services, have a look at Neil Bartlett's tutorial on EclipseZone, or at his more recent EclipseCon presentation. The former is based on Eclipse 3.2.2, the latter on the latest Eclipse 3.5 milestones, so there are some differences to the current release Eclipse 3.4.2. In particular, Eclipse 3.5 supports a newer version of the Declarative Services Specification (1.1 vs. 1.0 in Eclipse 3.4).

All of the following refers to Eclipse 3.4.2. For using Declarative Services with Equinox, you need the following bundles:

• org.eclipse.osgi
• org.eclipse.osgi.services
• org.eclipse.equinox.ds
• org.eclipse.equinox.util

The last two are not included in standard Eclipse distributions. Download them from the Equinox download area, or use the Eclipse update manager to install Equinox Bundles.

Follow the tutorials to implement a simple example. If you cannot figure out why your component does not start, enable console logging for the Service Component Runtime in your OSGi launcher by setting the following system properties.

• equinox.ds.debug=true
• equinox.ds.print=true

Make sure to include the OSGI-INF folder with your component descriptors into the binary build on the Build tab of the Eclipe manifest editor. This amounts to adding the folder to the bin.includes property in the build.properties.

Add a header Service-Component: OSGI-INF/components.xml to your manifest. (Choose any other filename if you like.)

Do not forget to specify the correct XML namespace http://www.osgi.org/xmlns/scr/v1.0.0 in your XML documents, or else your component definitions will not be recognized.

Seeing that Eclipse 3.4 has no graphical editors for the component descriptors, it is useful to install the Eclipse XML editor features and to work with XML validation on the component descriptors.

If your bundle contains more than one component, you can have one XML document per component and list them all in the manifest header, or you can have a single XML document containing all component descriptions, which is what I prefer.

To enable XML validation on a multi-component descriptor, write your own XML schema where the root element contains a sequence of 1..n component descriptors.

Here is an XML schema for this purpose:

<?xml version="1.0" encoding="UTF-8"?>
<schema xmlns="http://www.w3.org/2001/XMLSchema"
        xmlns:scr="http://www.osgi.org/xmlns/scr/v1.0.0" 
        targetNamespace="http://www.harmanbecker.com/anaconda/components" 
        xmlns:tns="http://www.example.org/components" 
        elementFormDefault="qualified">

   <import namespace="http://www.osgi.org/xmlns/scr/v1.0.0"/>

    <element name="components">
      <complexType>
        <sequence>        
          <element ref="scr:component" maxOccurs="unbounded"/>
        </sequence>
      </complexType>
    </element>
</schema>

Using this schema, a component descriptor for two components might look like this:

<?xml version="1.0" encoding="UTF-8"?>
<c:components xmlns:scr="http://www.osgi.org/xmlns/scr/v1.0.0"
      xmlns:c="http://www.harmanbecker.com/anaconda/components">


 <scr:component name="foo">              
   <implementation class="com.acme.foo.FooImpl"/>
   <service>
     <provide interface="com.acme.foo.Foo"/>
   </service>
 </scr:component>

 <scr:component name="bar">              
   <implementation class="com.acme.foo.BarImpl"/>
   <service>
     <provide interface="com.acme.bar.Bar"/>
   </service>
 </scr:component>

</c:components>

JDBC Drivers in OSGi

How do you create a JDBC connection for a given JDBC URL in an OSGi application? In particular, how do you avoid an explicit dependency of your application code on a given driver?

In a plain old classpath context, you would invoke DriverManager.getConnection(), maybe after loading the driver class using Class.forName(), if your driver does not support the Service Provider mechanism which is mandatory for JDBC 4.0.

The trouble is, even if your driver does provide the META-INF/services metadata, this does not work in OSGi, since DriverManager creates a number of class loader problems by using Class.forName() internally.

I'm working with a variant of the Zentus driver for SQLite, which I've modified slightly to build an OSGi bundle under Java 1.6. I added the service metadata, and indeed my application code now works without loading the driver via Class.forName() - as long as the code is running outside of OSGi.

There are the following issues:

1. The context class loader of the current thread is used to scan for META-INF/services resources and to load the referenced classes. This happens during initialization of DriverManager, so it will not catch resources from bundles which are not visible to the current context.
   


2. Additional drivers can register with DriverManager, they should do so in static initalization code, so that the driver gets registered when someone calls Class.forName("my.own.Driver").


3. DriverManager.getConnection() iterates over the registered drivers until it finds one that can process the given URL. Unfortunately, it does a fatal double check to see if the class of that driver is the same that would be obtained by the caller. This amounts to calling Class.forName("some.matching.Driver", ccl) where ccl is the caller's class loader.

(To get some information on what is happening, it is useful to call DriverManager.setLogStream(System.out).)

The third point means that DriverManager.getConnection() will always fail if the driver class is not visible to your bundle, so there really is no way to avoid a dependency.

Here is an outline of a partial solution:

• Create an OSGi bundle for each JDBC driver you want to use. Add a bundle activator that calls Class.forName("some.jdbc.Driver"). This will register the driver with DriverManager when the bundle is started.
• For each bundle that needs to create connections using DriverManager.getConnection(), add optional dependencies on the JDBC drivers packages for all drivers that you are planning to use.

This is rather ugly, because you need to edit your client bundle manifest whenever you want to add support for an additional driver. But at least you can deploy your system with any subset of the defined drivers, maybe with just a single driver.

If you do not need to support legacy code, do not bother using DriverManager and the Service Provider mechanism. After all, this is just a poor man's service registry, and you are much better off using the OSGi Service Registry.

I'll give an example in one of my next posts.

Anaconda: A New Architecture for Compiling Navigation Databases

In my previous post, I presented the Anaconda Workbench, without explaining about Anaconda. This is the code name for my current project, a Map Compiler for processing map data into a compact binary database for car navigation systems of Harman/Becker.

I gave a talk about Anaconda at the Eclipse Demo Camp in Hamburg in November last year. The slides of this talk explain the purpose of a Map Compiler and our usage of OSGi and Eclipse and lots of other fabulous Open Source components.

Hibernate and OSGi: An elaborate solution

A much cleaner and flexible way of osgifying a plain old JAR with lots of dependencies is adding the required OSGi headers to each JAR, so you will replace the megabundle with a bunch of small bundles, one per JAR, which you can reuse in other contexts.

In an ideal OSGi world, every JAR would be an OSGi bundle, so you would have nothing to worry about. Unfortunately, most Java libraries still come as plain old JARs and you have to add the headers on your own.

If you are lucky, someone has done the job for you already. There are a number of third-party repositories offering osgified versions of popular Java libraries, e.g. the OSGi bundle repository or the SpringSource Enterprise Bundle Repository.

SpringSource even has an OSGi bundle of Hibernate itself. However, their version does not work, at any rate not in my setup, so I had to build my own Hibernate bundle.

At least, I was able to use the SpringSource OSGi version for each dependency of Hibernate.

These are the problems I had with the SpringSource version of Hibernate (3.2.6.ga):

• The javax.transaction package does not resolve. It is imported with a specific version range. However, in Java 1.6.0, this package is contained in the JRE and comes from there without a version. I had to drop the version directive to make Hibernate use the javax.transaction package from the JRE.
• I got an exception on running a HQL query, since ANTLR could not load the Hibernate token class.
• I had a very mysterious ClassNotFoundException when accessing some of my model classes. As it turned out, the reason was my usage of lazy loading, where Hibernate injects CGLIB proxy classes into my model classes. The exception was due to the fact that my model bundle did not have access to the CGLIB classes. Rather than declaring a dependency on CGLIB for each of my model bundles, I added the following header to my Hibernate manifest:

Require-Bundle: com.springsource.net.sf.cglib;visibility:=reexport

This means that every bundle with a dependency on Hibernate automatically inherits the dependency on CGLIB. Even St. Peter the Evangelist who usually preaches about using Import-Package instead of Require-Bundle admits this to be one of the rare cases where the latter has its merits.

So far, with this approach, we have not used any Equinox buddy policies, but we still need to deal with the application model classes and resources.

For a while, I thought a fragment bundle would be the definitive solution, which would work not only on Equinox.

I created a bundle com.acme.myapp.hibernate.fragment, containing no Java classes, but only a manifest and some resources, i.e. the Hibernate configuration file and all my Hibernate mapping files. I added all model packages and all relevant JDBC driver packages to the Import-Package header (using resolution:= optional for the JDBC packages.) The host for this fragment is the org.hibernate.osgi bundle, of course.

This worked perfectly when launching my application from within the Eclipse IDE. However, I had an unpleasant surprise when running our batch builds which are based on the Eclipse PDE Ant runner.

PDE kept complaining about cyclic dependencies in my bundles. And not even in my own bundles, also between some of the third-party libraries used by Hibernate, in particular between jaxen and dom4j.

(I had a look at the sources of these two libraries to understand what was going on here: each of them contains some helper classes for the other one, and each must have been compiled against an older version of its friend - really scary...)

I had already spent half a day in working around this problem by repackaging each cycle of libraries in one bundle, and I was going to file an Eclipse bug report. When searching for similar issues, I found bug 208011 which not only describes the problem, but also offers a partial solution:

allowBinaryCycles = true

is a property you can set in your top-level build.properties file as of Eclipse 3.4. (Not a word about this in the Eclipse Online Help, and not even in the comments of the batch build template files!)

According to Chris Aniszczyk, this option will be accessible from the IDE UI in Eclipse 3.5M3.

allowBinaryCycles did suppress the error message regarding the dom4j-jaxen cycle.

But alas, now I had another error complaining about a cycle between Hibernate, my hibernate fragment and my model bundles. Apparently, Eclipse regards the fragment as part of its host (which is ok), and since the fragment depends on an application bundle, and the application bundle depends on Hibernate, so the PDE batch build now complains about a cycle between Hibernate and my application bundle.

So for now, I left all the resources and the JDBC dependencies in the Hibernate fragment, but reverted to using buddy declarations in Hibernate and my model bundles to break the dependency cycle.

I also created a fragment com.acme.myapp.antlr.fragment which imports org.hibernate.hql.ast.HqlToken and thus make the custom token class of Hibernate visible to ANTLR. (See the SpringSource bug report.)

All in all, this looks 95 % clean to me, so I think I'm going to leave it at that for a while...

Hibernate and OSGi: A pragmatic solution

The simplest thing you can do to turn a plain old JAR with a number of external dependencies into an OSGi bundle is wrapping this JAR and all its dependencies in another JAR and adding an OSGi manifest with all dependencies on the bundle classpath.

For example:

Bundle-SymbolicName: org.hibernate.osgi
Bundle-ClassPath: hibernate3.jar,
lib/antlr.jar,
lib/cglib.jar,
lib/commons-logging.jar,
...
Export-Package: org.hibernate,
org.hibernate.configuration,
...

With this megabundle, you can trivially solve all classpath or visibility issues between Hibernate and its dependencies.

You still have to do something to make the model classes of your application and the mapping files accessible to Hibernate.

Eclipse Equinox has an extension of the OSGi standard called buddy policies, which enables you to define some kind of classloader callback.

Your application depends on Hibernate, but you do not want Hibernate to depend on your application. Even if you wrap your own Hibernate bundle, you want to be able to use it in more than one of your applications.

There are several flavours of buddy policies, I will just mention one of them: Adding the header

Eclipse-BuddyPolicy: registered

to your Hibernate bundle manifest will tell Hibernate to ask all its buddies for classes it cannot load using its own classloader.

In each bundle of your application containing some Hibernate model classes, you then add a dependency on Hibernate and a header telling Hibernate that your bundle is a buddy:

Eclipse-RegisterBuddy: org.hibernate.osgi
Require-Bundle: org.hibernate.osgi

(The buddy thing does not work unless there is an actual dependency by Require-Bundle or Import-Package.) This will enable Hibernate to load classes and resources from your bundle, so it will also be able to locate your Hibernate mapping files if you put them into your bundle next to the model classes. Make sure to define your mappings in terms of resources, not files.

You can use the same approach for your JDBC drivers. More likely than not, your JDBC JAR does not come with an OSGi manifest, so you have to wrap it in a bundle anyway. Add a dependency on Hibernate and make your JDBC bundle a buddy of Hibernate. Logically, this is upside-down, but it has the advantage that you can exchange your JDBC drivers without changing the Hibernate bundle.

Alternatively, you can add optional dependencies to the Hibernate manifest on any JDBC driver you are planning to use, e.g.

Require-Bundle: com.microsoft.sqljdbc;resolution:=optional,
org.postgresql;resolution:=optional

Of course, this solution is a nightmare to any OSGi purist, but it does work and it is easy to set up.

There are the following major drawbacks:

• You are bound to Equinox. Other OSGi implementations do not have a buddy policy equivalent. To my best knowledge, you would have to resort to DynamicImport-Package which opens the gates much wider.
• The megabundle approach does not scale. It will be okay if Hibernate is the only component in your system you treat this way. If you have three or four of such heavyweight components you will end up wrapping general purpose JARs like commons-logging.jar over and over again.
• And you will be in real trouble if some of the wrapped third-party dependencies occur in more than one megabundle and are used on the API of your components. With multiple copies of the same JAR in different bundles, and each bundle having its own classloader, you will end up with class cast or assignment exceptions, because classes loaded via different class loaders are always incompatible, even if they have the same fully qualified name.

Hibernate and OSGi: The problem

Making Hibernate work in an OSGi environment is not trivial. Over the last couple of months, I have been experimenting with various approaches, and I'm not yet fully satisfied with the solutions I have found.

The funny thing is, there must be a whole bunch of people facing the same problem, and you do find a couple of postings or example code with the help of your favourite search engine, however, none of the articles or examples seem to match my environment or my requirements.

Now here is an outline of my approaches, which may or may not work for you. Take a look and give me some feedback.

Before talking about the solutions, we all know the answer is 42, but we haven't really worked out the question yet, so let me first list the issues you have to deal with:

• Hibernate has a lot of third-party dependencies.
• When you use an XML configuration file for your session factory, Hibernate must be able to load ist.
• Hibernate needs to load your JDBC driver classes.
  
• When using Hibernate mapping files for your entity classes, Hibernate needs to be able to load them as resources.
• While processing the mapping files, Hibernate reads the names of your entity classes and loads them using Class.forName().
• Hibernate internally uses ANTLR to parse HQL queries, and it passes a custom token class to ANTLR. Unfortunately, ANTLR requires you to specify this class by name and uses Class.forName() load the custom token class.
• When using lazy loading with the CGLIB proxy generator, Hibernate injects a dependency on CGLIB into your model classes. Thus, any bundle using you model classes may have to load CGLIB classes.

There may be more issues, depending on the way you work with Hibernate. In my project environment, some issues mentioned by other people simply do not occur, while others lead to new problems.

• We do not use Hibernate annotations.
• Our OSGi configuration is static. All bundles are installed and started during the framework startup phase, registering all services. We do not have to worry about adding new classes to the session factory when a bundle gets installed at run-time. (For this reason, the extender pattern approach is a bit too heavyweight for my taste.)
• We do not use a web container, or Spring, or anything else, just naked OSGi - well, Eclipse Equinox, in fact.
• Our IDE is Eclipse and we want to be able to launch our OSGi application including Hibernate from Eclipse.
• We also use Eclipse PDE batch builds.
  

Having described to problem in some detail, I will explain a pragmatic and a more elaborate solution in separate articles.

Why I use OSGi

I want modules and controlled dependencies in my Java applications. OSGi gives me both, that's why I use it.

It's as simple as that. Really.

Yes, OSGi gives you a lot more into the bargain, most notably services, which I also use, but the unique selling point for me is modules.

When you look at the sources and binaries of any software system (not only in Java), preferably one you haven't written yourself, you see a lot of files in a directory structure. Usually, the directory structure reflects the system architecture, subdirectories corresponding to subsystems or components.

Some of these directories correspond to binary artifacts (JARs, DLLs, whatever) which you may want to reuse independently or in another system.

Now when you start pulling out this library, you have to satisfy all its runtime dependencies. So you need to take a bunch of other libraries for this library to work. Identifying the dependencies may be a tedious trial-and-error process.

There may be additional compile-time dependencies you cannot recognize by looking at the binaries only. In C/C++, there is often some global.h directly or indirectly included by every source file in your system.

Even if there was some clever architect who designed the system in terms of components and allowed dependencies, there is usually no guarantee that this architecture will be adhered to, because nothing ever prevents a developer from including or importing something they are not supposed to use.

Java has packages and classes with visibility levels, but classes and even packages are rather too fine-grained from an architect's perspective. I want my modules to be larger than packages, and I want to use them as configuration units. The modules should be in one-to-one correspondence to libraries or JARs.

This is just what OSGi provides. A plain old JAR is turned into an OSGi bundle (that's what modules are called in OSGi-land) simply by adding a couple of special headers to its manifest.

The manifest headers declare the import and export relations between modules. A bundle B may have a package with public classes, but at runtime, no other bundle C can use these classes unless bundle B exports them.

When working with Eclipse, its Plugin Development Environment (PDE) already controls bundle imports and exports at compile time. You get a warning when you import a class that was not exported.

Of course this does not prevent you from adding import or export directives to your bundle manifests, thus diluting the original architecture. But Eclipse offers tools for the architect (or yourself) to inspect the bundle hierarchy and to easily detect any unintentional dependencies.

This is just one aspect of OSGi, but it's the one I find most valuable.