07 June 2011

Java EE 6 Server Comparison: Conclusion

Lessons Learned

Portability is not just a Java EE marketing promise. My application now runs on three different Java EE 6 servers, using the same WAR file, without any compile-time configuration.

An application is not portable until it has been ported. Of course, this is true for any specification with different implementations, not just for Java EE. Each of the three servers helped me discovering incorrect API usage in my application that just happened to work on another server.

Certification is not reliable. My application hit at least one major specification violation in each of the three Java EE 6 certified servers. Thus, the test coverage in the Java EE TCKs must be insufficient.

A certification process for an open specification should be open. Implementations of the specification may be closed source, but the test suites (TCKs) should be open source, and the user community should have the chance to contribute new test cases.

The next version of the Java Community Process is moving in the right direction, but if you read the proposal carefully, you will notice a lot of half-hearted may's and should's, which is not enough. TCKs shall be open, period.

(Note: Most of the JCP site is currently unavailable, due to restructuring - I read JSR 348 a few days ago. Oracle, did you ever hear about high availability and Cool URLs don't change?)

And the winner is...

Each of the three servers has its strong points and its weaknesses.

Only JBoss has an official release that is able to run my application (with a workaround). On the other hand, its memory footprint is enormous, the deployment speed is inacceptable and the weakest point is the lack of up-to-date documentation. The Eclipse integration is severely limited in not allowing to republish applications.

Resin has the smallest footprint by far, clearly in terms of memory, less clearly in terms of deployment speed, where Glassfish is close up and might scale better than Resin for larger applications due to Resin's code generation during deployment. Resin needs a few more releases to mature. Its Eclipse integration is too unstable for serious use.

Glassfish is the leader in end-to-end usability. Its ease of use and its excellent documentation are invaluable for promoting the benefits of Java EE 6, keeping the barrier low for newcomers.

Its speed is excellent (though outrivalled by Resin), its memory footprint needs to be improved, but is still a lot smaller compared to JBoss. Important to note (though without any impact on my experiments) is the fact that Glassfish is the only one of the three certified for the Java EE 6 Full Profile.

Compared to JBoss and Resin, Glassfish has the best Eclipse integration (but the standard Eclipse WTP Tomcat integration is still better).

Glassfish needs to shift the balance from new features to stability. There has been no production quality 3.x release so far. Glassfish 3.1.1 is on its way, it has a chance of becoming the first really stable release, and I'd rather wait for it a little longer to make sure that all major known bugs are fixed.

Glassfish would benefit from quarterly maintenance releases and more transparency in community communications.

All in all, if you ask me

  • Which server would you choose for an application to go live next week?
  • Which server would you choose for developing a new project to go live next year?
then my answer is Glassfish for both. And if I couldn't use Glassfish, I would choose JBoss for next week and Resin for next year.

Executive Summary

Glassfish is Business Class, JBoss is Baroque, Resin is Zen.

Read the whole story


Michał Huniewicz said...

Harald, would you recommend deploying a JEE app in those 3 servers as a mean to get the JEE configuration right and achieving portability?

Harald Wellmann said...

@Michał: I'd say it depends on the situation... When you have a problem with your application on server X which you can't figure out, you may gain some insight by deploying your app on server Y, e.g. from a more specific error message which points you to the root cause in your app, or by realizing the problem was in server X and not in your app.

When you don't have any problems but some time to spare, this exercise can still be useful to learn things about your application and/or Java EE you'd never noticed before, because they were handled differently on your favourite server.

Archimedes said...

My general opinion to what platform to deploy to is: whatever the client wants. In they end, they're the people that are paying. So if they are saying we want it to work on WebSphere, WebLogic, JBoss, Oracle, DB2, WebSphere MQ, etc, we just factor in the cost.

As for where I would develop stuff, I would use Glassfish + Eclipse + Maven. However, I don't use any extra things like MySQL (I use the built in JavaDB that comes with Glassfish) same with JMS.

This allows me to quickly ramp up developers since they have significantly less stuff to install. The development environment can be set up easily with full debug capability at the cost of time which if you have done once you don't really spend as much time later on and the memory footprint for development is a lot less.

The good part is 90% of the time it is easy to develop code that is portable across any server or database or JMS queue.

However, some of the more "expensive" and "enterprisey" ones that the customers use require a bit more hand holding for deployment even if they say they are compatible. I use an integration server for that that is close to what production would have minus the cluster (just to be cheaper). Then get some people from my team working on dealing with the "it's not a bug it's a feature" parts of the very expensive systems.

This is the same approach I would take with Internet Explorer 6. You write something simple and portable then add the IE6 compatibility junk for a premium.

PronoiaPieter said...

How do you feel the Eclipse/Tomcat is better than Eclipse/GlassFish?

Just curious.

Harald Wellmann said...

@Peter: As I wrote in the Glassfish part, the Eclipse plugin fails to redeploy your webapp when you change a source in a dependent module. This works with Tomcat.

Anonymous said...

The jboss tools plugin has support to publish the entire application or the modified resources, take a look at this screenshot

More details at

Paulo Cassiano said...

thank you so much!

vladchuk said...

Lately I've been evaluating a possibility of switching a fairly complicated enterprise application from JBoss 4.2 to Glassfish 3.1.

I did manage to "port" the app to Glassfish, at least to the extent that it deploys successfully. Immediately, I noticed a few differences, I'm afraid, not in favor of Glassfish. All parameters seem to be worse on Glassfish: more classes loaded, more threads runing and, most importantly, memory consumption is at least twice of that on JBoss. Start-up time is worse as well.

Perhaps it's not a fair comparison, given different generations of the two paltforms, different functionality, etc., but it surely makes one wonder...

Craig Ringer said...

I'm interested in finding out if you ever solved your issues with resource references and JNDI naming for data sources.

I've been unable to convince either glassfish or jboss to use indirect references to datasources via web.xml resource-ref and (glassfish|jboss)-web.xml resource-ref mappings. Both simply seem to ignore the mapping and report that the datasource cannot be found by the unmapped name.

Did you end up sticking with the hack you mentioned for JBoss, where you forced it to conform to the jdbc/blah naming scheme used by Glassfish? Or did you eventually figure out how to get resource-ref mapping working?

Harald Wellmann said...

@Craig: No I didn't, I've left it with jdbc/foo. I spent enough time trying to make the indirection work, without much success...

Craig Ringer said...

@Harald: I eventually concluded it was a bug in GF and filed it. The result was astonishing: it's not supposed to work!


"The resource-ref element that you refer to above defines an indirection within a component name space. A PersistenceUnit (EMF) is not initialized within a component and hence always uses the global name space to look up data sources."

So ... indirection is only for direct JDBC access and for EJBs, not for JPA. Apparently.

The JPA 2.1 spec leads are now aware of this issue and are considering addressing it for JPA 2.1. So hopefully it'll work down the track.

Corneil said...

Having looked at http://java.net/jira/browse/GLASSFISH-17024 I think the problem is not limited to JPA. We have a problem in that we can't get resource-ref to work for all kinds of jndi lookups using Spring including mail session and plain hibernate datasource.