Portability is not just a Java EE marketing promise. My application now runs on three different Java EE 6 servers, using the same WAR file, without any compile-time configuration.
An application is not portable until it has been ported. Of course, this is true for any specification with different implementations, not just for Java EE. Each of the three servers helped me discovering incorrect API usage in my application that just happened to work on another server.
Certification is not reliable. My application hit at least one major specification violation in each of the three Java EE 6 certified servers. Thus, the test coverage in the Java EE TCKs must be insufficient.
A certification process for an open specification should be open. Implementations of the specification may be closed source, but the test suites (TCKs) should be open source, and the user community should have the chance to contribute new test cases.
The next version of the Java Community Process is moving in the right direction, but if you read the proposal carefully, you will notice a lot of half-hearted may's and should's, which is not enough. TCKs shall be open, period.
(Note: Most of the JCP site is currently unavailable, due to restructuring - I read JSR 348 a few days ago. Oracle, did you ever hear about high availability and Cool URLs don't change?)
And the winner is...
Each of the three servers has its strong points and its weaknesses.
Only JBoss has an official release that is able to run my application (with a workaround). On the other hand, its memory footprint is enormous, the deployment speed is inacceptable and the weakest point is the lack of up-to-date documentation. The Eclipse integration is severely limited in not allowing to republish applications.
Resin has the smallest footprint by far, clearly in terms of memory, less clearly in terms of deployment speed, where Glassfish is close up and might scale better than Resin for larger applications due to Resin's code generation during deployment. Resin needs a few more releases to mature. Its Eclipse integration is too unstable for serious use.
Glassfish is the leader in end-to-end usability. Its ease of use and its excellent documentation are invaluable for promoting the benefits of Java EE 6, keeping the barrier low for newcomers.
Its speed is excellent (though outrivalled by Resin), its memory footprint needs to be improved, but is still a lot smaller compared to JBoss. Important to note (though without any impact on my experiments) is the fact that Glassfish is the only one of the three certified for the Java EE 6 Full Profile.
Compared to JBoss and Resin, Glassfish has the best Eclipse integration (but the standard Eclipse WTP Tomcat integration is still better).
Glassfish needs to shift the balance from new features to stability. There has been no production quality 3.x release so far. Glassfish 3.1.1 is on its way, it has a chance of becoming the first really stable release, and I'd rather wait for it a little longer to make sure that all major known bugs are fixed.
Glassfish would benefit from quarterly maintenance releases and more transparency in community communications.
All in all, if you ask me
- Which server would you choose for an application to go live next week?
- Which server would you choose for developing a new project to go live next year?
Glassfish is Business Class, JBoss is Baroque, Resin is Zen.