- Glassfish 3.x
- JBoss 6.x
- Resin 4.0.x
Taking an application developed on server X and porting it to servers Y and Z might cause some bias towards server X, so my scenario is the following:
- MyApp is an enterprise application originally developed on Tomcat 6 and Spring 3, using JPA 2.0 (Hibernate on MySQL), declarative transactions, dependency injection, and Spring MVC in the web layer. In the persistence and service layers, this application has no compile time dependencies on Spring, thanks to the fact that Java EE 6 annotations like @Inject or @TransactionAttribute are also supported by Spring.
- MyApp is built with Maven, my development environment is Eclipse 3.6.2 with m2eclipse.
- Make MyApp run on a Java EE 6 server (using JPA, JTA, EJB, CDI, Servlet, JSP) and restrict the run-time use of Spring to the web layer, using a CDI-to-Spring bridge.
- Build a WAR that can be deployed to each of Glassfish, JBoss and Resin.
- Work with the Eclipse plugin for each server, compare the edit-save-deploy cycles and developer productivity in general.
- Work with the stand-alone servers, and compare startup and deployment times and memory usage.
A note on Java EE 6 Profiles and certification: Glassfish is of course certified both for the Full Profile and for the Web Profile. JBoss AS 6.0.0.Final was announced as a fully certified implementation of the Java EE 6 Web Profile, but for some reason it is not yet listed on the official Java EE compatibility page. JBoss also claims to support the Full Profile, but has not taken the pains of undergoing certification for it. Resin is explicitly focused on the Web Profile and has recently been certified. Resin also supports JMS, which is not part of the Web Profile.
MyApp only requires the Web Profile. For my experiments, I've used the Full Profile installation of Glassfish, the default server configuration of JBoss, and the Open Source Resin installation (not Resin Pro).
Performance Measurement Scenarios
For each server, I ran the following tests:
- Empty server startup time. ("Empty" means without any applications deployed by the user, running the server out of the box, which may include some management applications.)
- Empty server total heap and PermGen size. (After starting the server and forcing a garbage collection from jvisualvm.)
- MyApp deployment time (copying it to the autodeploy directory of the running server).
- Server + MyApp restart time (restarting the server with my deployed application).
- Server + MyApp total heap and PermGen size. (After starting the server with the deployed applicaton and forcing a garbage collection from jvisualvm).
- MyApp redeployment time (after touching myapp.war in the autodeploy directory).
- Server + MyApp startup time from Eclipse. (Using Run on Server on the top-level web project).
- Server + MyApp total heap and PermGen size, running from Eclipse (after forcing GC as above).
- MyApp redeploy time after changing a source file in Eclipse.
- Checking for memory leaks after redeploying.
7 comments:
Harald,
Thanks so much for your kind words about Resin and the Resin team. We take your comments very seriously and will make sure to address them before the upcoming Resin 4.1 release. Please let us know if you have any more comments for us. This is the ideal time for that since we are currently focused on stability, bugs and performance before we have the 4.1 release.
Cheers,
Reza
Hi Harald!
Nice comparison, but I miss the most important number in the performance tests: How fast is the web application itself?
I found factor 15 differences when I first tested a big webapp on different EE servers and on tomcat + OpenWebBeans.
And it really makes a difference if a page renders in 300ms or in 5 seconds...
LieGrue,
strub
@strub: You're right, I left out runtime performance, but that was a deliberate decision.
It takes some kind of test harness, which I didn't have at hand, and most of the work in my application happens in the persistence level, so I don't know if my results would have been representative for apps with a lot of web traffic.
We'd actually be very interested to know that since runtime performance, especially vis-a-vis plain Servlet containers, has always been a focus for us. As far as I know, the only recent well-known benchmark in this area has been GlassFish (Grizzly) vs. Tomcat (in which Grizzly seemed to fare quite well).
You have a good point that pure web performance concerns do get outweighed by persistence tier performance for most realistic web applications.
Pitty this does not mention websphere, which is Java EE6 full profile certified. I thought this was a strict superset of the web profile.
@Jan: I published this series of articles before WebSphere AS 8.0 was released.
Maybe you can also look at http://siwpas.mechsoft.com.tr for Siwpas.
Post a Comment