Performance testing is an often overlooked and/or poorly executed activity.  A standard part of any project implementation is to test the system under load – often this is the first time the code has been tested in such a way and it highlights major problems which should have been found in the QA cycle.  I can’t count the number of times, in previous corporate lives, that I have had to jump on a flight at the last minute because there is a panic as the product has failed load testing.  In one case it was found that the throughput was a stunning 0.5 tps!!

Whilst all situations and loads cannot be catered for, there should be a standard test suite so that the performance of new releases can be checked and compared to previous releases.  A new release with lots of fancy features should not be slower than the previous release!

At bemoko, we use our own website as the test application.  This provides us with a benchmark to test all our releases.

We test for performance and memory leaks using the excellent Apache JMeter to generate the load.  We have a file containing all the URL’s on the website and another file which contains various user agents to run against the site.  We can then simulate any number of users continually accessing the pages on different devices.

Java profiliing is done using the YourKit profiling tool.  Of all the profilers I have used, this is the easiest and least intrusive one I have found.  It must be good if I can track down memory leaks using it!

I’ve just done the testing for our latest release and am pleased to report that under a load simulating 600 users we got an impressive 25tps with an average page load time of 414ms.  After a 6 hour run and a quick garbage collection, the memory use went back to the level at the start of the test – so no nasty memory leaks to track down this time.  Top marks for the technical team!

So I am now confident that we and, more importantly, our customers will not get any nasty surprises when it comes to performance testing the application.   And my wife and children will be pleased that I don’t have to disappear off on a flight at short notice!