Judging by the size of the recent thread on the momolondon mailing list, the thorny issue of testing mobile websites is a hot topic.

There are many different software solutions around which will allow you to deliver markup to the myriad of devices currently available. Development for traditional, HTML based, websites has always been made more difficult as the different browsers interpret the markup in different ways and there are many tweaks needed to get the site to display correctly on all browsers. Interpretation of the standards, particularly CSS, seems to vary immensely (especially when some people, large companies based in Redmond for instance, seem to invent their own standards!).

This problem is multiplied many times over for mobile site development. Device independent rendering software solves a lot of the issues of screen size, memory limitations, paging etc – but what appears to be correct markup renders very differently on different devices. In my previous job, at a company that made device independent software, we had to make a large number of specific modules for certain devices to get round some of the stranger problems, one example that stands out is a device that would only display a whitespace after an anchor if you put three nbsp’s followed by a space.

Every device (even from the same manufacturer) seems to have different quirks displaying the markup but most of the problems occur in the different CSS implementations. Manufacturers either intentionally leave bits of the spec out or implement it incorrectly.

So, once you have your markup correct you have to start testing on a large number of devices to ensure your new big idea is displayed correctly. What’s the best way forward? There are several options:

• Buy every device and test each one
• Use emulators
• Use a service such as DeviceAnywhere
• Find a cheap way to hire lots of people to test on their own phones
• Automate the testing process
• Ignore it and hope everything is OK

Clearly buying every device is not an option given the number that exist and the rate of change. Normally devices are broken down into 5 or so different categories based on capability which gives a smaller sample size, but this can still be a large number of devices.

Emulators have their uses, but are not adequate for testing. Most of the emulators I have used are not accurate enough to test on and I have seen some that let markup through that crashes the real device.

Services such as DeviceAnywhere provide the actual devices to test on through a web interface, this is the best way of getting to a large number of devices but it is still time consuming to test on them all, and it can get expensive. There is also limited coverage at the moment so you can only test for a limited number of geographies. For a good overview of all the services available hop over to an article by an ex-colleague of mine, Tarek

If you know a lot of students you might be able to get some good testing coverage, but this would be tough to organise. A new service could address this problem, Mob4hire is like a social testing network which could be useful. There are also a number of options for this in the Far East, but network coverage could be an issue.

The holy grail here would be automating the testing process, but this is certainly a far off, if not impossible dream. You can automate testing the markup and this is useful for regression testing but you cannot automate looking at the display on the phone to check it.

If you think ignoring the problem and hoping it will be OK is the way forward, then you are probably in the wrong industry.

I would like to have been more positive in my first blog for bemoko and to reward you for getting this far with the answer to testing on mobile devices. Unfortunately I can’t provide that answer but bemoko has a lot of experience in testing sites that have been produced and, while it’s not perfect, a good strategy can be built to give a decent testing coverage using a mixture of the methods I have outlined above.