Over the last couple of months we’ve started to use Canoo Webtest to functionally test our mobile web sites. We now use it for both product deployment cycles (we spin up a full version of bemokoLive and test a sample site prior to packaging up the platform) and site development.

What has particular impressed us about Canoo is the way we can rapidly create clean and succinct tests that address all of the functional aspect of a mobile web site that we can need to automatically test. There is a rich set of functions in Canoo to click around a site, interact with a page and test functionality and content of a page. We write all our scripts in Groovy which provides a language that is light on scaffolding syntax (i.e. code just what you want to do), allows us to dig into misc Java libraries as we choose and provides a functional language that is human readable (well to us techies).

For example, take the following test script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

package com.bemoko.webtest.live
 
import com.bemoko.commons.test.webtest.BemokoWebtest
import com.bemoko.commons.test.webtest.WebtestXmlParser
import com.bemoko.live.devices.data.conf.HeroDevices
import com.canoo.webtest.engine.StepFailedException
 
 /** 
  * Test that welcome pages are rendered correctly to the hero devices
  */  
class PagesTest extends BemokoWebtest {  
  void test() {
    def parser=new WebtestXmlParser()    
 
    // (1) Iterate over HeroDevices    
    new HeroDevices().all.each { deviceData ->
      webtest("PageTests : ${profile} : ${deviceData.id}") {
        config(liveConfig) { 
          deviceData.evidence.each() { 
            header(name:it.key, value:it.value) 
          } 
        }
 
        // (2) Invoke the welcome page and verify some of the content
        invoke "/welcome/i", 
         description: "Index page : ${profile} : ${deviceData.id}"  
        verifyTitle "bemokoLive - i"
        verifyLinkedContent 
          xpath:"/html/head/link[@rel='stylesheet']/@href", 
          accept:"text/css"
        verifyLinkedContent 
          xpath:"//img/@src | //input[@type='image']/@src", 
          accept:"image/gif;image/png;image/jpeg"
 
        // (3) Validate document using an XML parser
        groovy { 
          parser.parseText(step.context.currentResponse
            .webResponse.contentAsString) 
        }
      }
    }
  }
}

The bemokoLive test suite provides a collections of devices that allow you to easily simulate requests from particular devices. The HeroDevices class, in the script above, provides a collection of around 10 devices that provide a representative coverage of different types of devices. We (1) iterate over this collection and pass in the evidence for the device, i.e. the HTTP headers, into the webtest configuration. We then (2) invoke a particular page, test its title and verify that the linked content is good. Finally (3) we validate the document against the DTD in the document – great for making sure nothing bad has slipped into the page.

Reports get generated with a full break down on the tests and any step failures along with a quick summary, for example the following screen shot shows the summary of test run of 431 tests which we ran against one of our customer sites. I’ve deliberately included a report with failures to make the summary a little more interesting:

There’s much more you can do with canoo and the Groovy scripting approach – take a read of automated testing on our bemoko wiki for some more pointers.

With this approach we can build up a robust set of scripts that verify our deliver and provide great foundations for our real device testing – no more burning time on real device testing dealing with functional issues that could have been caught earlier in development. This is also good black box test, so whether you’re using bemokoLive or a.n.other approach it’s equally applicable. If you’ve got a mobile web site that you want us to test, or help you get started to run this testing yourself just get in touch.