Automated Testing

From bemoko developer wiki

Jump to: navigation, search

Contents

Introduction

bemoko test sites early in a site development process, in fact we often define our tests as one of the first steps of a project. For our testing we use Canoo WebTest[1] to create scripts that allow an excellent functional test coverage of (mobile) web sites. By creating these scripts to test a site from day one, we build a set of tests that allow us to verify that a site is behaving. This is especially useful as updates are applied you need that extra level of confidence that you have fixed what you expected to fix and have not caused regression issues.

Example : Testing the Welcome Site

When you install bemokoLive for the first time you probably saw the welcome site. It's a pretty simple site that we used as a sanity check that you got bemokoLive installed OK. We also use this site to test the whole bemokoLive stack before we release a new version. Let's take a look at some elements of how we do that, because it provides a great example in how you can test other (mobile) web sites.

/**
 * Copyright 2009 bemoko 
 */ 
package com.bemoko.webtest.live
 
import com.bemoko.commons.test.webtest.BemokoWebtest
import com.bemoko.commons.test.webtest.WebtestXmlParser
import com.bemoko.live.devices.data.conf.HeroDevices
import com.canoo.webtest.engine.StepFailedException
 
 /** 
  * Test that welcome pages are rendered correctly to the hero devices
  */  
class PagesTest extends BemokoWebtest {  
  void test() {
    def parser=new WebtestXmlParser()    
 
    // (1) Iterate over HeroDevices    
    new HeroDevices().all.each { deviceData ->
      webtest("PageTests : ${profile} : ${deviceData.id}") {
        config(liveConfig) { deviceData.evidence.each() { header(name:it.key, value:it.value) } }
 
        // (2) Invoke the welcome page and verify some of the content
        invoke "/welcome/i", description: "Index page : ${profile} : ${deviceData.id}"  
        verifyTitle "bemokoLive - i"
        verifyLinkedContent xpath:"/html/head/link[@rel='stylesheet']/@href", accept:"text/css"
        verifyLinkedContent xpath:"//img/@src | //input[@type='image']/@src", accept:"image/gif;image/png;image/jpeg"
 
        // (3) Validate document using an XML parser
        groovy { parser.parseText(step.context.currentResponse.webResponse.contentAsString) }
      }
    }
  }
}

This example is a Canoo WebTest written in groovy that allows us to test a particular page over a group of devices. The bemokoLive test suite provides a collections of devices that allow you to easily simulate requests from particular devices. The HeroDevices class provides a collection of around 10 devices that provide a representative coverage of different types of devices. We (1) iterate over this collection and pass in the evidence for the device, i.e. the HTTP headers, into the webtest configuration. We then (2) invoke a particular page, test its title and verify that the linked content is good. Finally (3) we validate the document against the DTD in the document - great for making sure nothing bad has slipped into the page.

What Can you Test with Canoo?

Canoo webtest has a rich set of functions[2] that allow you test for many aspects of a web sites:

  • Step through forms.
  • Verify that page markup (or content) on a site / page hasn't changed.
  • Make sure device is identified correctly.
  • Make sure device conditional content is displayed as desired - e.g. an ad that should only be delivered to a subset of devices.
  • Make sure the correct content / navigation is displayed.
  • Make sure that web site is functionally correct, such as user can login, unauthorised user can't access secure areas, click buttons, click links
  • ... the list goes on.


Now bemoko already provide coverage of many of these tests, such as device identification, so some of these tests may not concern you, but we do like to be open with you and share how we do this testing so that you can extend it for you needs. You might even want to use this approach to test sites which weren't created using the bemoko approach.

References

  1. Canoo WebTest Home - http://webtest.canoo.com/webtest/manual/WebTestHome.html
  2. Canoo Webtest manual - http://webtest.canoo.com/webtest/manual/manualOverview.html