Large-Scale Selenium-Based Testing

A well established practice in web application development is testing a web-based GUI as a whole in order to ensure its functionality.

At studiVZ, we have to test three different platforms in two languages. There are many platform-specific differences, in both wording and functionality. The scenario is even further complicated by the use of AJAX functionality in some features. With a release cycle of 14 days, this is indeed a challenge for a small quality assurance team.

In addition to Unit Tests, Acceptance Tests are the main part of our automated testing strategy. There are thousands of acceptance tests that have been written since the rewrite of the platform in early 2008. Because of our special requirements (tests have to work for three platforms) we decided not to use one of these capture-and-replay web testing softwares but to write our tests by hand. We chose Selenium RC and the Selenium RC extension for PHPUnit that allows to write and execute acceptance tests in a unit test style.

But writing acceptance tests is a rocky road to success. The main problem with acceptance tests is their fragility: they will break when the code changes. Another challenge is keeping the tests maintainable, foremost readable. This was the motivation to create a domain-specific language for PHP that allows us to write tests in the language of our business rather than in Selenium's language.

We want to share our insight into the specific challenges of acceptance testing and how to handle them. Furthermore we will talk about our experiences and learning curve with a DSL-based framework for Selenium tests.

Christiane Philipps is the Team Lead Quality Assurance and Max Horváth is the Team Lead Mobile Development at studiVZ Ltd.