Quality Assurance in PHP Projects

Almost Done

nospam@example.com (Sebastian Bergmann) — Fri, 30 Apr 2010 13:30:00 +0200

We are almost done with the manuscript for the German edition (preorder at Amazon.de) of the book. In a couple of days we can focus on getting the manuscript for the English edition (preorder at Amazon.com) ready for the publisher.

Update

nospam@example.com (Sebastian Bergmann) — Wed, 04 Nov 2009 09:30:00 +0100

It has been quiet on this website since we posted the last contributed chapter abstract in June and it is time to give you an update. But first, allow me to refresh your memory on the list of contributed chapters:

Best Practices
TYPO3: The Agile Future of a Ponderous Project
Unit Testing Bad Practices by Example
Quality Assurance at Digg
Servers and Services
Testing Service Oriented APIs
Testing a WebDAV Server
Testing Database Interaction
Architecture
Testing Symfony and Symfony Applications
Testing the ezcGraph Component
QA in the Large
Large-Scale Selenium-Based Testing at studiVZ
Continuous Integration with phpUnderControl
swoodoo: A True Agile Story
Non-Functional Aspects
Performance
Usability
Security

As we only have a deadline for the German edition so far and since we received the majority of contributed chapters in English, Stefan and I are currently busy translating the contributed chapters from English to German in an effort to meet our deadline. We are currently looking at a release date for the German edition in late spring / early summer of 2010.

Once we have finished the manuscript for the German edition, we will start working on the manuscript for the English edition. Hopefully, this process will be much faster as we only need to translate a couple of chapters from German to English. We are hopeful that the English edition will be available not much later than the German edition.

Unit Testing Bad Practices by Example

nospam@example.com (Benjamin Eberlei) — Fri, 05 Jun 2009 12:00:00 +0200

This is an abstract for a chapter from a book on Quality Assurance in PHP Projects.

Although Unit Testing is a recommended practice for any software project, care has to be exercised such that testing yields the desired benefits. Bad programming practices in both test code and production code can make Unit Testing a nightmare. Maintenance of an overly complex test suite can easily become a burden to the project team.

Situations of hard to maintain test code can have extremely negative outcomes for project quality. Programmers start to ignore tests, because of their inconclusive description of what is going wrong. New features might not be tested at all, because writing tests for the current architecture considerably extends development time. In the end the project manager might decide to stop Unit Testing alltogether, because the costs outweight the benefits.

Bad practices of Unit Testing manifest in so called "Test Smells". These are an early indicator of problems for long-run maintainability and utility of a project test suite. To constantly derive a benefit from Unit Tests, a commitment for high quality tests has to be made.

This case study will discuss Unit Testing bad practices and well as Test Smells and gives hints on how to avoid them. For each test smell, examples are shown from well-known PHP Open Source projects. As a result, the reader should be aware of the possible pitfalls of Unit Testing and that any test code requires the same care that is put into production code.

Benjamin Eberlei is a Project Lead with direkt effekt GmbH and contributes to the Zend Framework and other Open Source projects.

Testing Service-Oriented APIs

nospam@example.com (Matthew Weier O'Phinney) — Fri, 29 May 2009 12:00:00 +0200

This is an abstract for a chapter from a book on Quality Assurance in PHP Projects.

One of Zend Framework's key strengths and differentiators in the crowded PHP framework arena is its offering of web service consumer components. As of version 1.8.0, these include offerings for Adobe AMF, Akismet, Amazon (including EC2 and S3 support), AudioScrobbler from last.fm, Del.icio.us, Flickr, Google's GData services, Nirvanix, ReCaptcha, Simpy, SlideShare, StrikeIron, Technorati, Twitter, and Yahoo! -- and more are planned or under development.

Offering components to interact with web services is relatively trivial -- but testing them offers a very different story. For continuous integration purposes, it is typically better not to test against live services as this introduces network latency, and thus slows the testbed. Additionally, many services have API call limits, and automated test suites could easily rise far above these limits. There are also concerns about keeping sensitive API keys or credentials in the repository; some services are paid services, and such information would present a breach of security or contracts.

Over time, the Zend Framework team and contributors have developed a set of practices to address these issues that range from mocking the request and response payloads to offering different tests when credentials are provided via configuration. These practices can lead to well-tested service-oriented components and ensure that the end user experience is as documented.

Matthew Weier O'Phinney works for Zend Technologies Ltd. and is the Project Lead for the Zend Framework.

Continuous Integration with phpUnderControl

nospam@example.com (Manuel Pichler and Sebastian Nohn) — Fri, 22 May 2009 12:00:00 +0200

This is an abstract for a chapter from a book on Quality Assurance in PHP Projects.

One of the most important events in large and distributed team projects is the integration of all parts of the software system into a release. Often incompatibilities and hard-to-detect errors surface at that very moment and can harm the success of the project. The remedy for this problem is the practice of Continuous Integration: a merge of all parts of the system as often as possible during the lifecycle of the project.

In this case study, we will take a look into the preconditions to start with Continuous Integration as well as the procedural and technical rationales of this approach. We will take a look at entering Continuous Integration in PHP projects with phpUnderControl, a customization of the popular and established Continuous Integration server software CruiseControl.

Beyond Continuous Integration, this case study introduces the reader to the principles of Static Testing and how to instrument it to gain a better quality. Amongst other tools, the reader will learn how to successfully use tools such as PHP_CodeSniffer, PHP_Depend and phpcpd for Static Code Analysis.

In conclusion, advanced usage scenarios of the discussed tools will be discusses with a particular focus on how to foster further principles of Agile Software Development methodologies and how to solve discrepancies with other targets of agile methodologies.

Manuel Pichler is the creator of phpUnderControl, PHP_Depend, and phpmd. Sebastian Nohn is the Team Lead Quality Assurance at Ligatus GmbH.

Testing Symfony and Symfony Applications

nospam@example.com (Fabien Potencier) — Fri, 08 May 2009 12:00:00 +0200

This is an abstract for a chapter from a book on Quality Assurance in PHP Projects.

In this case study, we will talk about how the Symfony framework itself is tested, but also about the tools the framework provides to ease testing the customer applications.

Although developers are well educated now about tests, it is still the most neglected part of a web development process. Most developers still think they cannot afford writing tests because it takes too much time. The Symfony framework tries to ease the creation of both unit tests and functional tests with a unique and simple approach. For instance, functional tests are done by simulating a browser experience, but with the possibility to introspect all internal objects between each request. It is also possible to validate the generated content easily and precisely thanks to CSS3 selectors.

Testing the framework itself also proved to be quite challenging, and we learned it the hard way: from code that is too coupled to be tested thoroughly, to the usage of design patterns like the Singleton. Over the years, the Symfony framework quickly evolved from our testing experience, and now provides a well decoupled, but cohesive set of components. For Symfony 2, the introduction of a Dependency Injection container will be of great help to ease the testing process even more.

Fabien Potencier is the CEO of Sensio and the lead developer of Symfony.

TYPO3: The Agile Future of a Ponderous Project

nospam@example.com (Robert Lemke and Karsten Dambekalns) — Fri, 01 May 2009 12:00:00 +0200

This is an abstract for a chapter from a book on Quality Assurance in PHP Projects.

When Kasper Skårhøj wrote the first lines of PHP code laying the foundation for the TYPO3 CMS in 1998, he had not the slightest idea that a big part of the code he wrote would still be actively used ten years later. Adding the 3700 extensions contributed by an ever growing developer community resulted in an impressive amount of code - of varying quality.

Without surprise it soon became a challenge to assure a high quality and secure product. While the core developers were eager to add new features, the holy cow of backward compatibility was lurking at the next corner, waiting to be fed. The result is a feature-rich and solid but equally monolithic and complex pile of code that is, at best, hard to test. As a consequence we decided to rewrite TYPO3 from scratch.

In this case study we will share our experience with the technical and organisational techniques we have chosen for the development of TYPO3 version 5 and its foundation, the FLOW3 framework. It explains our pedantic pursuit of clean code, outlines our commit and coding guidelines and gives answers to tricky situations you face while writing and maintaining (true) unit tests. Finally we demonstrate how a clean architecture can help you tackling the complexity of your application.

Robert Lemke and Karsten Dambekalns are core developers of both TYPO3 and FLOW3.

A few recipes for writing good unit tests that are part of this case study are demonstrated in the "Delicious Test Recipes" episode of Robert Lemke's podcast.

Testing the ezcGraph Component

nospam@example.com (Kore Nordmann) — Fri, 17 Apr 2009 12:00:00 +0200

This is an abstract for a chapter from a book on Quality Assurance in PHP Projects.

The Graph component from the eZ Components project provides an easy to use and extensible library for creating graphical charts. In the eZ Components project we aim to develop test-driven, which resulted in two challenges developing this library.

The output generated by a Graph component is an image, which in most cases is binary-encoded data. Expectations are really hard to create by hand, it is only feasible to create functional tests replaying once created data. The actual image is generated by external libraries such as ext/gd. These external libraries may change their exact output at any time, or even between two runs with the same version of the library. Comparisons of the generated images need to respect that and implement special assertions for the given type of binary data.

Testing the created binary data has proven to be prone to errors. Since the actual image generation has been abstracted away, we are able to use of different image creation mechanisms. Thanks to the support for mock objects in PHPUnit, all calls to the driver layer can be checked for correctness without actually generating images.

Kore Nordmann is an active member of the PHP community and works for eZ Systems on the eZ Components.

Quality Assurance at Digg Inc.

Fri, 10 Apr 2009 12:00:00 +0200

This is an abstract for a chapter from a book on Quality Assurance in PHP Projects.

Digg's development philosophy has changed dramatically in the last year in an attempt to move towards a more agile and iterative approach. A large part of these sweeping changes is moving toward a test-driven development methodology with a minimum of 80% code coverage.

Integrating unit testing into coding practices is a daunting task at first glance. It requires adjusting how developers work in almost every respect, but it can also greatly assist in debugging and validating your code. We wanted to get this right the first time, so we decided to hire an outside contractor to help us formulate a plan of attack. Once we decided upon a course of action, it took only a week for the contractor and two developers to build a test harness and small suite of tests for our core classes, we even found a handful of bugs along the way. Next, we worked to instruct the remaining developers at Digg of best practices for writing unit tests.

Although in one week we went from having only a handful of previously written tests to having a framework to build unit tests much more easily, our work was far from complete. Even our QA team has dramatically changed how they operate to take advantage of the benefits unit testing offers.

Our case study will discuss in depth benefits and drawbacks of adopting test-driven development, cover the gamut of unit testing from simple assertions to mocking complex objects, and dissect the continuing challenges we face.

Robert Balousek, Matt Erkkila, Ian Eure, Bill Shupp, Jeremy McCarthy and Brian O'Neill all contributed to the case study and are employees of Digg.

Large-Scale Selenium-Based Testing

nospam@example.com (Christiane Philipps and Max Horváth) — Fri, 03 Apr 2009 12:00:00 +0200

This is an abstract for a chapter from a book on Quality Assurance in PHP Projects.

A well established practice in web application development is testing a web-based GUI as a whole in order to ensure its functionality.

At studiVZ, we have to test three different platforms in two languages. There are many platform-specific differences, in both wording and functionality. The scenario is even further complicated by the use of AJAX functionality in some features. With a release cycle of 14 days, this is indeed a challenge for a small quality assurance team.

In addition to Unit Tests, Acceptance Tests are the main part of our automated testing strategy. There are thousands of acceptance tests that have been written since the rewrite of the platform in early 2008. Because of our special requirements (tests have to work for three platforms) we decided not to use one of these capture-and-replay web testing softwares but to write our tests by hand. We chose Selenium RC and the Selenium RC extension for PHPUnit that allows to write and execute acceptance tests in a unit test style.

But writing acceptance tests is a rocky road to success. The main problem with acceptance tests is their fragility: they will break when the code changes. Another challenge is keeping the tests maintainable, foremost readable. This was the motivation to create a domain-specific language for PHP that allows us to write tests in the language of our business rather than in Selenium's language.

We want to share our insight into the specific challenges of acceptance testing and how to handle them. Furthermore we will talk about our experiences and learning curve with a DSL-based framework for Selenium tests.

Christiane Philipps is the Team Lead Quality Assurance and Max Horváth is the Team Lead Mobile Development at studiVZ Ltd.

Testing Database Interaction

nospam@example.com (Michael Lively Jr.) — Fri, 27 Mar 2009 12:00:00 +0100

This is an abstract for a chapter from a book on Quality Assurance in PHP Projects.

Proper interaction with relational databases is incredibly important in most pieces of software. The importance of this interaction is underscored by the fact that many software architectures have at least one entire layer or system devoted to data persistence and data loading. When dealing with critical data in enterprise level applications it becomes even more important to make sure your data is being stored and retrieved correctly.

As responsible developers, one of our goals should be to test database interaction and test it well. To help achieve this goal, we will take a close look at tools and techniques that can be used with projects of any size to help create reliable tests capable of validating your database interaction.

The discussion will begin with various techniques for testing your database layer. The focus will then move toward the functionality in the database extension for PHPUnit. Then, utilizing the tools provided in the database extension, examples will be given of how to create basic tests validating that the database is correctly manipulated and data from your database is successfully retrieved. This will be followed by an in depth discussion of "best practices" that can be utilized when you are developing "database aware" fixtures and tests.

Once the ground work is laid, we will look at how all of these tools and techniques can be utilized in both your unit tests and functional tests.

Michael Lively Jr. is the Lead PHP Developer for Selling Source, LLC.

Testing a WebDAV Server

nospam@example.com (Tobias Schlitt) — Fri, 20 Mar 2009 12:00:00 +0100

This is an abstract for a chapter from a book on Quality Assurance in PHP Projects.

The eZ Components project was started with the goal to create a high-quality set of independent building blocks for PHP 5 based application development. Making extensive use of unit tests and sticking to strict development and documentation standards were defined as the basic methodology. One of the biggest challenges in terms of testing and quality assurance so far was the eZ Webdav component.

The eZ Webdav component is intended to provide a modular WebDAV server to be embedded in any kind of web application. Its architecture is built to be as customizable and extensible as possible, while it tries on the other hand to compensate the weaknesses of the RFC and common WebDAV clients.

Testing a server is a challenge in general. Testing the WebDAV component brought us to the limit of unit testing and forced us to find new ways of using PHPUnit to ensure code quality and to obviate regressions. Unavoidable deep code dependencies in some parts of the server enforced the extensive use of mock objects to stick to the unit test paradigm. In some areas, like avoiding regressions in the adjusted behavior against special clients, unit testing was even impossible. For this reason, special record/playback-based tests were created and implemented using PHPUnit.

Tobias Schlitt is an active member of the PHP community and works for eZ Systems on the eZ Components.

swoodoo - A True Agile Story

nospam@example.com (Lars Jankowfsky) — Fri, 13 Mar 2009 12:00:00 +0100

This is an abstract for a chapter from a book on Quality Assurance in PHP Projects.

swoodoo, the Germany-based flight search engine, has gone through a wild evolution. In 2003 it started out as a peer-to-peer network with ActiveX browser plugins written in C++ to query cheap flights. It went through a Java-centric solution using various techniques for screenscraping data from airline websites and finally has evolved into a robust architecture where Java and MySQL are used in the backend and PHP plays a major role in all frontend-related tasks as well as in data gathering from airlines.

Besides technology, the software architecture has also changed immensely: from distributed objects to a "big ball of mud" and finally into a Service-Oriented Architecture (SOA) with clearly defined interfaces between separate services. In order to have a better control of swoodoo evolution, project management was switched to Extreme Programming (XP) in the beginning of 2007. At the same time we introduced unit testing using JUnit and PHPUnit as well as PHP-driven Selenium acceptance testing. Initially we made heavy use of Selenium but recently we have reduced the amount of Selenium tests - a tradeoff we had to make because of the nature of our SOA-based architecture.

In this case study we want to show the evolution of our agile management techniques, testing environment, tools and approaches we use and why we decided to change in case we did.

Lars Jankowfsky is the CTO of Germany-based swoodoo AG.

Performance Testing

nospam@example.com (Brian Shire) — Fri, 06 Mar 2009 12:00:00 +0100

This is an abstract for a chapter from a book on Quality Assurance in PHP Projects.

Incorrect application functionality is a detriment to any project, but poor performance and scalability can often lead to end-user dissatisfaction and exorbitant hardware expenses.

In the case of modern web application, expectations have made it necessary to provide responses within as little as 100 to 500 milliseconds. This puts a tremendous burden on not only achieving excellent performance, but maintaining consistent performance as code changes and new features are added.

We will review a variety of tools that can be used to test performance and scalability. While the stack will be assumed to be Linux, Apache, MySQL and PHP (LAMP), the basic skills and tools can be applied to a variety of systems (and often the ability to correctly test different system components is itself a valid test case).

An initial emphasis will be on the most basic end-to-end testing of the entire request and response cycle, but later will detail smaller benchmarks and developing custom tool sets once you require more precision. Operating System metrics are also key to ensuring an application's scalability and performance, so tools such as sar, strace, and oprofile will be covered. Because performance testing has unique requirements and challenges we will cover methodologies to correctly implement performance tests, benchmarks and the pitfalls to avoid. These include the short term assurance of accuracy, reproducing results, as well as long term goals and how to balance future expectations, growth, and priorities of an entire product or service.

Brian Shire is Facebook's technical lead for PHP internals and a developer for the Alternative PHP Cache (APC).

Introduction

nospam@example.com (Sebastian Bergmann) — Thu, 05 Mar 2009 12:00:00 +0100

Stefan Priebsch and myself, Sebastian Bergmann, are writing a book on "Quality Assurance in PHP Projects". The book will be published in English and German at the same time later this year.

The idea for the book is that Stefan Priebsch and I write the introductory as well as the concluding chapters while other authors contribute case studies for the middle part of the book.

The case study authors will not always agree with each other. In fact, we invite different opinions on how problems are solved to give the readers not "one single truth" but provide different approaches to problems and views on issues.

Our motivation to write this book is simple: there is no such book on the market and from our experience a book such as this is needed. We get questions from our clients all the time: which book can I get for my developers or recommend to potential job applicants as required reading for the topic of web application quality assurance? This book will be the answer to these questions.

The list of case study authors includes contributors to well-known Open Source projects from the PHP community as well as developers working on some of the largest websites built using PHP, which happen to be some of the Web's most popular sites. Other contributing authors write about practices and tools.

Over the course of the next weeks, we will introduce each case study author together with the case study he or she is contributing to the book.