Fifty Quick Ideas To Improve Your Tests now available

My new book, Fifty Quick Ideas to Improve Your Tests, is now available on Amazon. Grab it at 50% discount before Friday:

This book is for cross-functional teams working in an iterative delivery environment, planning with user stories and testing frequently changing software under tough time pressure. This book will help you test your software better, easier and faster. Many of these ideas also help teams engage their business stakeholders better in defining key expectations and improve the quality of their software products.

For more info, check out FiftyQuickIdeas.com

To improve testing, snoop on the competition

This is an excerpt from my upcoming book, Fifty Quick Ideas To Improve Your Tests

snoop on competitionAs a general rule, teams focus the majority of testing activities on their zone of control, on the modules they develop, or the software that they are directly delivering. But it’s just as irresponsible not to consider competition when planning testing as it is in the management of product development in general, whether the field is software or consumer electronics.

Software products that are unique are very rare, and it’s likely that someone else is working on something similar to the product or project that you are involved with at the moment. Although the products might be built using different technical platforms and address different segments, key usage scenarios probably translate well across teams and products, as do the key risks and major things that can go wrong.

When planning your testing activities, look at the competition for inspiration — the cheapest mistakes to fix are the ones already made by other people. Although it might seem logical that people won’t openly disclose information about their mistakes, it’s actually quite easy to get this data if you know where to look. Continue reading

Explore capabilities, not features

This is an excerpt from my upcoming book, Fifty Quick Ideas To Improve Your Tests

Exploratory testing requires a clear mission. The mission statement provides focus and enables teams to triage what is important and what is out of scope. A clear mission prevents exploratory testing sessions turning into unstructured playing with the system. As software features are implemented, and user stories get ready for exploratory testing, it’s only logical to set the mission for exploratory testing sessions around new stories or changed features. Although it might sound counter-intuitive, story oriented missions lead to tunnel-vision and prevent teams from getting the most out of their testing sessions. Continue reading

How to get the most out of Given-When-Then

This is an excerpt from my upcoming book, Fifty Quick Ideas To Improve Your Tests

Behaviour-driven development is becoming increasingly popular over the last few years, and with it the Given-When-Then format for examples. In many ways, Given-When-Then seems as the de-facto standard for expressing functional checks using examples. Introduced by JBehave in 2003, this structure was intended to support conversations between teams and business stakeholders, but also lead those discussions towards a conclusion that would be easy to automate as a test.

Given-When-Then statements are great because they are easy to capture on whiteboards and flipcharts, but also easy to transfer to electronic documents, including plain text files and wiki pages. In addition, there are automation tools for all popular application platforms today that support tests specified as Given-When-Then.

On the other hand, Given-When-Then is a very sharp tool and unless handled properly, it can hurt badly. Without understanding the true purpose of that way of capturing expectations, many teams out there just create tests that are too long, too difficult to maintain, and almost impossible to understand. Here is a typical example:

    Scenario: Payroll salary calculations

    Given the admin page is open
    When the user types John into the 'employee name'
    and the user types 30000 into the 'salary'
    and the user clicks 'Add'
    Then the page reloads
    And the user types Mike into the 'employee name'
    and the user types 40000 into the 'salary'
    and the user clicks 'Add'
    When the user selects 'Payslips'
    And the user selects employee number 1
    Then the user clicks on 'View'
    When the user selects 'Info'
    Then the 'salary' shows 29000
    Then the user clicks 'Edit'
    and the user types 40000 into the 'salary'
    When the user clicks on 'View'
    And the 'salary' shows 31000

This example might have been clear to the person who first wrote it, but it’s purpose is unclear – what is it really testing? Is the salary a parameter of the test, or is it an expected outcome? If one of the bottom steps of this scenario fails, it will be very difficult to understand the exact cause of the problem.

Spoken language is ambiguous, and it’s perfectly OK to say ‘Given an employee has a salary …, When the tax deduction is…, then the employee gets a payslip and the payslip shows …’. It’s also OK to say ‘When an employee has a salary …, Given the tax deduction is …’ or ‘Given an employee … and the tax deduction … then the payslip …’. All those combinations mean the same thing, and they will be easily understood within the wider context.

But there is only one right way to describe those conditions with Given-When-Then if you want to get the most out of it from the perspective of long-term test maintenance.

The sequence is important. ‘Given’ comes before ‘When’, and ‘When’ comes before ‘Then’. Those clauses should not be mixed. All parameters should be specified with ‘Given’ clauses, the action under test should be specified with the ‘When’ clause, and all expected outcomes should be listed with ‘Then’ clauses. Each scenario should ideally have only one When clause, that clearly points to the purpose of the test.

Given-When-Then is not just an automation-friendly way of describing expectations, it’s a structural pattern for designing clear specifications. It’s been around for quite a while under different names. When use cases were popular, it was known as Preconditions-Trigger-Postconditions. In unit testing, it’s known as Arrange-Act-Assert.

Key benefits

Using Given-When-Then in sequence is a great reminder for several great test design ideas. It suggests that pre-conditions and post-conditions need to be identified and separated. It suggests that the purpose of the test should be clearly communicated, and that each scenario should check one and only one thing. When there is only one action under test, people are forced to look beyond the mechanics of test execution and really identify a clear purpose.

When used correctly, Given-When-Then helps teams design specifications and checks that are easy to understand and maintain. As tests will be focused on one particular action, they will be less brittle and easier to diagnose and troubleshoot. When the parameters and expectations are clearly separated, it’s easier to evaluate if we need to add more examples, and discover missing cases.

How to make it work

A good trick, that prevents most of accidental misuse of Given-When-Then, is to use past tense for ‘Given’ clauses, present tense for ‘When’ and future tense for ‘Then’. This makes it clear that ‘Given’ statements are preconditions and parameters, and that ‘Then’ statements are postconditions and expectations.

Make ‘Given’ and ‘Then’ passive – they should describe values rather than actions. Make sure ‘When’ is active – it should describe the action under test.

Try having only one ‘When’ statement for each scenario.