SpecFlow + Selenium: The engineering behind decent Gherkin files

This article is in a series about Selenium and SpecFlow

1. Introduction
2. Why bother?
3. Basic plumbing
4. Page objects
5. The engineering behind decent Gherkin files

UPDATE: please see another article I’ve written on the costs and benefits of the approach I describe below.

Summary

If you are writing the implementation of your SpecFlow + Selenium tests, you will probably make your life easier in the long run if you follow this design principle: collect information as you are given it, and delay using it for as long as possible.

Introduction

SpecFlow allows you to build two or more levels of abstraction in your tests – at least Gherkin (the test scenarios) vs. the implementation.  However, if you’re not careful, the implementation details leak upwards and add unnecessary complications to the Gherkin.  This article shows how you can implement the tests in a way that stops this leaky abstraction problem.

If you look at a test scenario, its three sections have the following jobs:

• Given – get the world into the correct starting state;
• When – describe the action that moves the world from the starting state to a next state;
• Then – describe what you expect the next state to look like.

For simplicity, I will limit discussion to the Given part, although these principles could apply to the other two as well.  The Given part defines everything about the starting state for the test (ignoring the Background section for simplicity).  When testing a web site, this includes:

• What kind of user you are;
• What actions you or other relevant people have already taken before the test starts;
• What page you are on at the start of the test (this is a kind of action you’ve already taken, but is probably different and special enough to think about separately).

An example to show the problem

If you are testing an online HR system that tracks when people are working or not, an example of a single scenario might be:

Given I am logged in as a supervisor
And my company requires leave requests to be approved
And a member of my team has submitted a leave request
When I am on the team’s requests page
Then I can see the pending leave request

Note that this test is suitable to be run against a production system.  As long as there is a test company with suitable accounts, leave requests etc. set up then this test should pass.  In fact, you might have a number of test companies set up in your production system with things set up ready to test different things.  For instance, needing requests to be approved vs. not needing them to be approved, a leave request has been submitted vs. none has been submitted etc.

So, in such a situation you would need to pick the correct company for the test scenario, and within that pick the correct kind of user (e.g. supervisor vs. admin or normal user).  Once you have worked out which company and which kind of user, coupled with which environment the test is running against (production, test, developer’s machine etc.) then you can look up which credentials to use when you log in.

This means that you will have the information you need to login once you get to the end of the Given section above.  However, the Given line with the words logged in is the first one in the Given section.

There are at least three ways to tackle this problem.  I will describe the three I know, and why I think that two have problems.

Solutions with problems

The first way is to force the user to write the lines in the Given section in an order that delivers information before it’s needed.  In this example, the line with logged in must be after the other two.  This works, but it’s the leaky abstraction mentioned above.  Why should the Gherkin author care?  Even if they do care, they might forget.  In the words of Paul Boag: don’t make your problems into your users’ problems.

The second way is to merge all the separate lines into one long line, so that the information is all dealt with by a single step definition.  In some ways this could just be the first way in a slightly different form.  The author might write

Given I am logged in as a supervisor for a company that needs requests to be approved and a team member has submitted a request

This is painfully long and unnecessarily hard to understand.  Also, there’s nothing to stop them trying to put the different parts in a different order, other than the step definition’s regular expression won’t match.  (They would have to remember a magic order again, but this time for parts of a line rather than whole lines.)

You might solve this by using a table, to force the order of the parts of the one long line.  This is a bit better, but there is still another problem.  You still have to write one step definition that deals with the trio of user type, company type and the number of pending requests.  If you want to have another test for a supervisor, where it doesn’t matter what the company type is or how many pending requests there are, then you can’t re-use that step definition as the regular expression won’t match

It is much more flexible and useful if you have one step that deals with just user type, that you can re-use in all scenarios that need it.  (And so on for all the other steps / aspects of your test.)  You write a set of single-purpose steps, and then different test scenarios will compose whichever subsets they need.

My preferred solution

The third approach allows Given lines to be in any order, with each aspect of the starting state having its own line.

A summary is: collect information as you are given it, and delay using it for as long as possible.  So, during the three Given lines above, you collect the user type and the details of the company and store them away.  This could be on one or more context objects or wherever else makes sense.  At the end of the Given block you are now in a position to pick the credentials and log in.

It would be nice to do this via a hook that runs at the end of the Given block.  Unfortunately this doesn’t exist, but there is a near miss: a hook that runs at the end of each block – AfterScenarioBlock.  In this you can test the type of the current block, and if it’s Given then call code that sweeps up the information from the contexts and logs in:

if (ScenarioContext.Current.CurrentScenarioBlock == ScenarioBlock.Given) { GetInfoAndLogIn(...) }

This approach is similar to one I’ve used when using SpecFlow to test a Web API.  Often the easiest way to set all the various parts of the HTTP request (using a mixture of defaults and overrides specific to the test scenario) is to use some kind of builder object:

1. You make sure that the builder exists at the start of the test scenario;
2. As the steps match lines from the test scenario, they extract information from the lines and give it to the builder;
3. When the HTTP request is actually needed, the relevant step or a hook will ask the builder for the request, based on all the information collected from the steps and any defaults built into the builder.

Going further

You could extend this approach if you wanted to include the starting page in the Given section rather than in the When section (because the When is doing something else).  The code called by the AfterScenarioBlock can login and then go to the required starting page.

What if you wanted to go even further?  Imagine that you want to test an operation where the user needs to do three things, and you want to test what happens if they try to miss out one of them.  For instance, to submit a leave request you need to select a date and a number of days and then click a Submit button.  What if you try to click Submit without first selecting a date?  (There are several issues here I’m going to skip for clarity – e.g. you would be changing the state of the system and so this might not be one to run against production.)

A test scenario that describes this might look like this:

Given I am logged in as a normal user
And I am on the leave requests page
And I have selected 1 day
When I submit the leave request
Then I get an error about leave requests needing a date

The page object for the leave requests page can be created on the second Given line (the one that mentions which page is needed), but the browser won’t go to that page until the hook that runs after the end of the Given section.  Therefore, the step that matches Given I have selected 1 day can’t actually click or type things to select 1 day.

What it can do is use a delegate to add all a call of the relevant method in the page object to a list to do later.  This list can be cleared (by executing the delegates) wherever makes most sense.  For instance, the hook at the end of the Given block, or in the step definition that matches When I submit the leave request.

Adding the delegate would look something like this:

[Given(@"I have selected (\d+) days?")]
public void GivenIHaveSelectedNDays(int numDays)
{
     DeferredOperations.Add(() => leaveRequestPageObject.SelectDays(numDays));
}

where DeferredOperations is a list of delegates declared elsewhere e.g. earlier in the step definition file.

This is another example on the recurring theme: collect information as you are given it, and delay using it for as long as possible.