An excellent talk about A/B tests from someone who knows - Martin Goodson. My favourite part is an A/B test that found a 2.5% improvement to (sales) conversions between the two versions of the software being tested. Unfortunately there was a bug in the A/B testing framework, such that the old version was being tested … Continue reading How to mess up A/B tests
Category: Testing
Taming Combinatorial Explosions in SpecFlow
The Problem Imagine you are using SpecFlow to test a REST API. In the common REST way, the URL is built up of 1+ units, where each unit is one of these: Give me a list of X; Give me the instance of X that has id Y. You could easily have up to 3 … Continue reading Taming Combinatorial Explosions in SpecFlow
Make your test data more vicious
Imagine you have some code that works with data from a table that stores customer orders. Unfortunately your code has a bug - instead of using CustomerId for something it uses OrderId instead (both are fields on the table). What's worse, your tests pass and the test coverage looks fine. What's up? I'll assume that … Continue reading Make your test data more vicious
Here be dragons: testing your error handling code
Who tests the error handling parts of their code? You might want to start doing this after watching this very interesting video from Goto Conference 2016. Among other things, the speaker summarises a paper that investigates catastrophic failures in things like MapReduce, Cassandra etc. 58% of the catastrophic failures could have been prevented by testing … Continue reading Here be dragons: testing your error handling code
Gluing together the bits of your SpecFlow test
In my previous post on SpecFlow, I showed how a single test case (Scenario in SpecFlow-speak) maps to many different bits of C# that will actually implement it. One scenario will often map to 3 different bits - for the Given, When and Then respectively - but if you use And to extend any of … Continue reading Gluing together the bits of your SpecFlow test
An introduction to Behaviour-Driven Testing with Cucumber / SpecFlow
BDD (Behaviour-Driven Development) evolved out of TDD (Test-Driven Development), and is an approach to testing and development in general. I have found it has benefits even if you just use it as a way of doing testing, rather than having tests drive what you develop. There are a few different frameworks that allow you to … Continue reading An introduction to Behaviour-Driven Testing with Cucumber / SpecFlow
Random Tech Thoughts 