This is the first instalment of a series on Property-based Testing (PBT) with Swift. In this first post, we will introduce the topic of PBT; the examples will be in Swift, but the underlying concepts are equally applicable to other languages. Later, we will make use of a library to ease the writing and execution of the tests; in such case, you will need to look for a similar tool in your preferred language. I have taken inspiration from this series of posts in F# when writing this. Let’s get started!
You may be familiar with Example-based Testing. These are the tests that most of us start writing. For instance, let’s assume that we are pairing in a TDD session to build a function to concatenate two strings. We can write the following test:
This is a test written following Example-based Testing since we are providing examples of how the function should behave. After writing this test, we hand our keyboard to our pair and she writes code to make it pass:
You may think this implementation is wrong, but actually it isn’t. According to TDD rules, we should write the simplest thing that can possibly work, and later refactor if necessary. Our pair returns the keyboard; it seems we will need to write something more to force her write the right implementation:
In response, our partner writes this implementation:
Again, you may think this implementation is wrong, but it is not; this is the result of applying the Transformation Priority Premise. It seems we will have to work harder on the tests. Let’s create a bunch of examples and run the test on all of them:
We have created triples of values where the third component is the expected result of concatenating the first two. Then, we hand the keyboard to our partner and…
At this point I can agree with you that the implementation doesn’t seem to be the most correct one. This case seems obvious to us given its simplicity, but perhaps our partner is not able to see the repeating pattern in all these examples. Moreover, to the best of her knowledge, this implementation is valid for the whole universe of cases provided by the tests, so it must be correct!
Can you spot the problem? There is a relationship between the examples provided in the tests and the implementation. If we could hide the actual examples, then our partner would be forced to write a correct implementation. Assuming we have a function to generate random strings, we can write the following test:
And we repeat it multiple times to make sure that, by chance, our partner doesn’t hardcode the solution. However, we are leaking an implementation detail by using + in our test, so this test is wrong. Can we try a different approach?
Instead of using examples, we can use something that holds true for every two strings that we concatenate. For instance, if we concatenate two strings, its length is equal to the sum of the length of the two arguments. This is what we call a property. We can write a test like this:
In this case, our partner cannot hardcode any data and we are not leaking implementation details into the test (note that I have added a extension field to
String to get its length). What other properties does string concatenation have? For instance, if I concatenate three strings
c, I can do
(concat(concat(a, b), c) or
concat(a, concat(b, c)) and the result is the same. This property is called the associative property and can be tested like this:
And also, if we concatenate the empty string to any other string, we get the same string as a result; this is the neutral element property:
The tests written above have some common structure that fits in the following logical proposition: for all
x in a domain,
p(x) is true, being
p a property about
x. Since our tests need to be finite, we have simulated the universal quantifier (for all) by generating 100 random instances, assuming that, if the implementation is wrong, some of them will eventually fail.
This type of testing forces us to focus on the specification, rather than on concrete examples. It makes you think in a more abstract way about what characterizes a solution, instead of how it is implemented. I don’t mean you should drop Example-based Testing, since it is a very good tool for exploration and checking specific corner cases.
I hope you enjoyed the post and feel free to contribute on the comments if you have any question or feedback!
Tomás Ruiz-López DEVELOPMENT
swift iOS testing property based testing functional programming