Property-based testing suggests a new way to test software, going beyond the example-based approach and stressing your code with random, realistic inputs. Follow this 3-issue mini serie about property-based testing and Kenny Baas and João Rosa speech at Codemotion Rome 2019.
Introduction to property-based testing
In the previous issue of this mini-series about property-based testing, we introduced some basic ideas from which property-based testing is derived. To sum up, property-based testing introduces a degree of fuzziness in test inputs, allowing to spot cases when the system under test behaves in unexpected way.
Let’s see a trivial but concrete case, showing how to convert check for a simple method from unit testing style to a property-based approach. Following example will use Java language (and JUnit testing framework) and QuickCheck implementation for Java/Junit.
Implementing Tests
Our mission is to write and verify a method that checks if a discount can be applied. A really naive implementation of such method and related tests can be something like the following one:
Everytime we run those unit tests, we’ll repeat the same two checks again and again and again. Moreover, we are checking how our method is behaving using two values with no meaning or reason.
First step is write a rationally meaningful check. For example, using common testing techniques such as boundary-value analysis and equivalence partitioning we can choose a set of relevant values to test. At the same time, we can use some features provided by testing framework to parametrize our test run and avoid to have one separate test method for each chosen value.
So we can rewrite our tests in the following way:
Great! It’s a step forward, but it still lacks two valuable features of the kind of testing we are aiming for. First, our tests are still performed against some pre-chosen and fixed value on every execution. Second, our tests output will we a sequence of true and false, with no particular legibility or meaningfulness about the actual behavior.
Let’s try to rewrite those checks using a property-based framework such as QuickCheck and its runner for JUnit:
JUnit QuickCheck syntax should be simple to understand: basically, we are asking to perform 500 trials (i.e. runs) of each test method, providing each time a different value randomly chosen from the given ranges.
Starting from a trivial unit test, we have built a super test that will able to warn us if out code is doing an undesired behavior when feeded with an unexpected input.
When this happens, QuickCheck provides info about the faulty input and a seed to reproduce it and debug it.
Code as Your Domain Speaks
To be honest, the actual benefit of the example provided so far is a little deceptive.
Of course, it is useful to show how to move from unit testing to property-based testing, but it lacks some prominent traits from a production code perspective.
In fact, In order to isolate the DiscountService class and simplify our test code, we deliberately took no account for any other object in the domain that could interact with our discount check. In a real world scenario, the price we check against could be the sum of prices of different items in your basket, and maybe in your business logic you have to exclude some items from discount calculation.
Moreover, we used the Java double type to represent money and we set a test range up to 49.99. This means that, for example, 49.991 can be a valid input for our method, but it will never occur as test input (if you consider, for example, fuel prices per liter in Europe, 49.991 can be a valid price that must be rounded).
In the next issue we’ll see how property-based testing can use generators feature from QuickCheck library to create random input for our own data types and domain/business logics.