Skip to main content
Blog

Generating test data with junit quickcheck

This post will demonstrate generating test data using junit-quickcheck. It will focus on practical examples, rather than the theory behind property based testing.

JUnit-quickcheck has its own JunitRunner, JUnitQuickcheck, which must be set on the class containing the test using an annotation, @RunWith(JUnitQuickcheck.class). With this annotation in place, the methods of the class annotated with @Property are executed. These @Property annotated methods are much like @Test annotated methods except for two things:

  1. The methods are called, by default, a 100 times
  2. @Property annotated methods can have parameters, which will be provided by junit-quickcheck using random values

Example system-under-testA lot of data

The system-under-test is a rest service, with one endpoint serving 3 books. The first few tests will target a domain model and a resource to be transformed into JSON. That resource, being part of a port in hexagonal architecture, has methods to transform from and to the domain model.

Build in data types

Junit-quickcheck can, without any further configuration or code, provide random values for a number of build-in datatypes.

    /**
     * Example using build types, which junit-quickcheck can generate.
     * 
     * @param title generated by junit-quickcheck
     * @param author generated by junit-quickcheck
     * @param publisher generated by junit-quickcheck
     * @param publishedOn generated by junit-quickcheck
     */
    @Property
    public void testConversions(//
            final String title, //
            final String author, //
            final String publisher, //
            final LocalDate publishedOn) {

        // Given a book
        final Book book = new Book(title, new Author(author),
                new Publisher(publisher), publishedOn);

        // When we create a resource based on that book
        final BookResource resource = BookResource.fromBook(book);
        // And convert the resource back to a book
        final Book converted = resource.toBook();

        // Then we expected the result to have the same property values as the
        // original book
        assertThat(converted, SamePropertyValuesAs.samePropertyValuesAs(book));
    }

In this example, values of type String and LocalDate (parameters title, author, publisher and publishedOn) are provided by junit-quickcheck. This test is run 100 times with random values.

User defined datatypes

Using junit-quickcheck we can also inject values of user defined types, but we have to specify how it creates them. We have the option to use setters, with the @From(Fields.class) annotation, or to use the constructor with @From(Ctor.class).

    /**
     * Example using a user defined type, of which all constructor parameters are
     * build-in types.
     * 
     * @param resource generated by junit-quickcheck
     */
    @Property
    public void testConversions(// Given a resource
            final @From(Ctor.class) BookResource resource) {

        // When we convert the resource into a book
        final Book book = resource.toBook();
        // And we convert the book back to a resource
        final BookResource converted = BookResource.fromBook(book);

        // Then we expect the result to have the same property values as
        // the original resource
        assertThat(converted,
                SamePropertyValuesAs.samePropertyValuesAs(resource));
    }

Here we let junit-quickcheck generate BookResource objects. If we were to any field to both Book and BookResource, and change the toBook and fromBook method accordingly, this test would still pass.

With @From(Ctor.class) we specify that junit-quickcheck should use the constructor. Unfortunately, we can only use constructors if all arguments are build-in types. In this case, BookResource has only String and a LocalDate as parameters. But we cannot generate a Book this way.

Custom generator

To let junit-quickcheck generate Book objects, which has parameters of user defined types, we must define and use a Generator.

    /**
     * Because book has a constructor with user defined data types (Author and
     * Publisher), the Ctor.class generator will not work.
     * 
     * Therefore, we need to create a generator, to generate a book.
     */
    public static final class BookGenerator extends Generator {
        /**
         * Generator to generate strings for title, author name and publisher
         * name.
         */
        private final StringGenerator stringGenerator = new StringGenerator();
        /**
         * Generator to generate dates.
         */
        private final LocalDateGenerator localDateGenerator =
                new LocalDateGenerator();

        public BookGenerator() {
            super(Book.class);
        }

        @Override
        public Book generate(final SourceOfRandomness random,
                final GenerationStatus status) {
            final String title = stringGenerator.generate(random, status);
            final String author = stringGenerator.generate(random, status);
            final String publisher = stringGenerator.generate(random, status);
            final LocalDate publishedOn =
                    localDateGenerator.generate(random, status);
            final Book book = new Book(title, new Author(author),
                    new Publisher(publisher), publishedOn);
            return book;
        }
    }

    /**
     * Example using the BookGenerator defined above, to generate books
     * 
     * @param book generated by junit-quickcheck
     */
    @Property
    public void testConversions(// Given a book
            final @From(BookGenerator.class) Book book) {

        // When we convert the book to a resource
        final BookResource resource = BookResource.fromBook(book);
        // and we convert the resource back to a book
        final Book converted = resource.toBook();

        // Then we expect the result to have the same property values as the
        // original book
        assertThat(converted, SamePropertyValuesAs.samePropertyValuesAs(book));
    }

In order to generate values of user defined types, with constructor parameters featuring even more user defined types, we must create our own Generator. Then, we reference this generator in @From annotation. This generator is, unfortunately, tightly coupled to the Book class. As a result, any new parameters in the constructor of Book will break this generator.

Mocking and collections

In the following tests target a Controller serving JSON documents. It depends on BookService to find Book objects, transforms those to BookResource objects en returns them.

    @Mock
    private BookService bookService;

    @InjectMocks
    private BookController bookController;

    @Rule
    public MockitoRule mockitoRule = MockitoJUnit.rule();

Above, we set up mocking as usual.

    /**
     * Tests the book controller against a mock book service, using
     * junit-quickcheck to generate test data.
     * 
     * @param resources generated by junit-quickcheck
     */
    @Property()
    public void testBasedOnResources(//
            final List<@From(Ctor.class) BookResource> resources) {

        // Given an mocked bookservice, which returns generated data when
        // findAll() is called.
        when(bookService.findAll()).thenReturn(resources.stream()
                .map(BookResource::toBook).collect(Collectors.toList()));

        // When we call books() on the book controller
        final List results = bookController.books();

        // We expect the book controller to return book resources which match
        // the books
        Streams //
                .zip(resources.stream(), results.stream(), Pair::of)//
                .forEach(pair -> {
                    final BookResource reference = pair.getLeft();
                    final BookResource result = pair.getRight();
                    assertThat( //
                            reference, //
                            SamePropertyValuesAs.samePropertyValuesAs(result));
                });
    }

Here we let junit-quickcheck generate lists of BookResource objects. Note the @From(Ctor.class) annotation binds to the BookResource, the type parameter of List and not List itself.

Conclusion

Using junit-quickcheck we can inject random values in our tests. This frees us from the burden of hard coding test data. Moreover, it generates more data than we would hard code. And, if we manage to avoid generators, our tests will not be tightly coupled to a data model.

Full source of the examples: https://github.com/First8/junit-quickcheck-demo

More information

Junit-quickcheck is a QuickCheck implementation for java, specifically targeting JUnit. QuickCheck is a property based testing library written in Haskell. It is used to check properties of functions, by applying those functions on generated input and checking that a property holds. http://hackage.haskell.org/package/QuickCheck

Generated values are by default completely random. However, using the @Seed annotation, the generated values become pseudo random. Thus, using a @Seed, our test become deterministic and repeatable once more. Please see https://pholser.github.io/junit-quickcheck/site/0.8/usage/seed.html for more details.

Junit-quickcheck can help find the smallest possible test case that fails: https://pholser.github.io/junit-quickcheck/site/0.8/usage/shrinking.html