Skip to main content

Testing asynchronous code

By 16 mei 2013januari 30th, 2017No Comments

Topics this post does not cover (but do involve asynchronous code):

  • parallel builds in maven
  • parallel tests using testng
  • features in hardware

What about async code?
Common cases in which a developer will use asynchronous processing are for example:

  • use parallel processing to speed up a composite task
  • scheduling or have a task repeat itself
  • when decoupling components and systems

Testing of such code can be tricky if not highly complex. First of all, the processing code itself can be simple and quick to implement, but when it comes to testing whether the code works under all conditions, thing becomes quite a different matter.

People seem to always underestimate the complexity of systems when reasoning about parallel processing. Finally, add the fact that although the final state of a test might indicate a bug free system, the way parallel processes interact always adds a level of variance which means that the final state is still no guarantee for a properly functioning system. It might fail when running the test a few more times under very slightly different conditions.

In code, a developer can expect to encounter such asynchronous code when for instance using:

  • java.util.Timer
  • ThreadPools / Threads
  • some types of remoting
  • database triggers
  • etc.

Things that require special attention when creating tests:

  • Prevent breaking other tests (and/or their setup routines)
  • Prevent multiple tests on the same unit disturbing eachother
  • Testing code that relies on sharing data between multiple processes
  • Performance testing and stress testing (how much performace can be expected? when do things start to break?)

Old scool
Here is an example of how, back in the days, someone might would write some threading code.

class MyUsefulThread extends Thread {
    //aw ... can't extend anything else anymore, meh we've got static utils right?
    public void run() {
         System.out.println("my first thread does something but nobody listening...");
new MyUsefulThread().start();

And another example, this time making it a little more flexible, but still tedious.

class MyUsefulRunnable implements Runnable {
    public void run() {
          System.out.println("better but still difficult");
new Thread(new MyUsefulRunnable()).start();

New concurrency constructs
Nowadays, things are easier and a lot more concise. Using the java.util.concurrent api, this creates a tiny threadpool and launches a task that actually returns something to the current thread (if asked for).

Executors.newFixedThreadPool(1).submit(new Callable() {
      public MyDataType call() throws Exception {
            return new MyDataType();

Here’s another example that uses scheduling. A thread pool with 5 threads, scheduling a task to run after 10 seconds.
In the mean time the code in this thread can do ‘some things’ that need to be done but can run independently.
Afterwards the future #get() is called synchronizing the result of the callable so that it can be used in the current thread.
Or if some kind of problem was found in the callable, an exception may be synchronized (a very useful thing)!
Note that a timeout is used in case the other thread never returns (or at least not within acceptable time limits).

ScheduledExecutorService schedular = Executors

ScheduledFuture scheduledAction = schedular.schedule(new Callable(), 10, SECONDS);

// (do some other stuff here while we give the world some time to evolve)

MyDataType result = scheduledAction.get(2, SECONDS); // could throw exception from thread or timeout

Awaitility is a domain specific language in java that uses a fluent api to help ease the pain of polling for test conditions.
Say that you would use #haveWeAlreadyMetTheConditions() to check certain state of your system, but you can’t be sure when this state will be reached.
You might still be able to specify the bounderies of when the state will need to be reached, between 0 and 10 seconds for instance.

Callable myTest = new Callable() {
    public Boolean call() throws Exception {
        return haveWeAlreadyMetTheConditions();
Awaitility.await().atMost(3, SECONDS).pollInterval(200, MILLISECONDS).until(callable);

This tiny library saves and lot of useless coding and eases the pain of polling state when testing asynchronous code.
Now, what does this have to do with the threading example code above? Well, if threads are used, then there is no certainty of ‘when’ a piece of code will be executed, so this often means either locking/joining and possibly wait forever, or you just start polling which is probably a lot easier to understand as well!

Some words of advice
The following seems to be clear for coding asynchronous systems:

  • There is no need anymore to touch the Thread class, leave it be
  • Let dedicated constructs/api’s/frameworks do the dirty locking/synchronizing work, they’ve already made all the mistakes before you!
  • If possible, don’t let threads share state, just use local variables (and callables & futures for instance)
  • If you think you know how locking/synchronizing works on the JVM level, then you probably don’t know enough about it yet 😉

I’ve found the following to be wise when testing asynchronous systems:

  • Try to first define clear expectations for the behavior of the system, bounderies of what is acceptable, before implementing anything
  • Don’t rely on result value end state only, this is usually no guarantee for a properly working system