Jenkins shared libraries: tested

Jenkins is a very neat tool to implement a continuous delivery process, mainly due to its flexibility. Sometimes it can be hard though to keep complexity low, and when that happens, (automated) tests become far more important.

Jenkins should in fact be running tests that verify the scripts running tests, proving they actually work. Warning: this dog will chase its own tail.

 

Jenkins

Jenkins pipeline scripts

Once, not so long ago, the way to go was to manage jobs manually using the GUI. At the moment Cloudbees is proposing pipelines as scripts, either scripted procedurally or declarative.

Declarative pipelines are still in development. Their primary intent is to be suitable to be edited from a simple GUI.

 

Extending Jenkins scripting

In the world of open source, integration is becoming the only major thing a company needs to implement themselves.

Using just a single script however will get you only so far. At a certain point you will want to introduce abstractions.

 

The Jenkins Shared library

Cloudbees recently introduced a way of having more than just a script, but keeping things relatively easy and secure to use.

The ‘shared library’ is a Jenkins concept that allows projects to declaratively import a full repository of classes and scripts.

In short, any shared library will have a few common folders:

 

It comes with tests

As with all new areas of applying code, once maturity is reached, tests become relevant. Fortunately there is a way to run tests for the pipeline code, be it scripts or (typed) classes.

These tests will lean towards unit tests even though the unit boundaries for pipelines are not that clear. Interaction with external systems is of course not something best covered using unit tests. The following examples are aimed primarily at testing groovy compilation, execution and (emulated) interpretation by Jenkins. When defining clear units (classes), these can become proper targets for unit tests once more.

 

JUnit example

Note that the code is groovy, but it can just be java, just add semi colons 😉 (okay maybe a few anonymous classes etc, but should be possible).

Spock example

Since the pipeline scripts use groovy, one might as well write tests using groovy.

 

Project layout

To be able to work with a shared library, a project setup would be convenient, so why not use groovy through gradle? The layout of a Jenkins shared library is not standard for gradle. This means that a little config is needed.

Once the configuration is in place, the project will build quite easily. There are some snags though, which will show in the gradle config file below. The little problems together make the build.gradle file not quite as trivial.

 

 

Gradle config

Check the following build.gradle config, there are hints on possible difficulties in the comments.

Running the tests

Since gradle can be started using the brilliant gradle wrapper, starting the build from Jenkins is easy.

It’s enough to create a typical ‘multi-branch job’ and have a Jenkinsfile with the following:

Now each push to the shared library repository will start a test run producing early warnings if something is wrong.

 

More info

 

Chasing tails

The recently developed option to test pipeline code is great. As with all testing options though it’s important to keep in mind whether the extra testing effort is worth it.

Having Jenkins run tests on test code that runs tests seems kind of pointless, after all tests will (hopefully) fail both when the production code is broken and when the tests are broken. It turns out that having some trivial unit tests for pipeline code does deliver a lot of value by creating a very short feedback loop. In terms of knowing whether the pipeline actually does the ‘right’ thing is a different matter. Tests that can assert this are far more complicated, true system tests are probably more suitable for that purpose.

 

Some things to think about:

  • what is continuous passing style?
  • what would be a convenient strategy to actually integration-/system-test new pipeline code?