For a project I wanted to add strict HTML 5 syntax checking. It is not something that is easy to do in the build since in general it requires the web application to be deployed. It is a simple Spring MVC project with Velocity templates so it is easy to make HTML mistakes, and verifying partial templates doesn’t really help.
We use Jenkins for most of our continuous integration setups and in this case the application is automatically deployed onto a publicly accessible web environment. After some googling I found a simple little plugin called ‘Jenkins Unicorn Validation Plugin’ which uses W3C’s Unicorn validation service. This plugin did what I needed: you give it a url and it verifies both HTML and CSS.
And of course, after installing the plugin in Jenkins, configuring it as a post-build step it worked. It immediately broke the build since we are using Bootstrap as a layout template and of course in the real world, CSS hacks are needed to get stuff done. Out of the box Bootstrap appears to have 54 errors and 152 warnings. Luckily you can configure the allowed number of errors and warnings before it fails the build. I added a few more validation steps, for most crucial and prototypical pages and it worked.
However, I dislike having these random number of errors in the build. Also, the primary reason for doing this validation is for SEO purposes: we needed the HTML to be correct, the CSS is also important but since you need hacks there anyway, validation does not seem to be the holy grail. Mixing CSS and HTML errors in a single field is not very reliable. Unfortunately, while the W3C validator allows you to specify what you want validated (e.g. only HTML), the plugin has no support for it. It is mentioned on the TODO list somewhere. Since the plugin is open source, that shouldn’t pose much of a problem.
The code is easy to check out:
git clone https://github.com/jenkinsci/unicorn-plugin.git
The online documentation of Jenkins and this plugin is minimalistic, to say the least. Apparently Jenkins (and thus the unicorn plugin as well) uses something called ‘Jelly’ for XML programming and templating. The whole concept of programming in XML is as horrible as the documentation of this project. So I decided to stick to the bare minimum I needed for my purpose and not to waste too many brain cells on it. The bare minimum as I defined it, was just compact enough to put in a custom url, instead of the default url that Unicorn uses for validation. On the website of the Unicorn validator, I could simply put in the page to verify, set the correct options and then copy-paste the url into the plugin configuration. Unfortunately, the plugin did not support this; it constructed the url itself, so there was some hacking to be done there. I decided to simply try and add a new field customUrl which, if filled in, was to be used. The existing logic would be an alternate configuration if the customUrl was not specified.
The plugin is simple enough, it has three classes:
- UnicornValidationBuilder, which basically is the plugin management
- UnicornValidation, which handles the validation call and parsing the result
- Observer, which is simply a value object for a result of the validation containing the error and warning counts
There are also some Jelly templates. Most of them are empty, only the config.jelly had some content. This, as the name suggests, is the template for configuring the plugin.
Since there was not much of documentation on how it is supposed to work, I used the tried and tested method of copy-paste to simply duplicate the ‘unicornUrl’ existing field to my new ‘customUrl’ field. I added it on top and tried to compensate the lack of user experience by changing the labels a bit. (As I said before, I was aiming for bare minimum. Fit for purpose as they say in project management.)
The field attributes suggest that this is a link to a sort of backing bean, and sure enough, it is there in UnicornValidationBuilder. Apparently it is passed along in the constructor so I added it there as well. There was actually a nice hint in the form of a comment which showed I was on the right track.
// Fields in config.jelly must match the parameter names in the "DataBoundConstructor"
@DataBoundConstructor
public UnicornValidationBuilder( String customUrl, String unicornUrl, String siteUrl,
String maxErrorsForStable, String maxWarningsForStable,
String maxErrorsForUnstable, String maxWarningsForUnstable) {
this.customUrl = customUrl;
this.unicornUrl = unicornUrl;
this.siteUrl = siteUrl;
this.maxErrorsForStable = maxErrorsForStable;
this.maxWarningsForStable = maxWarningsForStable;
this.maxErrorsForUnstable = maxErrorsForUnstable;
this.maxWarningsForUnstable = maxWarningsForUnstable;
}
It turned out that although the field was there, it wasn’t saved. After some more digging, I found out the new field also needed a getter (duh, makes sense). Sure enough, after adding the getter, it magically worked.
Now to implement the functionality. The idea was that the plugin would have two modes of operation: using the customUrl or falling back on the default configuration, if no customUrl was supplied.
In the Builder, I added a function to represent this:
private boolean useDefaultUrl() {
return customUrl==null || customUrl.trim().isEmpty();
}
There is also a perform(...) function which seems to be the entry point for the plugin. It configured the UnicornValidator class with the default url’s, so it seemed a logical place to hack my customUrl in:
// unicorn validation
unicornValidation = new UnicornValidation();
if (useDefaultUrl()) {
unicornValidation.setUnicornUrl(unicornUrl);
unicornValidation.setSiteUrl(siteUrl);
} else {
unicornValidation.setCustomUrl(customUrl);
}
Extending the UnicornValidation class was fairly trivial as well. Aside from adding the new field, I had to refactor the callUnicornService() to actually use the new field:
public void callUnicornService() throws IOException {
if (useDefaultUrl()) {
unicornDoc = Jsoup.connect(unicornUrl + "check").data("ucn_uri", siteUrl).data("ucn_task", UNICORN_TASK).userAgent(CONNECT_USERAGENT)
.cookie("auth", "token").timeout(CONNECT_TIMEOUT).get();
} else {
unicornDoc = Jsoup.connect(customUrl).userAgent(CONNECT_USERAGENT).cookie("auth", "token").timeout(CONNECT_TIMEOUT).get();
}
outputString = unicornDoc.toString();
}
private boolean useDefaultUrl() {
return customUrl == null;
}
With this, the plugin worked as expected.
Final note:
Am I happy with this change? It is hard to say: I spend a little time on making our build more robust and having it validate exactly what I want. So that is good, better than a workaround mixing CSS and HTML errors. So from a project manager point of view it is fit for purpose. It does exactly what I need and nothing more.
But from a developer point of view I feel dirty. The code isn’t pretty: the decision on which validation should be used is implemented in two spots, the configuration user interface doesn’t make clear how to use it and I did not help any future developers by cleaning up and documenting some things.
I actually could have improved the user interface with e.g. option boxes and some nice labels. But googling for some documentation or hints did not turn up anything helpful. And spending a lot of time learning Jelly and Jenkins internals was not really part of ‘fit for purpose’. And cleaning up and refactoring the code would also require some direction to refactor it to. I hit some form of the broken window syndrome: there was already some mess so a bit more shouldn’t hurt, right?
So in the end, we could say: documentation isn’t always necessary to get things done. But it sure helps for doing things the proper way.
See also:
