Skip to main content
Blog

JavaOne 2014 – Day 4 – Everything You Wanted to Know About Writing Async, Concurrent HTTP Apps in Java

By 3 oktober 2014januari 30th, 2017No Comments

Yoav Landman elaborated on the efforts on implementing parallel file downloads with stop, start and resume functionality.

The UrlConnection did not facilitate enough for their requirements. While researching their options, they concluded that they have to use the reactor design pattern (see image).

reactor-pattern2

When saying reactor, NIO comes to mind. Becaus handling reactor events is complicated, they tried a couple of libaries (they didn’t want to use a server like netty, grizzly, etc.) among which are Apache Mina, Ning Http async client and Apache Http async client.

Ning and Apache are similar in functionality. To postpone the decision they made an abstraction layer to be able to switch between libraries. In the end they chose the Apache library (because it has better performance).
The HTTP 1.1 spec specifies that at most 2 clients can be served in parallel and in the case of file downloads that would result in a maximum of two parallel file parts. Initially all browsers forced this behavior, but nowadays this is not a hard requirement for all regular browsers. Another problem they encountered in the HTTP spec is that the content-length in combination with compression is not clear.
In the HTTP 2.0 spec the above problems have been fixed (multiplexing is used to fix the parallel issue). Next to this also header compression and pushing to a client is covered in the new spec.

They chose to use a single part file with a range to write to for each parallel download. A java.nio.channels.FileChannel can be used to do this (Random access is not threadsafe). When implementing parallel downloads you have to take care of the file parts to prevent data corruption. You have to use VM file locking and OS file locking to guarantee that the data does not become corrupt. Furthermore, when using a single file, you have to use a shared lock. Windows only allows exclusive locks, but you can use a workaround for that.