Yoav Landman elaborated on the efforts on implementing parallel file downloads with stop, start and resume functionality.
The UrlConnection
did not facilitate enough for their requirements. While researching their options, they concluded that they have to use the reactor design pattern (see image).
When saying reactor, NIO comes to mind. Becaus handling reactor events is complicated, they tried a couple of libaries (they didn’t want to use a server like netty, grizzly, etc.) among which are Apache Mina, Ning Http async client and Apache Http async client.
They chose to use a single part file with a range to write to for each parallel download. A java.nio.channels.FileChannel
can be used to do this (Random access is not threadsafe). When implementing parallel downloads you have to take care of the file parts to prevent data corruption. You have to use VM file locking and OS file locking to guarantee that the data does not become corrupt. Furthermore, when using a single file, you have to use a shared lock. Windows only allows exclusive locks, but you can use a workaround for that.