In the previous blog post we talked about vertical scaling with Open Shift. This blog posts investigates the next step: horizontal scaling.
At some point we’ll run out of bigger sizes gear. And separating out chunks of an application to different gears doesn’t always help. To access a separated part of the application (e.g. the database), you most likely need to go over a network and that is many orders of magnitude slower than a simple in-JVM Java function call. So vertical scaling has its limits.
The next thing we could do is use multiple gears for the same part of the application. E.g., run the web application part on more than one gear. Of course, for this your web application has to be functionally able to run on more than one server. Ideally, your web application is stateless. If that isn’t the case you’ll have to make sure that you have some mechanism in place to synchronise state between the different instances. This is a topic for a different blog post so I’ll simply assume a stateless web application for now.
If you created your application marked as scalable, two things changed in the configuration. First, as mentioned in the previous vertical scaling post, any database cartridge that will be added will be deployed on a separate machine. The second thing that happened is that HAProxy is installed in front of your web application. This high availability proxy runs on the same gear as your web application (until you scale above a certain number) and functions as a load balancer. Traffic no longer goes directly to your Tomcat but passes through HAProxy first. This HAProxy then can pick any Tomcat instance to handle the traffic.
Since your Tomcat cartridge is now considered scalable, OpenShift can use multiple gears for it. By default, OpenShift will automatically add and remove gears. To get a bit more grip on how it works, let’s first disable that and do it manually.
To disable autoscaling, you can simply place a marker file called ‘disable_auto_scaling‘ in your checkouts marker directory: mywebsite/.openshift/markers. (Don’t forget to commit and push!).
If you just started out with a scalable application and a database, the deployment would probably look something like this:
There are two gears in use, one for the database and one for the application server, in this case Tomcat. There is also a HAProxy in front of Tomcat, colocated with that Tomcat. You can verify this with the rhc command tools using rhc app show:
$ rhc app show mywebsite
mywebsite @ http://mywebsite-first8.rhcloud.com/ (uuid: 554742be4382ec93270000d6)
Created: May 04 11:58 AM
Gears: 2 (defaults to small)
Git URL: ssh://email@example.com/~/git/mywebsite.git/
Deployment: auto (on git push)
haproxy-1.4 (Web Load Balancer)
Gears: Located with jbossews-2.0
jbossews-2.0 (Tomcat 7 (JBoss EWS 2.0))
Scaling: x1 (minimum: 1, maximum: available) on small gears
postgresql-9.2 (PostgreSQL 9.2)
Gears: 1 small
Connection URL: postgresql://$OPENSHIFT_POSTGRESQL_DB_HOST:$OPENSHIFT_POSTGRESQL_DB_PORT
Database Name: mywebsite
Note that the first line of output shows you the uuid. Also note the domain (or namespace). We will need both of these values later. Now, if the gear running tomcat can’t handle the load anymore, we can scale out. To do that we need to login at the main gear (the one with HAProxy and tomcat). We could use the enormous url or simply use rhc ssh mywebsite for that. On that server we can tell Open Shift to add another gear for the application using the add-gear script:
$ rhc ssh mywebsite
Connecting to firstname.lastname@example.org ...
[mywebsite-first8.rhcloud.com 554742be4382ec93270000d6]> add-gear -a mywebsite -u 554742be4382ec93270000d6 -n first8
With the add-gear instruction, Open Shifts creates a new gear with tomcat running on it. Repeat the add-gear instruction if you want to add even more gears. The deployment now looks like this:
Removing the additional gear can be done in a similar way with the remove-gear script.
Open Shift also has support to automatically scale up as you might have guessed. That’s the part we turned off by setting the disable_auto_scaling marker. If we remove the marker and restart the application, Open Shift will automatically add and remove gears. But how does it decide when to add or remove gears?
This is managed by the haproxy_ctld.rb script. Have a look at it if you want more details. Roughly it counts the number of sessions per gear. If there are more than 16 sessions per gear, it will invoke the add-gear script to scale up. If the sessions go below the threshold it will scale down again after 10 minutes.
To customize this, you can either tweak the variables of that script or write your own. The easiest way is simply to increase or decrease the sessions per gear variable. For a complex application, 16 sessions per gear might be too much. For simple or efficient applications it might be that a gear can handle more than 16 sessions. You can change the number to e.g. 24 by setting an environment variable (check the script for more options):
$ rhc set-env OPENSHIFT_MAX_SESSIONS_PER_GEAR 24 -a mywebsite
You can also set a hard minimum or maximum of gears per cartridge:
$ rhc cartridge scale tomcat7 -a mywebsite --min 1 --max 12
Of course, each application scales differently. For better autoscale functionality it might require that you use different parameters than sessions-per-gear. Maybe CPU usage or memory usage is a better indicator for your application. The best way to do that is to modify the haproxy_ctld.rb script. If you place your modified version in .openshift/action_hooks directory of your checkout, it will overwrite the implicit default version.