First8 is gespecialiseerd in het pragmatisch ontwikkelen van bedrijfskritische Java toepassingen waarbij integratie van systemen, hoge eisen aan beveiliging en veel transacties een belangrijke rol spelen. Op deze pagina vindt je onze blogs. Op www.first8.nl vind je ons nieuws, de agenda, vacatures en meer.

Grails 3.1 multi tenant databases

Grails and databases, easy right?

Having Grails and GORM makes a programmers life really easy when a simple database is your target storage. Grails generates a bunch of things for you, the conventions allow for almost zero configuration and presto: there’s your working application running on spring boot.

Things change though when you need to, for instance, white-label your application. A third party might actually host and/or maintain your application database and data. These kinds of setups are like renting out parts of a building, this goes by the term “multi tenant”.

Miss Eveleen Tennant, John Everett Millais

Approaches to multi tenant data

  1. Separate databases: different customers need to get connected to different (virtual) hosts
  2. Separate schemas: customer data is kept separate in the database by defining schemas for database users
  3. Partitioned data: same database & schema, but all data is always tagged with a customer identification

The first two approaches allow for a very generic setup and clear security boundaries, which can be convenient.

The third option requires quite some ‘intelligence’ in the application, and can become very tricky if data security is high priority. Customer data must be kept separate, and only the (client) application knows how.

 

Multiple approaches may apply

Our legacy application has all the approaches at once. Tenants are defined by schemas, but not all schemas are on the same host and within a tenant, the data is again spread over multiple sub-tenants. How we handle the sub-tenants is not part of this story, it is basically the ‘partitioned data’ approach (3) for which some tooling exists.

 

grails-wide

 

How Grails handles things

Grails 3 has no clear guidelines on how to handle multi tenant data. It assumes by default that only 1 data source is used. GORM does allow specifying on which data source a table is located, but of course that is not multi tenancy, rather just a different way to spread data over multiple databases.

For Grails2 (also assumes one data source by default) there were a few ready made options in the form of plugins which made GORM have support for multi tenancy.

Hibernate has explicit support for multi tenant databases, but only starting from version 4 (and partitioned data from ~maybe~ version 6). How this interacts with GORM however is something else. The latest GORM (not the default for Grails) promises actual support. However, since we also want to be able to use the lower level groovy Sql approach in a seamless way, we would like to avoid fiddling with Hibernate configuration magic.

 

To the drawing board

We need to connect to all the relevant databases and schemas and always pick the one that our customer needs.

The ideal way for us is to have the ‘normal’ Grails data sources, normal GORM, normal Hibernate and still have all the flexibility.

 

A solution using Spring

Spring provides a so-called switching data source, somewhat like a facade to the real data sources.

Our solution relies on the fact that the tenant can be safely assumed to stay the same during the processing of a request to the server. In other words, as long as the same request is still being handled, only 1 tenant is relevant.

A consequence of this approach is that things like second level cache are non trivial since that would mean that data from multiple request are cached in that same ‘bucket’. This would mean that data from multiple tenants would get mixed. This in turn can be prevented with another filter at the cache level, but…. -things get complicated here- (and we don’t need it). Disabling fancy caching is fine for now.

 

The details

Ingredients:

  1. an interceptor that identifies the customer and relates it to a tenant
  2. a thread local construct to store the tenant id during the request processing
  3. a switching data source, Spring provides one jay!
  4. some configuration for the various data sources and how they relate to tenants

 

1) The Interceptor

Grails interceptors are very powerful and easy. They come with dependency injection, access to configuration, etc.

This is the place to get to know which tenant is relevant for a request, before the request is actually handled. Combining with logging using MDC seems convenient, you might want to know which tenant is logging something (or have different log files for instance).

The interceptor should also make very sure that the current Thread is clean when the request handling is finished. Threads are reused, so consecutive requests could potentially interfere!

Example interceptor:

2) A Thread local to go

We need a place to store the ‘current’ tenant, a Thread local variable is the simplest way to achieve this.

It is good practice to encapsulate. Watch out with unit tests, a Thread local might ‘stick’ and tests could again (like requests) influence each other.

Thread local Tenant holder:

Note that you may put anything in this Thread local, you are not limited to Strings or primitives. Having a configuration specific to the tenant ready here might be convenient.

 

3) A switching data source

Spring provides a ready made abstract class that just needs a little logic to somehow determine what data source to select.

It expects a list of data sources to be configured elsewhere. This implementation logs some of the internals when in trace mode for debugging.

A warning about multiple data sources (switching or not): Grails 3 needs the default data source to be named ‘dataSource’. There is no way around this!

Switching data source:

 

4) Some configuration

Linking tenants to specific data sources and providing the actual data sources as a list to the switching data source (the facade).

In the example code the tenantId parameter matches the actual data source name, this is of course a simplification and not something you might want in a production setup.

application.groovy (does not exist by default):

resources.groovy:

Done. Now let’s see what using all this actually looks like.

Using groovy Sql

Using Domain classes

It seems that only regular groovy Sql is supported correctly. Where Grails2 had no problems with a switching default data source, Grails3 unfortunately is not as forgiving. For real GORM (domain classes) support, see the information on the recently introduced GORM multi-tenancy support.

 

Seeing everything together

  1. A request arrives at the server.
  2. The interceptor sets, based on the request, a marker on a thread local variable.
  3. Whenever the ‘database’ is requested for querying, the switching data source will choose the correct source based on the thread local variable and the configuration.
  4. All the following queries are made on the correct ‘real’ data source.
  5. When the request ends the thread local is cleared, the data source is left as-is and ready to be reused.

 

Further reading

Hibernate on multi tenancy:

 https://docs.jboss.org/hibernate/orm/4.3/devguide/en-US/html/ch16.html

Somewhat dates but still relevant information from spring:

 https://spring.io/blog/2007/01/23/dynamic-datasource-routing/

An implementaton in java using some AOP:

 http://fedulov.website/2015/10/14/dynamic-datasource-routing-with-spring/