Skip to main content
Blog

Forward compatible messaging

By 11 september 2015januari 30th, 2017No Comments

If you have services or domains in your architecture, you probably have them to be able to develop them independently. In most cases, Message Queue’s are used to communicate between these services. But how does this work if one service upgrades the messages it is sending while the consuming services are still on an older version? In this blog post, we’ll dig into how Java Serialization deals with this.

Setup

To experiment, we’ll set up a Wildfly 9 container with a message queue. On it, we will deploy two domains or microservices, one that produces messages and one that consumes them.

First, setup a queue. Start Wildfly with standalone-full.xml profile so HornetQ is enabled.

$ ./standalone.sh -c standalone-full.xml

Next, we’ll add a testQueue for our purposes.

$ ./jboss-cli.sh

You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
[disconnected /] connect
[standalone@localhost:9990 /] jms-queue add --queue-address=testQueue --entries=java:/jms/queue/testQueue,java:jboss/exported/jms/queue/testQueue
[standalone@localhost:9990 /]

Note: the second address contains jboss/exported which is required to make it accessible from the outside. You can leave it out if you want.

The code

Now, for some code. We will be sending an Event which looks like this:

public class Event implements Serializable {
  private static final long serialVersionUID = 1L;
  private String payload;
  public Event(String payload) {
    this.setPayload(payload);
  }
  public String getPayload() {
    return payload;
  }
  public void setPayload(String payload) {
    this.payload = payload;
  }
}

Our producing side could look something like this:

@Resource(mappedName = "java:/jms/queue/testQueue")
private Queue testQueue;

@Inject
private JMSContext context;

public void hello() {
  try {
    context.createProducer().send(testQueue,
      context.createObjectMessage(new Event("hello")));
  } catch (Exception e) {
    e.printStackTrace();
  }
}

And our consuming side uses a Message Driven Bean to consume the messages.

@MessageDriven(name = "EventMdb", activationConfig = {
@ActivationConfigProperty(propertyName = "destinationLookup",
  propertyValue = "/jms/queue/testQueue"),
@ActivationConfigProperty(propertyName = "destinationType",
  propertyValue = "javax.jms.Queue"),
@ActivationConfigProperty(propertyName = "acknowledgeMode",
  propertyValue = "Auto-acknowledge") })
public class EventMdb implements MessageListener {
  private final static Logger LOGGER = Logger.getLogger(EventMdb.class.toString());

  public void onMessage(Message rcvMessage) {
  try {
      if (rcvMessage instanceof ObjectMessage) {
        ObjectMessage msg = (ObjectMessage) rcvMessage;
        Event event = (Event) msg.getObject();
        LOGGER.info("Received Message from queue: " + event );
      } else {
        LOGGER.warning("Message of wrong type: " + rcvMessage.getClass().getName());
      }
    } catch (JMSException e) {
      throw new RuntimeException(e);
    }
  }
}

Testing forward compatibility

For our test, we deploy these two classes as separate WAR’s, bundled with their own Event class. Thus, this message is implemented twice, one in the producing domain and one in the consuming domain. You could also have a shared library between these domains where this class is placed. The same problem will arise if the producing domain upgrades to a newer version of that shared library.

Now, see what happens if we want to extend the Event class, exposing more information:

public class Event implements Serializable {
  private static final long serialVersionUID = 1L;
  private Date timestamp;
  private String payload;
  public Event(String payload) {
    this.payload = payload;
    this.timestamp = new Date();
  }
  public String getPayload() {
    return payload;
  }
  public void setPayload(String payload) {
    this.payload = payload;
  }
  @Override
  public String toString() {
    return "Event [timestamp=" + timestamp + ", payload=" + payload+ "]";
  }
}

If we simply add fields but keep the serialVersionUID the same, the consuming side can still deserialize it, even when it has an old version of the Event class. If we upgrade the serialVersionUID, it will explicitly tell the runtime that the versions are incompatible and you will get a nice exception:

Caused by: javax.jms.JMSException: nl.first8.mq.Event; local class incompatible: stream classdesc serialVersionUID = 2, local class serialVersionUID = 1

So we know that we can add fields without too much hassle. We could also delete fields. They will simply end up being null at the consuming side. There isn’t really a convenient way to set a reasonable default in those cases so be careful there. What else can’t we do?

  • change the package name, the class name or the superclass
  • changing the name or the type of a field

But what if you do need to e.g. change a field?

One strategy is to add a new field and fill out both versions of that field. Then, wait until all consumers have upgraded and have started to use the new fields. Then, you could safely delete the old field. E.g. if we want to change the payload from a String to an int, we could use this Event as an intermediate version:

public class Event implements Serializable {
  private static final long serialVersionUID = 1L;
  private Date timestamp;
  @Deprecated private String payload;
  private Integer value;
  public Event(Integer value) {
    this.value = value;
    this.payload = value.toString();
    this.timestamp = new Date();
  }
  @Deprecated
  public String getPayload() {
    return payload;
  }
  @Deprecated
  public void setPayload(String payload) {
    this.payload = payload;
  }
  public Integer getValue() {
    return value;
  }
  public void setValue(Integer value) {
    this.value = value;
  }
  @Override
  public String toString() {
    return "Event [timestamp=" + timestamp + ", payload=" + payload + ", value=" + value + "]";
  }
}

 

Alternatively, you could write your own serializing mechanism (see e.g. Externalizable). You then can make up your own handling for these scenario’s, coping with forward compatibility somehow. Or you could switch to a different protocol or serializer that provide more forward compatibility or even cross-language capabilities. Some commonly used alternatives are kryo or protocol-buffers.

These days JSON is quite trendy so you could also consider to use a TextMessage instead of an ObjectMessage. The text could then be a serialized JSON object. If you want to go this way, you would need to tell the JSON serializer how to handle unexpected fields. In Jackson, you can e.g. add an annotation @JsonIgnoreProperties(ignoreUnknown=true) to get the same behaviour as with Java Serialization.

Read more: