Java and JSON ain't friends

Most application frameworks provide some REST support, which is – depending on the language you are using – either dirt cheap, or quite complex. In the Java world part of these frameworks is some kind of mapping from JSON to Java and vice versa, most of them using the Jackson mapping framework. It feels quite natural: you model your domain objects directly in Java. If you don’t have any constraints, the JSON might even follow your model. If the JSON is predefined (as part of the API), you can either design your Java classes so they fit the generated JSON, or provide Jackson with some mapping hints to do so. But you know all that, right? So what am I talking about here?

The point is: domain models may vary in size from a few properties to x-nested sky high giants… and so are the resulting Java model classes. What makes things even worse, is that domain models change over time. Often you don’t know all the requirements front of, also requirements change over time. So domain models are subject to change. All that is still not a big problem, as long as you don’t need to communicate those domain models to others. If other parties are involved, they have adapt to the changes. Let’s take the following JSON describing a person:

{
 "id":"32740748234",
 "firstName":"Herbert",
 "lastName":"Birdsfoot",
}

We can write a simple Java class that will de-/serialize from/to this JSON:

public class Person {

 private String id;
 private String firstName;
 private String lastName;

 public String getId() {
  return id;
 }

 public void setId(String id) {
  this.id = id;
 }

 public String getFirstName() {
  return firstName;
 }

 public void setFirstName(String firstName) {
  this.firstName = firstName;
 }

 public String getLastName() {
  return lastName;
 }

 public void setLastName(String lastName) {
  this.lastName = lastName;
 }
}

We can write a simple test to verify proper mapping to JSON:

public class PersonTest {

 private final ObjectMapper mapper = new ObjectMapper();
 private String personJsonString;

 @Before
 public void setUp() throws Exception {
  personJsonString = IOUtils.toString(this.getClass()
    .getResourceAsStream("person.json"));
 }

 @Test
 public void testMapJsonToPerson() throws Exception {
  final Person person = mapper.readValue(personJsonString, Person.class);
  checkPerson(person);
 }

 protected void checkPerson(final Person person) {
  assertNotNull(person);
  assertEquals("32740748234", person.getId());
  assertEquals("Herbert", person.getFirstName());
  assertEquals("Birdsfoot", person.getLastName());
 }

If we run the test, everything is nicely green:



That was easy. But now we have new requirements: we need to extend our person with some address data:

{
 "id":"32740748234",
 "firstName":"Herbert",
 "lastName":"Birdsfoot",
 "address":{
  "street":"Sesamestreet",
  "number":"123",
  "zip":"10123",
  "city":"New York",
  "country":"USA"
 }
}

If we run our test against that JSON we will get red



If you have a look at the StackTrace, Jackson is complaining about unknown properties

com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: 
Unrecognized field "address" (class rst.sample.Person), not marked as 

ignorable (3 known properties: "lastName", "id", "firstName"])
 at [Source: {
 "id":"32740748234",
 "firstName":"Herbert",
 "lastName":"Birdsfoot",
 "address":{
  "street":"Sesamestreet",
  "number":"123",
  "zip":"10123",
  "city":"New York",
  "country":"USA"
 }
}; line: 5, column: 13] (through reference chain: rst.sample.Person["address"])
 at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.

    from(UnrecognizedPropertyException.java:51)
    ...

Now we have two choices. We can extend our Java model by the missing properties. That’s quite easy. And if the producer of that JSON is using Java either, we might just copy their model, can’t we? Well….you may. I have seen this in a current microservice project. People have been passing model classes around on every change, often asking for some common domain model lib. Don’t do that. Never ever. First of all ask yourself: are you really interested in the data introduced by the change, or are you only adapting to satisfy the serialization? If you need all the data, you have to adapt. If you don’t need the data, don’t adapt, there are mechanisms to prevent serialization complaints. There is e.g. an easy way to tell Jackson to ignore superfluous properties:

@JsonIgnoreProperties(ignoreUnknown=true)
public class Person {
...

If you run your test again, everything is green again. In fact, application frameworks utilizing Jackson like e.g. Spring are configuring Jackson to always ignore unknown properties, since this makes sense in most situations. But be aware of it since it is not explicit to you.



So what about that anger on common domain model libs? I have seen this quite often in customer projects: developers start to create some common domain model libs, so everyone out in the project may use it. The point is: over time people are extending the domain models with their specific needs… whether anyone else needs it or not. And this leads to models bloated with any kind of niche domain functionality depending on a hell lot of other bloated domain objects. Don’t do it. Duplicate the models and let every party evolve its own view to the domain.

You have to choose where to pay the price of complexity. Because DDD is about reducing complexity in the software, the outcome is that you pay a price with respect to maintaining duplicate models and possibly duplicate data.
Eric Evans – Domain Driven Design

But – as always – things might not be that easy. What if you do not care about that superfluous data, but you have to pass it to another party. Hey, we give them the person ID, so they can retrieve all the data they want on demand. If this is the case, you are safe. But sometimes you don’t want to pass data by handing a (foreign) key, which actually means: by reference. Depending on the business case you may have to pass a snapshot of the current data, means: by value. So what about that case, do I have to copy the model classes again in order to specify all possible properties?!? Damn, in dynamic languages like Javascript or Clojure the JSON is generically “mapped” to the object, and I do not have to care for any class schema. Couldn’t we do that in Java also, at least in some – well, less comfortable – way? If you search online for solutions on that problem, you will often find this one:

public class Person {
 
 private String id;
 private String firstName;
 private String lastName;
 private final Map<String, Object> map = Maps.newLinkedHashMap();
...
 @JsonAnySetter
 public void add(final String key, final Object>  value) {
  map.put(key, value);
 }

 @JsonAnyGetter
 public Map<String, Object> getMap() {
  return map;
 }
}

Let’s give this solution a try. We will extend our test in order to check if the JSON output created by serialization equals the original input:

public class PersonTest {
...
 @Test
 public void testMapJsonToPersonToJson() throws Exception {
  final Person person = mapper.readValue(personJsonString, Person.class);
  final String newJson = mapper.writeValueAsString(person);
  JSONAssert.assertEquals(personJsonString, newJson, true);
 }
}

Let it run and, tada:



… it fails, er?!? Yep, the solution described above works for simple properties, but not for nested ones. So we got to do better. Instead of Object, use Jackson’s JsonNode:

public class Person {
...
 private final Map<String, JsonNode> map = Maps.newLinkedHashMap();
...
 @JsonAnySetter
 public void add(final String key, final JsonNode value) {
  map.put(key, value);
 }

 @JsonAnyGetter
 public Map<String, JsonNode> getMap() {
  return map;
 }
}

Run the test again, and <drumroll>:



Phew, green :-)

So this is the solution that solves our problem. It is both compatible to changes – as long as the properties we are actually using are not subject to change – and reconstructs the original JSON as we received it. Work done.

That’s it for today
Ralf

The only way to have a friend is to be one.
Ralph Waldo Emerson