NML

Serialization with Newtonsoft JSON

By Charl Marais

Serialization and deserialization is hard. Very. Serialization to an un-typed, semi-structured format, and deserialization back from it, is even harder, as I recently re-discovered to my chagrin.

There are plenty of good .Net serialization implementations. On the binary end of the scale, there is the trusty, if somewhat maligned, built in BinaryFormatter, and a slew of other binary serialization implementations like protocol buffers, message pack, wire, BSON etc.

On the non-binary side you have 2 built in serializers in XmlSerializer for XML, and DataContractSerializer for both XML and JSON, and a host of implementations for more formats than you can shake a stick at.

Newtonsoft provides an extremely well structured, easy to use, but fully extensible, serialization framework for both BSON and JSON serialization formats.

Here are some solutions to common problems one faces when serializing to and from a format like JSON.

#Type resolution The first important thing to realise about JSON serialization is that JSON is not a strictly typed format. Consider the following:

{
  "MessageId" : "message-1",
  "Payload" : {
    "Name" : "Charl",
    "LastName" : "Marais"
  }
}
{
  "MessageId" : "message-1",
  "Payload" : {
    "Steps" : [0, 1, 2, 3],
    "LastStepName" : "Step 4"
  }
}
public class Message
{
   public string MessageId {get; set;}
   public object Payload {get; set;}
}

Both JSON objects above are valid serialized representations of the Message type. However, during deserialization, there is no information in the JSON representation to tell the serializer what the expected type is to deserialize too, and the .Net type does not help either, as Payload is of type object. This isn’t only a problem when using object as the property type. Generic implementations can suffer the same type ambiguity.

Luckily the Newtonsoft JSON serializer has an option for overcoming that shortcoming: TypeNameHandling. With that, you can tell the serializer to included the type information of the an object in the JSON representation.

{
  "$type" : "MyProject.Message",
  "MessageId" : "message-1",
  "Payload" : {
    "$type" : "MyProject.Entities.Person",
    "Name" : "Charl",
    "LastName" : "Marais"
  }
}
{
  "$type" : "MyProject.Message",
  "MessageId" : "message-1",
  "Payload" : {
    "$type" : "MyProject.Processing.StepRunner",
    "Steps" : [0, 1, 2, 3],
    "LastStepName" : "Step 4"
  }
}

Now the serializer can see the target type to deserialize the Payload property into. You specify how the Newtonsoft serializer must implement TypeNameHandling as follows:

var serializer = new Newtonsoft.Json.JsonSerializer();
serializer.TypeNameHandling = TypeNameHandling.All; //None, Objects, Arrays, All, Auto

//OR

var serialized = JsonConvert.SerializeObject("", new JsonSerializerSettings {TypeNameHandling = TypeNameHandling.All});

#SerializationBinder Irrespective of whether you include type information into your serialized respresentations, you can take more control of type resolution by creating a custom implementation of the System.Runtime.Serialization.SerializationBinder abstract type.

public abstract class SerializationBinder
{
  public virtual void BindToName(Type serializedType, out string assemblyName, out string typeName)
  {
  }
  public abstract Type BindToType(string assemblyName, string typeName);
}

You specify the SerializationBinder as follows:

var serializer = new Newtonsoft.Json.JsonSerializer();
serializer.Binder = new MyCustomSerializationBinder();

//OR

var serialized = JsonConvert.DeserializeObject(instanceToDeserialize, new JsonSerializerSettings {Binder = new MyCustomSerializationBinder()});

BindToName allows you to customize the name that a type is represented by. For example:

public override void BindToName(Type serializedType, out string assemblyName, out string typeName)
{
  if(serializedType == typeof(MyProject.Processing.StepRunner))
  {
    assemblyName = null;
    typeName = "FooBar";
    return;
  }
}

This will result in something like:

{
  "$type" : "FooBar",
  ....
}

Generally one would leave BindToName well enough alone, unless there are specific considerations that need to be taken into account. If you want to obfuscate type names for security reasons, or provide mapped short representations for serialized data size reasons.

The more interesting method is BindToType. Bind to type is called during deserialization, and it allows you to specify the type you want associated with the name. If a type has moved, you need only check for this in BindToType and return the correct type.

public override Type BindToType(string assemblyName, string typeName);
{
  //Specially named
  if(string.IsNullOrEmpty(assemblyName) && typeName.Equals("FooBar"))
    return typeof(MyProject.Processing.StepRunner);

  //Individual has become Person and moved to MyProject.Entities
  if(assemblyName.Equals("MyProject") && typeName.Equals("MyProject.Individual"))
    return typeof(MyProject.Entities.Person);

  return Type.GetType(typeName);
}

#Serialization Constructors Newtonsoft works seamlessly with the built-in .Net ISerializable interface and deserialization constructors. I highly recommend using the built-in constructs for customizing and controlling serialization before going to Newtonsoft specific mechanisms.

The following deserialization constructor will be called by the Newtonsoft serializer with any additional setup or configuration required:

public class Person : ISerializable
{
  public string FirstName {get;set;}
  public int Age {get;set;}

  public Person(string firstName, int age)
  {
    FirstName = firstName;
    Age = age;
  }

  public Person(SerializationInfo info, StreamingContext context)
  {
    FirstName = info.GetString(nameof(FirstName));
    Age = info.GetInt(nameof(Age));
  }

  public void GetObject(SerializationInfo info, StreamingContext context)
  {
    info.AddValue(nameof(FirstName), FirstName ?? "");
    info.AddValue(nameof(Age), Age);
  }
}

Be careful with the deserialization constructor, because if a property isn’t specified in the JSON representation (perhaps it was null during serialization), trying to access the property on the SerializationInfo will throw an exception. One should always access serialized property values in deserialization constructors within a try...catch, to cater type mutation over time.

#DataContracts Newtonsoft respects DataContract serialization definitions, which allows you to use a .Net accepted and widely used approach to customizing serialization. This also means that your data types are transferable to implementations not relying on the Newtonsoft JSON library, but perhaps using the built-in DataContractSerializer. Again, I highly recommend you follow the wider .Net implementation paradigms before going to Newtonsoft specific solutions.

[DataContract]
public class Person
{
  [DataMember(Name="first_name")]
  public string FirstName {get;set;}

  [DataMember(Name="age")]
  public int Age {get;set;}
}

This above type will serialize to:

{
  "$type" : "MyProject.Entities.Person",
  "first_name" : "string",
  "age" : 0
}

#JsonConverters One can also implement custom JSON converters for types.

public abstract class JsonConverter
{
  public virtual bool CanRead {get;}
  public virtual bool CanWrite {get;}

  public abstract void WriteJson(JsonWriter writer, object value, JsonSerializer serializer);
  public abstract object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer);

  public abstract bool CanConvert(Type objectType);
}

The method signatures are self explanatory, but note the following:

  • You have to implement WriteJson and ReadJson as they are defined as abstract, but the implementations can throw a NotImplementedException as long as the accompanying CanWrite or CanRead return false
  • The JsonReader is a forward only token reader. If you read a token and move on, you can go backwards to query it again.
  • The JsonWriter is a forward only token writer. You cannot go back and rewrite a token.

With JsonConverter, you can write an implementation that can deal with outdated serialized versions of an object, or provide different implementations for different versions, or allow for dealing with inheritance or generic types.

JsonConverters can be added to the serializer, or associated with a type using the JsonConverterAttribute.

serializer.Converters.Add(new PersonJsonConverter);

OR

[JsonConverter(typeof(PersonJsonConverter))]
public class Person{}

Noteworthy

Take some time to look at the customizations that can be specified for Newtonsofts serializer. In addition to TypeNameHandling, you can also configure:

  • Formatting
  • MissingMemberHandling
  • ReferenceLoopHandling
  • ObjectCreationHandling
  • NullValueHandling
  • DefaultValueHandling
  • ConstructorHandling
  • DateFormatHandling
  • DateTimeZoneHandling
  • DateFormatHandling
  • FloatFormatHandling
  • FloatParseHandling
  • StringEscapeHandling