High-Performance Object Serialization based on Ahead-of-Time Schema Generation

Many of today’s Big Data systems are developed in the Java programming language and require fast object serialization and deserialization for efficient and high-speed message exchange. We address this by introducing Skema, a library providing high-performance serialization of Java objects. Skema works based on Ahead-of-Time schema generation, so that only a minimal number of operations need to be performed at runtime to serialize an object. This approach allows Skema to also serialize individual objects only partially which is a great advantage for network communications with fixed buffer sizes. Furthermore, targeted optimizations, such as caching the size of the serialized form of objects that have a fixed size so that they do not have to be recomputed for each operation, are presented. In addition to a more in-depth explanation of the implemented procedures, this paper also presents benchmark results comparing the Skema library against Kryo, FST and the native Java serialization function. The results of these experiments achieve very good values in comparison with the remaining libraries. For example, one experiment shows that there can be nearly a hundredfold increase in the speed of deserializing objects when compared to the native Java serialization mechanism by using Skema. We also show that deserializing objects using the native Java mechanism requires a large amount of additional memory per operation, which puts a burden on the garbage collector afterwards. Finally, to investigate the scalability of the implemented solution, the benchmarks are also performed using different numbers of threads and the results are presented graphically.