jump to navigation

Everything old is new again July 10, 2008

Posted by James Webster in : development , trackback

Back in the day, I worked at a startup whose mission was to build a multi-application smartcard lifecycle management system. One of the things I remember a colleague (hey Jelte!) developing was an ASN.1 encoder/decoder which was necessary for exchanging data with EMV smartcard applications. Around about this time XML/XSD was also foisted on the world - which you could possibly claim is a bloated text version of ASN.1 (which some would also claim is a bloated and vague specification itself).

So it was interesting to see that Google have open-sourced their Protocol Buffers project:

Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data – think XML, but smaller, faster, and simpler.

In other words, their interpretation of an interoperable efficient binary encoding scheme but without the complexity of its forebears (ie. ASN.1 and Corba IDL).

At the moment there are only bindings to C++, Java and Python but I expect a .Net version to appear from the OSS community sooner or later.

Some thoughts/random ideas:

Comments»

1. harry - July 13, 2008

writing a asn decoder is not for the faint hearted… the pb stuff seems magnitudes of order simpler

2. James Webster - July 13, 2008

Oh I agree completely. ASN.1 and it’s encoding rules (DER/BER/XER/?ER) are very heavyweight.

3. Ade - July 13, 2008

I think that manually writing a .proto file for describing FIX messages would be a big undertaking. It might be more interesting to write something which auto-generates the .proto files from the FIX schema or whatever representation you already have.

I expect that you would see significant improvements in memory footprint, file size and parsing times compared to your existing XML solution. But why not write some tests to prove it. The PB mailing list is currently looking for complicated examples to use for benchmarking the implementations of PB in different languages and I think FIX would be perfect for this.