How to build a scalable derivatives risk & pricing platform using open source software March 30, 2007
Posted by James Webster in : software, finance , trackbackJust some brief thoughts on how a selection of open-source software might be used to build a architecture stack supporting a derivatives risk & pricing platform, a common system found on derivatives trading floors.
- QuantLib: A popular quantitative finance library implemented in C++ with multiple language bindings. It would provide a good base upon which to build more exotic (and proprietary) models. There are a number of commercial toolkits available as well.
- Hadoop: A Java implementation of Google’s MapReduce algorithm. Hadoop would support horizontal distribution across a server grid of the pricing calculations implemented by QuantLib within a Monte Carlo method framework. A commercial alternative (although much broader in scope) would be DataSynapse. Also have a look at the recently released IBM alphaWorks MapReduce Tools for Eclipse.
- OpenTick: You have to pay the exchange fees for the data but otherwise OpenTick provides an open API for accessing real-time and historical ticks for a broad range of US exchanges and ECNs.
- Ehcache: Although Hadoop includes a Distributed File System it is geared towards large files. A better solution for ensuring the prices received and the figures calculated are updated across the server grid would be a distributed cache. Ehcache has had basic distributed caching for a while now. A commercial alternative would be Tangosol Coherence.
- Esper: Event stream processing (aka complex event processing) is becoming a big part of trading systems. Plugging an ESP toolkit into our stack might provide a way to support algorithmic trading or identification of arbitrage opportunities. There are many commercial players vying for success in this market.
- Amazon EC2: And finally, as a deployment option why build your own grid of servers when you can rent one? There is already a HOWTO for running Hadoop on EC2. One drawback to using EC2 is the current lack of support for IP multicast, but I suspect this will change one they are able to effectively firewall off multicast packets from separate EC2 domains. Another drawback that may be harder to overcome is the latency arising from the Internet links between OpenTick, Amazon EC2 and the trading workstations that ultimately receive the output of this platform.
Whether some of the open-source alternatives I have suggested are appropriate substitutes for their commercial brethern is highly debatable. Nevertheless the commoditization of software continues.
Comments»
Thanks for this useful architectural stack.
Do you have similar architectural stacks for other end-uses, for example, “How to build a healthcare electronic patient record system?”
I hadn’t taken a look at QuantLib in a while. I found the multiple language targets to be very interesting.
One of the often repeated jokes at my last company was:
Q: How do you tell how many quants have worked on a given code base?
A: Count the number of date libraries which have been written.
It seems most banks are happy to use open source software - especially because they don’t ship products, but it’s hard to get agreement to open source any internal software or contribute changes back to the community.
[…] while ago I suggested that Hadoop and Amazon EC2 could be use to construct an open source derivatives risk pricing grid platform as an alternative to commercial offerings such as Datasynapse or Digipede. Well now there is a […]
EHCache needs multicast, so no dice running it on EC2 at least until it is available.
Aside from EHCache, what else were you thinking you need multicast for on EC2? From what I’ve been reading, EC2 apps tend to towards an event driven design based on SQS.
@Gavin: Just caching and clustering really; Open Terracotta, EHCache, Tangosol all really require multicast to work property if not at all.
[…] using Amazon EC2 to quickly and cheaply scale up a grid computing environment, much as I asked a while back. As a way to scale up processing of complex overnight risk reports over exotic derivative books it […]