Building a Robust Environment from the Ground Up

Data Feeds

We have worked closely with Cambridge Trading Research (CTR) to build a custom solution from the ground up, though there are a wide number of vendors available.

There are countless data feeds available to an FX trader, whether from vendors such as Interactive Data, or retail and institutional brokers such as Currenex, EBS, KCG HotSpot and Liquid-X.  A common practice is to plot charts using just one price i.e. the bid, offer, mid or ‘last’.  The problem with this, as we have seen, is that there isn’t just one price for any currency pair at a given moment.  To use the AUDJPY example, we would get very different rates if we plotted just the bid from AUDJPY, or the mid or the offer.  Although it’s impossible to know the true high of the day in FX, unless taking data from every single bank and broker, we know that if the markets were 93.19/21, then we could sell at 93.19 and buy at 93.21.  If the market then widened due to time of day, or just prior to a data release and the market became 93.09/31, it does not mean the low has now become 93.09, nor the high 93.31.  It simply means the spread has widened.  Therefore when plotting the data, it’s more accurate to use the best(lowest) offer as the low and the best(highest) bid as the high; as we know the market was dealable at those rates, representing ‘true’ prices.

We can take that theory one step further and also calculate the best ‘synthetic’ bids and offers for both the Euro and USD components, from HotSpot, Currenex and Liquid-X; all quoting only actual dealable rates, as opposed to ‘indicative’.  This gives us both a very accurate feed, with dealable rates for the highs and lows, plus of course redundancy should one feed go down. Of course the more feeds the better, as long as it doesn’t overload the CPU; the update frequency can be very high, particularly during periods of high volume, which can overload a CPU.  This can actually cause Excel to hang, so some feeds may need ‘throttling’ to prevent an excessive amount of updates per second.

All of these calculations could be done within the CTR software, but we have chosen to do these basic calculations in Excel, so that we can physically see and monitor what is happening and quickly identify where any issues are.

Once these rates are captured, they need to be stored in a database.  Capturing tick data across many currency pairs means potentially thousands of updates per minute, so the database must be robust, with plenty of storage ‘headroom’; a full HDD will quickly crash any machine.


This could be an entire article in itself, but suffice to say this environment cannot be run on a standalone computer and the only robust solution is to house the server in a data centre, of which there are many to choose from.  Typically in London though, for a low latency connection to the main ECN’s, it means locating in ‘LD1 through 5’ – and more often LD4 in Slough.

There are many other equally good data centres located in other locations, only a few milliseconds further away, but in the world of high frequency trading (HFT) where low latency is vital, even having a server a few feet closer to the exchanges’ servers, can make a difference; and there is now a high premium to pay for both the location of the data centre, as well as location within the DC itself.  The old adage of ‘location, location, location’ seems to hold true even in the world of HFT.  There are similar data centres and premiums on space, in every major financial centre.

trading systems data centers.png

IT has come a long way in recent years and there are now Virtual Private Server (VPS) solutions available, which are both more robust than a single physical server as well as relatively inexpensive to maintain; the server actually residing on a ‘cluster’ or servers, meaning that any one server can completely fail, yet allow the VPS to continue running uninterrupted with no downtime.  From this perspective, the playing field has been very much levelled in favour of smaller firms and individual traders, who can now afford similar levels of redundancy and raw processing power, which would have cost as least ten-fold the amount, until relatively recently. 

<<Previous     Next>>