6. Test Application Stability with Rising Market Data Rates

A serious concern for market data teams is assuring stable application behavior as market data rates rise. Market data applications often exhibit crashes or other unstable behavior in response to short-term surges in market data rates. Hence there's a natural interest in knowing maximum market data rates and assuring stable operation at those maximum rates.

In the old days when market data arrived over serial lines, maximum message rates could be easily calculated. Stability of systems under maximum load could be easily tested by artificially generating market data at line maximum rates or by playing back captured data at the maximum rate. The difference between average and peak rates was relatively small (e.g. 10x).

These days, market data arrives over packet-based IP networks making it much more difficult to calculate a maximum possible rate or to generate traffic at that rate. In many cases, the exchanges generating market data can't meaningfully specify maximum possible peak message rates today, let alone accurately predict maximum possible peak rates in the future. (This is yet another good reason to favor latency measurements over data rate measurements. See Section 2 above.) The difference between average and peak rates is now much larger--often 50x or more.

We believe it's best to capture arriving market data with high-resolution time stamps so that it can later be played back at multiples of the actual arrival rate for load testing. As suggested above, measure application latency while replaying. Compare message latency as you increase replay rates to find the rates where queuing and loss-recovery latency becomes significant.

Imagine an experiment where you successfully replayed market data captured on your busiest day to date at 2x the actual arrival rate. Wouldn't it be comforting to know that your applications wouldn't crash if market data arrived tomorrow at 2x today's rate? Wouldn't it be helpful if you could quantify the latency that would be added to your applications if market data rates jumped to 2x tomorrow while using today's hardware?

The LBM and UME products from 29West have features that allow you to build applications that are stable at any market data rate. There are also features that allow fine-grained measurement of latency.