8. Use Uncertainty to Your Advantage

If spikes are a fact of life and your applications won't always be able to keep up with them, then it may be important for your applications to take advantage of uncertainty. If processing a spike without loss would push your application over its established latency boundary, then it might be best to configure the messaging system to deliver unrecoverable loss events instead of adding additional latency attempting to recover lost messages. See Section 4 above for details.

This could leave your application in a position where it knows the current price, but also knows that it has missed some of the trades from the market that lead to the current price. Applications built requiring full knowledge of the market before trading may be at a disadvantage compared with applications that could still trade (though perhaps less aggressively) with current prices and and imperfect knowledge of market history.

Messaging systems can detect packet loss and request retransmission to repair the loss. This provides lossless message delivery to receiving applications, but adds latency at the receiver while the loss is repaired. This provides another opportunity for applications to deal with uncertainty.

The most latency-sensitive applications may want to know as soon as packet loss is detected by the messaging system. Such applications would continue to receive messages after the loss, hence avoiding the repair latency on them. If and when repair is completed, the application would then receive the older messages that had been lost. This requires the receiving application deal with uncertainty of loss repair and to make sense of messages that arrive in a different order than they were sent. In return for this additional complexity in the application, messages that did not have to be retransmitted will have no latency due to loss repair in the messaging system.

Avoiding repair latency requires sequence number coordination between the messaging system and the application. The LBM and UME products from 29West have the necessary features to support applications that want to eliminate loss repair latency.