4. Establish a Latency Boundary for Each Receiver

If spikes are inevitable, then some amount of latency due to queuing is inevitable. Latency-sensitive applications receiving spikes are faced with two alternatives:

• loss-free message delivery with additional queuing latency, or

• message loss with reduced queuing latency.

Some applications would prefer to deal with latency spikes by discarding older messages rather than spending time processing a long but lossless queue. Picking an age at which a message is too old to be worth processing may be difficult, but it may be the best way to keep message latency low and relevance high.

The best latency-sensitive applications that can tolerate loss understand this trade off and gracefully deal with loss as a means to keep latency low.

We've found it helpful if each receiving application is designed and built with a "latency boundary" that is appropriate for its needs. Below this boundary, latency is added as messages are queued, but no messages are lost. Above this boundary, the oldest messages are intentionally dropped to keep latency in check.

29West customers using our LBM product for demanding applications like automated market making often set latency boundaries of 100 ms. Other market data applications may have latency boundaries of 1 or perhaps even 10 seconds.