2. Message Batching

Many messaging implementations are written to minimize network overhead and maximize overall throughput by batching together short application messages and sending several of them in a single network packet. Network bandwidth is best conserved when packets are consistently at or near the MTU (Maximum Transmission Unit). The TCP protocol uses Nagel's algorithm to do this batching, and many other messaging systems use similar methods.

However, this batching introduces delays in the transmission of application messages, which can be undesirable for latency-sensitive applications. Some system designers want to eliminate this batching completely. For TCP-oriented messaging, it can be done with socket option TCP_NODELAY, with the result that even very short messages are sent immediately, each in its own packet. Alternatively, UDP-oriented messaging can be used (UDP does not include any batching in the protocol stack). These methods eliminate the Nagel-like delays, but also reduce efficiency, increasing both the CPU load and network bandwidth usage. The result is a reduction in latency with a potential increase hardware costs (both computer and network) to achieve a desired overall throughput.

There is a case where batching can lead to a improvement in average message latency. If the application has several messages that it wants to send, combining them into a single network packet can save a significant amount of overhead, resulting in a decreased average latency for the group of messages. However, since most batching algorithms use a timer to trigger the transmission of the last bit of data that doesn't fill a packet, those algorithms tend to erase the latency savings made through the reduction in overhead.

The 29West LBM product allows batch transmission to be based on both time and packet size, or batching can be turned off. Moreover, LBM allows batching to be configured on a transport session basis, so that the same application can reduce or eliminate batching for latency-sensitive data while allowing less-sensitive data to be batched, increasing both network and computer efficiency. Finally, LBM supports explicit batching, allowing the application to delimit groups of messages that can be combined while forcing the last message to be sent immediately (instead of by a timer).