Copyright © 2007 - 2008 29West, Inc.
February 2008
Abstract
This paper explains how our LBM messaging product differs from others in the marketplace and the benefits that result from these differences. Reports from existing 29West customers indicate the benefits of these differences. This paper groups these differences and presents them roughly in order of decreasing benefit to our average customer.
Disclaimers. 1) We know our own products well, but have an incomplete knowledge of other products. 2) The benefits you experience from LBM may be different from those of our existing customers.
Each of the following sections is named for a potentially significant product difference between LBM and other messaging systems. Each section provides the following: our current understanding of other messaging systems; how LBM is different from them; a description of the potential benefits that the difference provides.
Because the sections are self-contained, you can move among them or read them in the order you choose without losing meaning. We often provide links between sections and to other documents to point you to related topics.
Different audiences may have different areas of interest. For example: Application Developers might want to start with Section 4 on API differences. Network Administrators can focus first on Section 3 describing reliable multicast differences and Section 7.7 on our gateway. System Architects could consider Section 2 which describes architectural differences. Managers are invited to scan the whole document lingering on "benefits" and noting the discussion in Section 5 of how 29West differs from other messaging vendors.
Other systems copy and filter messages using host CPU power to run daemon or server processes acting as intermediaries in the flow of messages.
LBM uses a direct path from source to receiver with no intermediaries. Messages are never copied needlessly.
LBM Benefits. Lower latency and higher throughput because there are no CPU bottlenecks in daemons and servers and no bandwidth bottlenecks at the server's NIC.
Section 6 explains the performance impact of this architecture. See Section 7.5 for how this architecture impacts reliability and resilience designs.
Server-based systems use a single transport session (or "connection") from each sender to the messaging server, and daemon-based systems send on one transport session per sending daemon. The three potentially negative consequences of using a single transport session are detailed below.
Mixing messages, of interest to different receivers, on a single transport session burdens the receiver CPU or messaging server with message filtering.
LBM Benefit. A source using multiple transport sessions allows network equipment to filter at wire speed.
Mixing messages with different priorities on a single transport session prevents network equipment from telling the difference.
LBM Benefit. A source using multiple transport sessions allows network equipment to apply prioritization and other Quality of Service (QoS) policies to messaging traffic.
Mixing messages to separate topics on a single transport session prevents them from being sequenced and transmitted separately.
LBM Benefits. A source using multiple transport sessions allows messages on separate topics to be sequenced and delivered independently of each other. This feature is particularly handy for messages of significantly different sizes. The transmission time of large messages need not add latency to small messages from the same source because the sending of small messages can be interspersed within the time taken to send a large one.
See Section 7.2 to learn more about how receivers discover the transport sessions in use by sources.
Other systems were designed when machines had 1 CPU, 1 NIC, and 64 MB of RAM.
LBM Benefits. LBM is designed specifically to leverage today's multi-core, multi-threaded, multi-gigabit-per-second, and multi-gigabyte hardware. LBM lets you bring all available CPU power, NIC bandwidth, and memory to bear on your messaging problems.
See Section 6 for more on the performance impact of this design.
Other systems offer only TCP or only reliable multicast transport protocols.
LBM applications choose the transport protocol with the right characteristics for the job at hand: Use TCP when flow control is needed so that receivers can set the pace of messaging. Use reliable multicast when sources should set the pace of messaging and when scalability is required. Use reliable unicast when sources set the pace but multicast isn't possible.
LBM Benefit. LBM offers a single, clean messaging API that can be used for all messaging needs.
Messaging systems that use daemons burden the receiving daemon with filtering irrelevant messages. Those with servers burden the server.
LBM can filter messages at the source so that they never even leave the source if there are no interested receivers. LBM can filter messages in network hardware. That's done at wire speed and in parallel for all receivers in modern network hardware. LBM can filter messages in the operating system kernel of the receiving machine. LBM can filter messages the receiver's messaging library. See Section 7.4 for more on how LBM efficiently filters irrelevant messages that arrive at receivers.
LBM Benefit. Application architects have a the choice of filtering irrelevant messages in the best location for their application. They have a choice of source filtering, network filtering, kernel filtering, or receiver filtering.
Other systems only sequence the datagrams that make up the reliable multicast streams used to deliver messages on many topics. When a datagram is lost, it must be recovered before delivery of any of the topics on the transport session can continue. Recovery latency is added to all topics on the transport session.
LBM sequences messages on each topic independently and also sequences datagrams at the reliable multicast transport layer. LBM continues to deliver messages as they arrive after datagram loss for topics that have not suffered message loss--even while it continues with loss repair efforts.
LBM Benefit. When faced with lost datagrams, LBM does not add recovery latency on topics that did not suffer message loss.
See Section 4.2 for other ways that LBM allows for the lowest-possible latency in receivers.
Hardware built today specifically for messaging won't benefit from future improvements in CPU, network, and operating system performance unless it is redesigned and reimplemented.
LBM Benefits. LBM runs on commodity hardware and commodity operating systems. LBM will automatically take advantage of future improvements in these areas without redesign.
Other systems implementing reliable multicast protocols often perform well when they repair occasional loss but they behave badly when they confront the massive loss of a crybaby receiver. A lossy receiver like this can easily push a source to spend all its efforts retransmitting, thereby sharply limiting or even stopping new message transmission. The symptoms of such behavior include abundant negative acknowledgments (NAKs) leading to its description as a NAK storm.
LBM sources monitor the rate of retransmissions and do not go beyond a configurable limit. This policy lets an administrator limit the extent to which lossy receivers can slow lossless receivers. The number of lossy receivers doesn't matter--they must all share the available retransmission rate limit.
LBM Benefits. Retransmission rate controls have eliminated concern about NAK storms occurring with reliable multicast. LBM networks continue stable operation under all traffic and loss conditions.
Other messaging systems are known to have problems when sources send faster than even the fastest receiver can receive. This situation is most common when a source is connected to a network with a faster port speed than all of its receivers. Sending in excess of the receivers' port speed triggers loss for all receivers causing them all to request retransmission and making more work for the source at a time when it already has a lot to do.
LBM sources monitor the rate that data is being sent and will not allow it to exceed a configurable limit. Sources have a choice of adding latency or having send requests fail until the data rate drops below the limit.
LBM Benefit. LBM sources never make more work for themselves and can operate with stability in any network configuration.
Other reliable multicast protocols were designed at a time when multicast worked only within a LAN. Design decisions that were made that did not anticipate the modern world where multicast routing is common and delivers many benefits. These messaging systems require gateway daemons using TCP connections to relay multicast between LANs. Such daemons are a common source of latency and administrative headaches.
LBM's reliable multicast protocol is designed with multicast routing in mind. It supports receivers on LANs at remote sites as well as receivers on the same LAN as the source. Direct communication between source and receiver offers so many benefits that 29West actively discourages the use of gateway daemons even though we supply one as part of LBM at no extra cost.
LBM Benefit. Using WAN multicast routing, instead of gateway daemons, lowers latency and provides better reliability.
See Section 7.7 for a description of a gateway bundled with LBM.
Other systems using UDP for messaging often force each message into its own datagram, sacrificing efficiency for small- to medium-sized messages. Each datagram sent consumes CPU power at the source, protocol overhead on the network, and CPU power at the receiver. This lack of batching offers low latency at low message rates, but the CPU power and network bandwidth consumed makes latency much worse as message rates peak.
LBM allows sources to control the batching of messages into datagrams. Batching parameters control the batch size and maximum latency to be added in the interest of efficiency. Applications may also explicitly flush the batching buffer with the sending of a message. Managing batching efficiency can be critically important in high-throughput, low-latency applications like options feed handlers.
LBM Benefit. Applications have the batching controls needed to get high efficiency and the lowest-possible latency when they need it most--when messaging rates peak.
Other systems deliver messages only in the order they were sent. Messages that arrive after a temporary loss occurs make the lost message irrelevant or greatly diminish its value for some types of message streams. Latency must be added as the system attempts to repair the loss if messages must be delivered in the order they were sent.
LBM allows receivers to avoid this latency by specifying that messages should be delivered in the order they arrive. Each message is delivered with the sequence number that LBM automatically added at the source. The receiver gets messages with the lowest-possible latency but it also has the information it needs to determine the relevance of each message. The ordering decision is local to each receiver and each topic. It does not affect the source.
LBM Benefits. Applications need to incur message reordering latency only when they benefit from ordered delivery.
Some messaging systems deliver messages only on a thread from the receiving application. A thread context switch from the messaging system to the application is required for the delivery of each message, thus limiting throughput and adding latency.
Other messaging systems deliver messages only on a thread owned by the messaging system. No thread context switch is required, but the application message delivery callback code cannot block without bringing the messaging system to a halt.
LBM gives each receiver the option of running the message delivery callback code on an LBM thread for the lowest-possible latency. Alternatively, a receiver may choose to use an event queue mechanism built into LBM. The application then provides a thread to dispatch events from the queue. Event callbacks may block without impacting LBM. Simultaneous execution of messaging and application threads makes optimal use of modern multi-core and multi-CPU systems. Any number of queues and dispatching threads may be used.
LBM event queues can be configured with limits on the latency or size of the queue. This allows applications fine-grained control over their queues.
LBM Benefit. Application developers can choose the lowest-possible latency or the freedom to have their callbacks take as long as desired with parallel execution of messaging and application threads.
Other messaging systems vendors often have interests lying well outside of high-performance messaging. Their main revenue drivers tend to be EAI or SOA products. They often develop their messaging products to support these other products rather than to support financial market application developers.
29West focuses exclusively on the needs of the high-performance messaging market. We understand the work of financial market application developers and design our products to specifically target their needs.
LBM Benefits. Other sections of this paper list many benefits that flow directly from our business focus on high-performance messaging. Our products match the needs of financial market application developers well since that is our design focus.
The 29West team has extensive experience in designing, implementing, and deploying messaging systems. Our experiences date back to the days of the very first all-digital trading floors. We have worked on high-performance in the infrastructure supporting messaging, including network protocol stacks and operating systems. We have advanced the state-of-the-art in research on reliable multicast protocols.
LBM Benefits. Our products are designed and built with deep knowledge of the demanding environments in which they will be used.
29West understands that messaging systems are only one part of high-performance applications. All messaging systems are subject to the constraints of the operating system and network hardware that support them. 29West is a Cisco Partner and also has many less formal relationships with other infrastructure vendors.
LBM Benefits. Products from 29West get the best-possible performance from the network and operating system infrastructure upon which they depend.
Some other messaging systems seem to be little more than academic research projects that have been dressed up for commercial sale. Others are open source efforts staffed with volunteer developers.
29West's products are professionally engineered, tested, and supported by top professionals.
LBM Benefits. 29West customers can depend on our products and support when building infrastructure and applications that are critical to their business success.
Other messaging systems may be slower because their designs are older, not because they are ill-conceived or poorly implemented. Our products are not merely optimized implementations of their old designs.
Rather, LBM is faster than its competitors because of its more modern design--a design focused on performance first. Features that impact performance can be disabled when they provide no benefit. LBM is regularly benchmarked well in excess of two million messages per second for message sizes up to 100 bytes. See Section 2 for more on how the architecture of LBM contributes to this performance. See Section 7.4 to learn one way LBM achieves higher performance. See Section 5.2 for the background of the design team.
LBM Benefit. Many 29West customers report direct bottom line benefits that are due to the speed of LBM.
Prospective customers often benchmark LBM latency against that of other messaging systems. Customer use their own engineers and their own equipment to conduct these tests.
Our record in such tests remains unblemished. See Section 2 for the architectural differences that enable lower latency. See Section 4.1 for an example of an LBM feature that contributes to lower latency.
LBM Benefit. 29West customers in the financial markets are more likely to win a latency race against their competitors.
Some other messaging systems have a rigid notion of a hierarchical topic space. Wildcard matches are permitted only within the limits of this space.
LBM imposes no structure on the topic space at all. Our topics are simply strings, though applications often impose their own hierarchical or otherwise organizational scheme on them. Wildcard matches in LBM are performed using regular expressions or user-defined matching functions. See Section 7.4 for details on how LBM efficiently filters messages for wildcard receivers.
LBM Benefits. LBM's unstructured topic space frees developers to use non-hierarchical structures when appropriate. The power of regular expression matching adds flexibility to production systems.
Other systems need to manually configure information about sources into all interested receivers. An example of such information is the transport multicast address that a source uses.
LBM receivers use a built-in topic resolution mechanism to dynamically discover transport information that sources use.
LBM Benefits. Dynamic Transport discovery drastically simplifies the administrative tasks involved in the building and ongoing operation of a messaging system. It is easy to reconfigure LBM sources without having to reconfigure receivers.
Other messaging systems limit configuration choices hoping that by limiting them they will reduce the support effort needed to diagnose possible customer misconfiguration. Limited configuration choices often lead to assumptions and compromises that then result in sub-optimal performance or functional problems for some applications. Some of the many LBM configuration choices are covered in Section 2.4, Section 4.3, and Section 4.1.
LBM has been designed with many configuration options that can be used to optimize performance and functionality for most applications. A staff of professionals experienced in high-performance messaging systems is available to answer support questions on even the most demanding applications. See Section 5.4 for more on how LBM is supported.
LBM Benefit. LBM can be easily configured for the highest-possible performance in any application.
Other messaging systems include the topic string with every message payload sent. Transporting the topic string, especially with longer topics, may account for a substantial fraction of the effort required to deliver a message.
Other systems require that the topic string accompanying each arriving message be compared against a list of topics indicating receiver interest. At best, such designs involve hashing the topic string for each arriving message. At worst, it requires repeated string comparison.
LBM sources automatically assign a unique integer for each topic on each transport session. LBM receivers learn of the correspondence between topic strings and integers through topic resolution traffic. This traffic is carried separately from message transport sessions and sent only when needed. Receiver interest is checked for each arriving message by a simple integer hash that is independent of topic string length. See Section 7.2 for another benefit of the topic resolution process.
At wildcard receiver creation time, LBM evaluates the regular expression pattern against all known topics and again later against each new topic as it is discovered. LBM transparently creates a list of single-topic receivers that match the wildcard pattern and associates it with the wildcard receiver object. Filtering of arriving messages for wildcard receivers is thus performed with the integer hash speed of single-topic receivers. The regular expression is not evaluated for each arriving message.
LBM Benefits. LBM reduces the impact of long topic strings and the network bandwidth used to carry topic strings. It provides extremely high-performance receiver-side message filtering, even for wildcard receivers.
Messaging systems built with daemons or servers have a single point of failure in the daemon or server. They may use complex logic to detect and recover from failures of these components. Further, applications may need their own logic to detect and recover from failures of messaging infrastructure.
LBM has no daemons or servers and hence no need for internal failure detection or recovery logic. See Section 2.1 for details. Applications do not need logic to detect failure of the messaging system or recover from such failures. All message delivery code is implemented as shared libraries that dynamically link into the address space of the application. LBM is ready for messaging as long as the application is running.
LBM Benefits. You can use LBM to easily build and operate highly reliable systems.
Other messaging systems use a hot/cold failover model to achieve failure resilience. Some of these systems require a cold message source to monitor the operation of the current hot source (sending messages) and decide when to become hot. Some require that receivers monitor the current hot source and decide when to switch to the cold source. Either way, after a failure time must pass before messages can start flowing again. This failover latency is inherent in all hot/cold designs because only the passage of time without messages can trigger failover.
LBM offers an optional hot/hot failover model that has zero failover latency. Two or more sources remain hot at all times. An LBM hot failover receiver monitors message sequence numbers from all sources and delivers the first copy of each sequence number to arrive. Duplicate messages with the same sequence number from other sources are dropped instead of being delivered to the application.
This process does require enough receiver bandwidth for every message to arrive at least twice. However, use of features like multiple transport sessions (see Section 2.2) and a choice of filtering locations (see Section 2.5) often drastically reduce the arrival rate of irrelevant messages. Applying such features results in resilient LBM receivers that often use less bandwidth than even non-resilient receivers in competing systems.
LBM detects duplicate messages by using a simple integer hash, as is used for filtering irrelevant messages. See Section 7.4 for details.
LBM Benefits. Zero failover latency. Lower average message delivery latency since there's a latency race in the delivery of every message.
Other messaging systems often provide gateway daemons that encapsulate their reliable multicast inside TCP for transport across a WAN. (See Section 3.3 to learn why LBM does not require gateway daemons.) TCP decides when one gateway is allowed to send to its partner, but messages arrive at the gateway whenever a source decides to send them. The gateway is therefore required to add latency by queuing messages for transmission until TCP is willing to accept them. Queues and latency grow large whenever congestion on the WAN or slowness in the receiving gateway prevents messages from being sent.
29West generally discourages the use of gateway daemons because our reliable multicast protocol is designed for WAN use (see Section 3.3). However, LBM comes with a gateway process for use as required. Our gateway can use unicast UDP across a WAN link, thereby avoiding TCP's congestion control and flow control behavior, which is undesirable for latency-sensitive applications. See our whitepaper for more information on TCP latency.
Our gateway uses the event queue mechanism (see Section 4.3) so that administrators can set limits on the number of messages queued or their latency.
LBM Benefits. When gateways are required, LBM's gateway operation is stable and gateway latency can be easily controlled.
Gateway daemons in other messaging systems sometimes include logic to detect when other gateways are unreachable and to route around the failure. The messaging servers of other systems often include similar logic to route around server connectivity failures. The work of maintaining routing tables adds latency spikes whenever routing paths change.
LBM gateways leave the work of maintaining routing tables to network hardware, which is far more adept and flexible at detecting failures and routing around them with lower latency. We see no benefit in duplicating routing work in the messaging layer which the network layer already handles competently.
LBM Benefits. LBM provides lower latency when messaging paths through the network must change.
LBM uses a modern architecture designed to get the most out of hardware today and in the future. The daemonless and serverless design produces performance and reliability benefits while reducing administrative headaches. System Architects and Application Developers have the flexibility and features needed to handle all messaging tasks. LBM's reliable multicast protocol is designed for WAN use and free of problems like NAK storms that plague others. 29West has a pedigreed team focused on the needs of the financial markets.
The benefits discussed here are often best observed first hand. 29West offers fully-functional code for free evaluation. This is the best path we've found for assessing the benefits to your organization.
Copyright 2007 - 2008 29West, Inc.