How RabbitMQ Can Improve Your Software Design and Architecture
I had been searching and testing various messaging systems up until I discovered RabbitMQ, a very unique software that I highly recommend as a subsystem in your software solutions due to the fact that it’s matured and battle-tested. Unfortunately, while RabbitMQ is a robust system with many different uses, I have seen many people used it for the wrong reasons and created systems around RabbitMQ that did not reflect its optimal usage. RabbitMQ is a feature rich and reliable messaging system as long as you know how to use it. This guide will show you the optimal usage of RabbitMQ.
RabbitMQ is a Reliable Messaging Platform
RabbitMQ is an Open Source software with a commercial-friendly license. I currently use it in many of the production systems that I manage. It has become an invaluable tool in my solutions toolbox. RabbitMQ is very effective when used at integrating multiple heterogeneous systems by providing client libraries for multiple languages and frameworks, such as Java, Spring Framework, Scala, Groovy and Grails, Clojure, Ruby, Python, .Net, PHP, Perl, C++, Node.js, Go, Erlang, Haskell and many more. Instagram uses it to process millions of transactions daily. The flexibility of RabbitMQ, due to its implementation of the AMQP protocol, allows you to implement the following:
- Reliable messaging using store and forward queues
- Horizontal scaling through task distribution
- Building scalable RPC services
- Decoupled middleware through Pub/Sub and RPC services
Use it as a store and forward messaging system
One of the key features of RabbitMQ is its ability to reliably move messages from one system to another. RabbitMQ accomplishes this by using message queues. RabbitMQ also has the ability to decouple the publisher and consumer. The publisher can put messages on the message queue while the consumer is disconnected. Likewise, the consumer can consume the messages from the message queue without the publisher being present. In such case, RabbitMQ will store the messages until it can be consumed. With RabbitMQ, you no longer have to worry about your codes failing to reliably deliver messages from your system to another system. The number of messages that RabbitMQ can store is only limited by the hard drive space available for its usage. This is a very powerful system to use as it guarantees that messages will never be lost if the workflow is implemented correctly.
Use it as a task distribution system
Another great way to use RabbitMQ is to scale out your workload. RabbitMQ’s main job is to manage message queues and reliably move messages from one system to another. By creating multiple consumers that are consuming from the same message queue, RabbitMQ is able to round-robin dispatch the messages to each consumer. Each message in the message queue represents a description of a particular task that needs to be performed. This is why the message queues are repurposed as task queues. By adding multiple consumers, you can linearly scale out the workload and reduce the overall time needed to process complete tasks by manyfold. For example, if it takes 60 seconds for one consumer to process 10 tasks, however, by adding another consumer, we can cut down the total time to 30 seconds or in others words, half the time. This is because each consumer now processes only 5 messages. This is the ideal situation, but in order to use task queues properly, you must have the right configuration.
Use it as a RPC system
RPC is short for Remote Procedure Calls and a very common communication protocol in client/server architectures. RPC communication can also be used in RabbitMQ but leveraging two message queues; one queue to relay the requests from the client to the server and the other queue to relay the response from the server back to the client. RPC implementations in client/server systems suffer from tight-coupling and are prone to failure when the RPC service is not available. RabbitMQ can mitigate this single point of failure by allowing you to create multiple instances of the same RPC service. With multiple RPC service instances, when one instance fails or goes offline, the other instances can continue to process transactions without interruption.
Use it as a Pub/Sub system
In addition to reliably passing messages from one system to another, RabbitMQ can also broadcast the same message to multiple systems without having to wastefully clone the message. Through the uses of RabbitMQ Exchanges, you now have the power to broadcast the message to multiple systems and the only limitation is physical resource. RabbitMQ even makes broadcasting message more efficiently by providing you with three types of Exchanges: fanout, direct, and topic. Learning to take advantage of each of the three types of exchanges will make you an enterprise messaging expert.
RabbitMQ will save you a lot of work
In the past, I had tried many messaging systems such as IBM MQSeries, Apache ActiveMQ, Tibco Rendezvous and etc.. By far, RabbitMQ is the best messaging system in terms of features, reliability, and cost (who doesn’t like free softwares). RabbitMQ has been used as messaging system of choice in all my production systems. Unfortunately, many developers have not received the right training on how to properly implement the software to reap its full benefits. This resulted in developers blaming RabbitMQ for their failed projects when the fact is that they didn’t use RabbitMQ correctly. Sometimes when we have a hammer, all we see are nails. There will always be some cases where RabbitMQ will excel and other cases when some other technologies are better suited to solve your problems.