13 RabbitMQ Facts I Wish I Knew From the Start

rabbitmq

Maybe you’re working on an ETL job that needs to make use of massively parallel processing. Or maybe there are some aspects of your application that need to be decoupled into asynchronous micro-services. RabbitMQ is a feature-packed, open-source AMQP broker that can help with the above problems and many more. Written in Erlang (though you don’t need to know Erlang to use it) and built over TCP, it’s sure to hit the performance needs for your application. If you need proof, read about how Pivotal and Google collaborated to achieve one million messages per second using RabbitMQ.

While it’s pretty easy to get started with RabbitMQ, there are many facts about how the broker is put together that I’ve learned over the years that would have saved me significant time and headaches to know from the beginning. I’d like to convey those in this article, which assumes a basic knowledge of RabbitMQ such as creating connections, channels, exchanges, queues and publishing/subscribing.

Continue reading

The Right Amount of Randomness: Creating Unique Identifiers

Screen Shot 2015-11-11 at 10.04.28 AM

At Image.wiki we need to be able to quickly assign a unique identifier to each image that users upload to our system.  Reconciling identical images is an important task but not the subject of this blog – we must treat every image we receive like a separate entity even if the file already exists in our database, as it may be accompanied by unique metadata. This seems like a trivial topic – just find an algorithm to spit out some random numbers! But there are many considerations and no pre-baked solutions out there.  And once we start issuing these IDs they’ll become a vital part of our system, referenced in many places and influencing the way we structure our data.  This is not something we want to re-do in the future. Continue reading