Competing Consumers

Enable multiple concurrent consumers to process messages received on the same messaging channel. 

When to use this pattern?  

• The workload for an application can is divided into tasks that can run asynchronously.  

• Tasks are independent and can run in parallel  

• The volume of work is highly variable, requiring a scalable solution  

• The solution must provide high availability, and must be resilient if the processing for a task fails 

When NOT to use this pattern?  

• It’s NOT easy to separate the application workload into discrete tasks  

  • There’s a high degree of dependence between tasks  

• Tasks must be performed synchronously, and the application logic must wait for a task to complete before continuing  

• Tasks must be performed in a specific sequence. 

Problem to solve: Reading the message by a single consumer, use a consumer pool each of them will process different messages when there is a high volume of messages in the queue.

Considerations  

• Message Ordering  

• Design Services for Resiliency  

• Detect Poison Messages  

• Handling Results  

• Scaling the Messaging System  

• Ensuing Reliability of the Messaging System