According to industry research findings, the premium Application-to-person (A2P) and Person-to-Application (P2A) messaging market size is expected to grow from ~US$56 billion in 2016 to ~US$72 billion by 2021, at a Compounded Annual Growth Rate (CAGR) of 5.23% from 2016 to 2021. Enterprise messaging system help organizations build scalable, distributed, loosely coupled, asynchronous, fault tolerant and high available applications. It is used to integrate applications developed in distributed technology and platform. It is also used to build micro service architecture with services communicating via messages. Popular messaging systems are IBM MQ, TIBCO, ActiveMQ, RabbitMQ. The main advantage of messaging over REST/SOAP API are:
- Systems can function without the two interacting applications available at the same time. Messaging system store and relay messages to receiving systems once they are available.
- Absorb peak load without the worker application scale for the maximum load. Messaging system buffers the messages and worker application process the messages at its own pace.
Challenges in enterprise messaging system
Here are the issues of managing high available, scalable messaging system:
- One time capital investment for hardware to support peak load
- One time capital investment for license and support cost
- System unavailability for system upgrade/patch that runs messaging system
- Costly system upgrade as messaging product reach end of support cycle
Enterprise messaging system in AWS Cloud
Amazon Simple Queue Service (SQS) is a managed service which is scalable, distributed message queuing service with one message producer and one message consumer system. It solve the challenges of managing enterprise messaging system by taking management overhead on itself and let organization use it for connecting application asynchronously. AWS Simple Notification Service (SNS) is another scalable, distributed publish/subscribe service for the use cases where one message producer and multiple message consumer system need to communicate asynchronously. SQS and SNS support TLS 1.2 security standards. SQS and SNS are integrated with AWS Cloud Watch for automated central monitoring. AWS also recently announced SQS support for JMS API. With this enhancement on premise application using massage brokers can be migrated to AWS cloud without changing application architecture or code.
Architecting Scalable application with SQS
SQS uses asynchronous replication of data amongst different storage devices. This makes for making it available and durable even if some hardware fails or network segment is not reachable. This has some side effect; the messages are not granted to be delivered in the same order and duplicates can be introduced. Wipro has developed custom reusable module to manage sequence and handle duplicate messages.
Automated tools can be used or built to speed up migration of an application using messaging system. These tools identify need to change code and configuration for application migration do the changes to large extent. Like any other application migration, migrating from messaging system to SQS also needs some code/configuration change. These tools can identify the required changes for migration of application using JMS. It also change the code to call AWS SQS API from existing code automatically to a large extent.
SQS performance can be enhanced by adding a framework for pooling SQS connections and reusing it. The framework creates a pool of limited number of connection and uses those to fulfil connectivity need for large number of application thread. This helped improving performance of the application and achieve target throughput with smaller infrastructure. This reduced cost as well.