Migrate data from AWS SQS or AWS SNS to Alibaba Cloud Simple Message Queue (SMQ, formerly MNS) - Simple Message Queue (formerly MNS)

Simple Message Queue (SMQ, formerly MNS) offers seamless data migration from services like AWS Simple Queue Service (SQS) or AWS Simple Notification Service (SNS). This topic describes the differences between SMQ, SQS, and SNS based on features, limits, and SDKs. It also describes the process of migrating data from SQS or SNS to SMQ.

Background information

SMQ, SQS, and SNS have similar features. SMQ provides the queue-based messaging model and the topic-based messaging model. SQS provides the queue-based messaging model. SNS provides the topic-based messaging model.

Comparison of features

The following table compares the features of SMQ, SQS, and SNS by queue-based messaging model and topic-based messaging model.

Model	Feature	SMQ	Amazon SQS/Amazon SNS
Queue-based messaging model	Message lifecycle	Supported. The message state machine has the following states: Active Inactive Deleted Expired	Supported. The message state machine has the following states: Active Inactive Deleted Expired
	Custom retention period for messages	Supported	Supported
	Message delay (scheduled period)	Supported	Supported
	Visibility timeout period	Supported	Supported
	Maximum message length	Supported	Supported
	Dead-letter queue	Supported	Supported
	FIFO queue (sequencing)	Supported	Supported
Topic-based messaging model	Data filtering	Data filtering by tag	Data filtering by JSON policy
	Queue subscription	Supported	Supported
	HTTP subscription	Supported	Supported
	Text message subscription	Supported	Supported
	Email subscription	Supported	Supported
	Mobile push	Supported	Supported
	Function compute	Supported	Supported
	Dead-letter queue	Supported	Supported
	FIFO topic (sequencing)	Supported	Supported

Comparison of limits

The following table compares the limits of SMQ, SQS, and SNS by queue-based messaging model and topic-based messaging model.

Model	Feature	SMQ	Amazon SQS/Amazon SNS
Queue-based messaging model	Queue naming	Queue names are case-sensitive. Each name must be 1 to 120 characters in length.	Queue names are case-sensitive. Each name must be 1 to 80 characters in length.
	Message retention period	7 days	14 days
	Long polling interval	Valid values: 0 to 30. Default value: 15. Unit: seconds.	Valid values: 0 to 20. Unit: seconds.
	Maximum message payload	64 KB. If you want to increase the maximum message payload, submit a ticket.	256 KB
	Visibility timeout period	Valid values: 1 to 43200. Default value: 30. Unit: seconds.	Valid values: 1 to 43200. Unit: seconds. No default value is defined.
	Queue ID	Each queue name is unique for a region. Each message ID is globally unique. The name of each receipt handle is a custom string.	Each queue name is unique for a region. Each message ID is globally unique. The name of each receipt handle is a custom string.
	Message delay	The DelaySeconds parameter specifies the message delay. Valid values: 0 to 604800. Unit: seconds.	The DelaySeconds parameter specifies the message delay. Valid values: 0 to 604800. Unit: seconds.
	Maximum number of accumulated messages	No limit	No limit
	Maximum queries per second (QPS)	No limit is imposed on the queue resources used by users. The QPS of queue-based messaging and topic-based messaging is limited, up to 20,000 for each Alibaba Cloud account in each region. For more information, see Throttling policy.	No limit
Topic-based messaging model	Topic name	Each topic name must be 1 to 120 characters in length. Topic names are not case-sensitive.	Each topic name must be 3 to 64 characters in length. Topic names are case-sensitive.
	Maximum message length	64 KB	256 KB
	Retry policy	Backoff mode: The system performs three retries. The interval between two consecutive retries ranges from 10 seconds to 20 seconds. Exponential decay mode: The system performs 176 retries within only one day.	Backoff mode: The system performs three retries. The interval between two consecutive retries is 20 seconds. Exponential decay mode: You can change the number of retries.
	Maximum number of subscriptions to a single topic	100. If you want to increase the limit, you can submit a ticket.	12500000
	Maximum QPS	No limit is imposed on the queue resources used by users. The QPS of queue-based messaging and topic-based messaging is limited, up to 20,000 for each Alibaba Cloud account in each region. For more information, see Throttling policy.	300 QPS per topic or 10 MB/s. The system starts to count when a topic receives the first query.

Comparison of SDKs

SDK	Alibaba Cloud SMQ	Amazon SQS	Amazon SNS
API operations	API reference	API reference	API reference
HTTP SDK	SDK for Java, SDK for Python, SDK for C#, SDK for PHP, SDK for C++, SDK for Go, and SDK for JMS	SDK for Java, SDK for JavaScript, SDK for PHP, SDK for Python, SDK for Ruby, and SDK for .NET	SDK for Java, SDK for JavaScript, SDK for PHP, SDK for Python, SDK for Ruby, and SDK for .NET

Migration process

To ensure that your applications are smoothly migrated from SQS or SNS to SMQ, we recommend that you synchronize queues, topics, and subscribed metadata between the services. Then, you must enable the dual-read and dual-write policies on the application side to migrate data. The following figure shows the process of how data is migrated from SQS to SMQ. Process:

Simple Message Queue (formerly MNS):Best practice for migrating data from AWS SQS/SNS to SMQ

Background information

Comparison of features

Comparison of limits

Comparison of SDKs

Migration process

Stage 1: The existing workloads produce or consume messages based on SQS.

Stage 2: The consumer services simultaneously consume messages from both SQS and SMQ.

Stage 3: The producer services are connected to SMQ.

Stage 4: The producer services are disconnected from SQS after the existing messages are consumed.