ApsaraMQ for MQTT:Unified management of Cloud Storage Gateway (CSG) instances based on MQTT

Terms

Message Queuing Telemetry Transport (MQTT protocol)

The MQTT protocol is an industry-standard protocol for the Internet of Things (IoT) and mobile Internet, which is suitable for data transmission between mobile devices. ApsaraMQ for MQTT The MQTT protocol is supported by default.

CSG

CSG is a gateway service that can be deployed in your on-premises data center and in Alibaba Cloud. CSG uses Alibaba Cloud Object Storage Service (OSS) buckets as back-end storage devices. Moreover, by using low-cost virtual machines, CSG provides on-premises and off-premises applications with standard file storage services over the Network Attached Storage (NFS) and Common Internet File Systems (CIFS) protocols, and block storage services over the Internet Small Computer Systems Interface (iSCSI) protocol. For more information, see Cloud Storage Gateway.

MQTT broker

ApsaraMQ for MQTT An MQTT broker is a broker node that interacts with the MQTT protocol and is used to receive and forward messages.

MQTT client

An MQTT client is a mobile node that interacts with an MQTT broker. In this topic, it typically refers to a client that uses CSG.

ApsaraDB for RDS

A highly available and scalable online database service provided by Alibaba Cloud.

Log Service

Alibaba Cloud Log Service is used for auditing and tracing.

Background information

CSG is a storage service that helps you seamlessly integrate existing on-premises applications, IT infrastructure, and data storage with Alibaba Cloud. You can deploy virtual devices compatible with standard storage protocols on the premises and in the cloud. In this way, you can seamlessly connect on-premises storage applications and workload to Alibaba Cloud storage and computing services.

CSG can be deployed in the following two modes:

• Deploy and run CSG in your on-premises data center through virtual machines.
• Deploy and run CSG in your Elastic Compute Service (ECS) instance in the virtual network environment of Alibaba Cloud.

Under the two deployment modes, CSG instances use the intranet, especially virtual machines deployed in your on-premises data center. To ensure security, the CSG console cannot directly access non-public IP addresses. The communication of CSG instances deployed in Alibaba Cloud with the Alibaba Cloud console through public network IP addresses is also inappropriate because this may cause security hazards and vulnerability to attacks.

ApsaraMQ for MQTT enables the connection between the CSG console and CSG instances under these two deployment modes.

For more information about the architecture of CSG, see. Hybrid cloud storage deployment Shown in.

Solution architecture

ApsaraMQ for MQTT Message Queue for MQTT (MQTT) is used to push and collect messages. It can display the statuses of CSG instances in the CSG console by giving commands and collecting these statuses through message pushing and active pulling. Based on ApsaraMQ for MQTT in addition, the output API is used to manage each CSG instance. Solution architecture shows the overall architecture:

Highlights

The following describes the benefits of this scheme.

• Powerful service capabilities that can be automatically scaled are available.
• ApsaraMQ for MQTT The message transmission capability is infinitely scalable, allowing you to increase the number of CSG instances without compromising system capabilities.
• ApsaraMQ for MQTT Messages are pushed to more than a million devices in milliseconds, and no congestion occurs to control commands issued by CSG.
• ApsaraMQ for MQTT Encryption based on the Transport Layer Security (TLS) and Secure Sockets Layer (SSL) protocols is supported to relieve you from the worry about data leakage.
• All service nodes are highly available and stable.
• Multi-language SDKs make it convenient to develop MQTT brokers and applications.
• Both the internal network and public network support different network requirements.

Precautions

The preceding process briefly describes how to use ApsaraMQ for MQTT and ApsaraMQ for RocketMQ A unified cross-network management architecture is implemented. For more information about the SDK, see ApsaraMQ for MQTT and ApsaraMQ for RocketMQ document.

Where use ApsaraMQ for MQTT when constructing cross-network signaling, follow these principles for message type and parameter design:

• MQTT client ID mapping
  The MQTT protocol requires that each client have a globally unique client ID. A client ID consists of two parts concatenated with the @@@ separator. The final client ID must be unique and its total length cannot exceed 64 characters. The following describes the two parts of a client ID:

  • Prefix Group ID: The Group ID must be ApsaraMQ for MQTT apply for the console. We recommend that you classify your group IDs based on the Alibaba Cloud regions where the CSG console resides. For example, you can use different group IDs for China (Hangzhou) and China (Shanghai) so that problem regions and message queues can be easily located.
  • Device ID that is the suffix in a client ID: A device ID is generated by an application. Device IDs can be mapped to the IDs of CSG instances one by one to ensure global uniqueness.

  For more information about client IDs, see Terms.

• Topic name mapping

  Use ApsaraMQ for MQTT to send and receive messages, you must understand the publish /subscribe pattern of the MQTT Protocol. For more information, see agreement document and official documentation.

  The MQTT protocol is a message protocol based on the publish/subscribe pattern. The subscriptions and topics follow the directory tree format. Topics can be divided into parent topics and subtopics. The total length of a topic including the parent topic and subtopics cannot exceed 64 characters. The following describes the types of topics:

  • Parent topic: The topic at the first level of the directory tree is a parent topic. The parent Topic must be ApsaraMQ for MQTT can be used only after applying for permissions in the console. A Namespace is equivalent to a Namespace.
  • Subtopic: A topic under a level-1 topic in the directory tree in MQTT is a subtopic. You can specify subtopics as needed without having to apply for them.

  For more information about topics, see Terms.

  When designing a topic for sending and receiving messages, you must comply with the following principles:

  • Different parent topics are used for uplink messages and downlink messages. Uplink messages are messages sent from CSG instances to management and control services. Downlink messages are messages sent from management and control services to CSG instances.
  • Messages with different priorities or large size differences use different parent topics.

  For the interaction process described above, we recommend that you use ApsaraMQ for MQTT P2P messages provided by. P2P messages do not need to be subscribed, and therefore the sender can directly specify the peer end to receive them. For more information, see P2P Messaging model (MQTT).

• Parameter design for message sending and receiving
  The CSG console uses MQTT to simulate RPC calls. Therefore, the messages sent by the console directly use noreply without return values. Previous messages do not need to be received again. After receiving the error code, the console prompts you to issue the command again. You need to configure your MQTT client as follows:

  • Set the cleanSession parameter to true.

  For more information about cleanSession, see Terms.