Based on the services provided by Message Queue for MQTT, a unified framework can be adopted. Specifically, the Alibaba Cloud console can be used to manage CSG instances deployed in different modes, and to achieve secure and compliant communication between a public network and the internal network. This provides a consistent user experience.

Terms

Message Queuing Telemetry Transport (MQTT protocol)
The MQTT protocol is an industry-standard protocol for the Internet of Things (IoT) and mobile Internet, which is suitable for data transmission between mobile devices.Message Queue for MQTT The MQTT protocol is supported by default.
CSG
CSG is a gateway service that can be deployed in your on-premises data center and in Alibaba Cloud. CSG uses Alibaba Cloud Object Storage Service (OSS) buckets as back-end storage devices. Moreover, by using low-cost virtual machines, CSG provides on-premises and off-premises applications with standard file storage services over the Network Attached Storage (NFS) and Common Internet File Systems (CIFS) protocols, and block storage services over the Internet Small Computer Systems Interface (iSCSI) protocol. For more information, see What is CSG?.
MQTT broker
An MQTT broker is a broker node that interacts with the MQTT protocol and is used to receive and forward messages.
MQTT client
An MQTT client is a mobile node that interacts with an MQTT broker. In this topic, it typically refers to a client that uses CSG.
ApsaraDB for RDS
ApsaraDB for RDS (RDS) is a stable, reliable, and scalable online database service provided by Alibaba Cloud.
Log Service
Alibaba Cloud Log Service is used for auditing and tracing.

Background

CSG is a storage service that helps you seamlessly integrate existing on-premises applications, IT infrastructure, and data storage with Alibaba Cloud. You can deploy virtual devices compatible with standard storage protocols on the premises and in the cloud. In this way, you can seamlessly connect on-premises storage applications and workload to Alibaba Cloud storage and computing services.

CSG can be deployed in the following two modes:

  • Deploy and run CSG in your on-premises data center through virtual machines.
  • Deploy and run CSG in your Elastic Compute Service (ECS) instance in the virtual network environment of Alibaba Cloud.

Under the two deployment modes, CSG instances use the private network, especially virtual machines deployed in your on-premises data center. To ensure security, the CSG console cannot directly access non-public IP addresses. The communication of CSG instances deployed in Alibaba Cloud with the Alibaba Cloud console through public network IP addresses is also inappropriate because this may cause security hazards and vulnerability to attacks.

Message Queue for MQTT enables the connection between the CSG console and CSG instances under these two deployment modes.

Figure 1 shows the architecture of communication between a public network and the private network under the two deployment modes of CSG.

Figure 1. Hybrid cloud storage deployment

Architecture

Message Queue for MQTT is used to push and collect messages. It can display the statuses of CSG instances in the CSG console by giving commands and collecting these statuses through message pushing and active pulling. In addition, an output API operation is enabled based on the message pushing function of Message Queue for MQTTto manage every CSG instance. Figure 2 shows the overall architecture:

Figure 2. Architecture
Hybrid cloud storage gateway

Benefits

The following describes the benefits of this scheme.

  • Powerful service capabilities that can be automatically scaled are available.
  • The message transmission capability of Message Queue for MQTT is infinitely scalable, allowing you to increase the number of CSG instances without compromising system capabilities.
  • Message Queue for MQTT supports the capability of pushing messages to more than a million devices in milliseconds, and no congestion occurs to control commands issued by CSG.
  • Message Queue for MQTT supports encryption based on the Transport Layer Security (TLS) and Secure Sockets Layer (SSL) protocols to relieve you from the worry about data leakage.
  • All service nodes are highly available and stable.
  • Multi-language SDKs make it convenient to develop MQTT brokers and applications.
  • Both the private network and public networks are supported to meet your needs for various networks.

Notes

The preceding process briefly describes how to use Message Queue for MQTTand Message Queue for Apache RocketMQ (RocketMQ) to enable a unified cross-network management architecture. For more information about SDK description, see Message Queue for MQTT and Message Queue for Apache RocketMQ.

When using Message Queue for MQTTto construct a cross-network signaling channel, you must comply with the following principles for the message type and parameter design:

  • MQTT client ID mapping

    The MQTT protocol requires that each client have a globally unique client ID. A client ID consists of two parts concatenated with the @@@ separator. The final client ID must be unique and its total length cannot exceed 64 characters. The following describes the two parts of a client ID:
    • Group ID that is the prefix in a client ID: You need to apply for a group ID in the Message Queue for MQTTconsole. We recommend that you classify your group IDs based on the Alibaba Cloud regions where the CSG console resides. For example, you can use different group IDs for China (Hangzhou) and China (Shanghai) so that problem regions and message queues can be easily located.
    • Device ID that is the suffix in a client ID: A device ID is generated by an application. Device IDs can be mapped to the IDs of CSG instances one by one to ensure global uniqueness.

    For more information about client IDs, see Terms.

  • Topic name mapping

    To use Message Queue for MQTT for message sending and receiving, you need to understand the publish/subscribe pattern of the MQTT protocol. For more information, see Protocols and Message Queue for MQTT.

    The MQTT protocol is a message protocol based on the publish/subscribe pattern. The subscriptions and topics follow the directory tree format. Topics can be divided into parent topics and subtopics. The total length of a topic including the parent topic and subtopics cannot exceed 64 characters. The following describes the types of topics:
    • Parent topic: The topic at the first level of the directory tree is a parent topic. A parent topic must be applied for in the Message Queue for MQTTconsole before it can be used. After being approved, this parent topic is equivalent to a namespace.
    • Subtopic: A topic under a level-1 topic in the directory tree in MQTT is a subtopic. You can specify subtopics as needed without having to apply for them.

    For more information about topics, see Terms.

    When designing a topic for sending and receiving messages, you must comply with the following principles:

    • Different parent topics are used for uplink messages and downlink messages. Uplink messages are messages sent from CSG instances to management and control services. Downlink messages are messages sent from management and control services to CSG instances.
    • Messages with different priorities or a large gap in size use different parent topics.

    For the interaction process described in the preceding section, we recommend that you use point-to-point (P2P) messages provided by Message Queue for MQTT. P2P messages do not need to be subscribed, and therefore the sender can directly specify the peer end to receive them. For more information, see P2P messaging model.

  • Parameter design for message sending and receiving

    The CSG console uses MQTT to simulate RPC calls. Therefore, the messages sent by the console directly use noreply without return values. Previous messages do not need to be received again. After receiving the error code, the console prompts you to issue the command again. You need to configure your MQTT client as follows:
    • Set the cleanSession parameter to true.

    For more information about cleanSession, see Terms.