All Products
Search
Document Center

IoT Platform:Configure a data forwarding rule

Last Updated:Jun 01, 2023

You can use the data forwarding feature of the rules engine of IoT Platform to forward data from a specified topic to other topics or other Alibaba Cloud services. This topic describes how to configure a data forwarding rule. To configure a data forwarding rule, perform the following steps: create a rule, write an SQL statement for data processing, specify a data forwarding destination, and then specify a destination for error messages if data forwarding fails.

Procedure

  1. Log on to the IoT Platform console.
  2. On the Overview page, select an environment, find the instance that you want to manage, and click the instance ID or instance name.

    Important

    This step is required only if Enterprise Edition instances are available. If the Enterprise Edition instances are unavailable in the region that you selected, skip this step. For information about supported regions and instances, see Overview.

    Overview
  3. In the left-side navigation pane, choose Message Forwarding > Data Forwarding.
  4. On the Data Forwarding page, click Create Rule.

    Important

    If the Data Forwarding page of the latest version appears, click Back to Previous Version in the upper-right corner of the page. When the Data Forwarding page of the previous version appears, click Create Rule.

  5. Configure the parameters and click OK. The following table describes the parameters.

    Parameter

    Description

    Rule Name

    The name of the data forwarding rule. The name must be 1 to 30 characters in length, and can contain letters, digits, underscores (_), and hyphens (-).

    Data Type

    The format of the data that you want to process by using the rule. Valid values: JSON and Binary.

    Note
    • The data forwarding feature processes data based on topics. You must select the format of the data in the topics that you want to process.

    • If you select Binary, the rule cannot be used to process messages from basic communication topics or Thing Specification Language (TSL) communication topics. The rule also cannot be used to forward data to Tablestore (OTS) or ApsaraDB RDS.

    Rule Description

    The description of the rule.

  6. After the rule is created, the rule details page appears. Write an SQL statement to process messages, specify a data forwarding destination, and then specify a destination to which error messages are forwarded.

    1. Click Write SQL Statement to write an SQL statement. The SQL statement is used to process message fields.

      For more information about how to write SQL statements, see SQL statements and Functions.

      Parameter

      Description

      Rule Query Expression

      The SQL statement. The SQL statement is automatically generated based on the values that you specify for the Field, Topic, and Conditions (Optional) parameters.

      Field

      The field of the message to be processed. This parameter follows the SELECT keyword in the SQL statement.

      For example, if you enter deviceName() as deviceName, the deviceName field in the message is selected. For more information about the functions that can be used in the field, see Functions.

      Note

      The data of basic communication topics and TSL communication topics is in the Alink JSON format. Before data is forwarded to the rules engine, the data is parsed based on the corresponding TSL model. For more information about data parsing, see Data forwarding procedure. For more information about the formats of parsed data, see Data formats. When you write an SQL statement, specify the fields based on the format of parsed data.

      Topic

      The source topic of the message to be processed. This parameter follows the FROM keyword in the SQL statement. For more information about optional topics, see the following Topics table.

      Important

      If you set the Data Type parameter of the rule to Binary, set this parameter to Custom.

      Condition

      The trigger condition of the rule. This parameter follows the WHERE keyword in the SQL statement.

      Table 1. Topics

      Topic

      Description

      References

      Custom

      The topic that is used to forward data in custom formats. The format of this topic is the same as the format of a custom topic. Format: /${productKey}/${deviceName}/user/${TopicShortName}.

      ${TopicShortName} specifies a custom topic category, which is the suffix of the custom topic.

      The value can contain wildcard characters, including plus signs (+) and number signs (#).

      • All equipment (+): indicates all devices of the specified product.

      • /user/#: indicates all topics of the specified device.

      Use custom topics for communication

      Device Status Change Notification

      The topic that is used to forward notifications when the status of a device changes between online and offline. Format: /as/mqtt/status/${productKey}/${deviceName}.

      Submit device status

      TSL Data Reporting

      The following topics are provided:

      • /${productKey}/${deviceName}/thing/event/property/post: This topic is used to forward device properties.

      • /${productKey}/${deviceName}/thing/event/${tsl.event.identifier}/post: This topic is used to forward device events.

      • /${productKey}/${deviceName}/thing/property/batch/post: This topic is used to forward device properties in batches.

      • /${productKey}/${deviceName}/thing/event/batch/post: This topic is used to forward device events in batches.

      • /${productKey}/${deviceName}/thing/downlink/reply/message: This topic is used to forward messages that a device returns as responses to IoT Platform commands.

      The following topics are used to submit raw device data:

      • /sys/${productKey}/${deviceName}/thing/event/property/post: This topic is used to submit device properties.

      • /sys/${productKey}/${deviceName}/thing/event/${tsl.event.identifier}/post and /sys/${productKey}/${deviceName}/thing/event/${tsl.functionBlockId}:{tsl.event.identifier}/post: These topics are used to submit device events.

      • /sys/${productKey}/${deviceName}/thing/event/property/batch/post: This topic is used to submit device properties and events in batches.

      Device Changes Throughout Lifecycle

      The topic that is used to forward messages when a device is created, deleted, disabled, or enabled. Format: /${productKey}/${deviceName}/thing/lifecycle.

      Submit lifecycle changes

      Sub-Device Data Report Detected by Gateway

      The topic that is used to submit and forward the information about a new sub-device when a gateway detects the sub-device. This topic is specific to gateways. Format: /${productKey}/${deviceName}/thing/list/found

      Submit information about detected sub-devices

      Device Topological Relation Changes

      The topic that is used to forward notifications when topological relationships between sub-devices and the gateway are created or deleted. This topic is specific to gateways. Format: /${productKey}/${deviceName}/thing/topo/lifecycle.

      Device Topological Relation Changes

      /sys/${productKey}/${deviceName}/thing/topo/change: This topic is used to submit device data.

      Notify gateways of changes of topological relationships

      Device tag change

      The topic that is used to forward messages when device tags are changed. Format: /${productKey}/${deviceName}/thing/deviceinfo/update.

      Submit device tag changes

      /sys/${productKey}/${deviceName}/thing/deviceinfo/update: This topic is used to submit device data.

      Submit tags

      TSL Historical Data Reporting

      The following topics are provided:

      • /${productKey}/${deviceName}/thing/event/property/history/post: This topic is used to forward historical properties.

      • /${productKey}/${deviceName}/thing/event/${tsl.event.identifier}/history/post: This topic is used to forward historical events.

      /sys/${productKey}/${deviceName}/thing/event/property/history/post: This topic is used to submit historical TSL data.

      Device properties, events, and services

      Device status notification

      The following topics are provided:

      • /${productKey}/${deviceName}/ota/upgrade: This topic is used to forward over-the-air (OTA) update results.

      • /${productKey}/${deviceName}/ota/progress/post: This topic is used to forward the progress of updates.

      The preceding topics correspond to the following topic: /ota/device/progress/${productKey}/${deviceName}. This topic is used to submit update progress.

      Submit the update progress to IoT Platform

      Submit a module version number

      The topic that is used to forward messages when the version number of an OTA module for a device is changed. Format: /${productKey}/${deviceName}/ota/version/post.

      Submit OTA module versions

      The preceding topic corresponds to the following topic: /ota/device/inform/${productKey}/${deviceName}. This topic is used to submit the version number of an OTA module.

      Submit OTA module versions to IoT Platform

      Batch status notification

      The topic to which IoT Platform sends messages when the status of an OTA update batch changes. Format: /${productKey}/${packageId}/${jobId}/ota/job/status.

      Submit the status data of OTA update batches

      Job Event

      /sys/uid/${uid}/distribution/${jobId}/lifecycle: The topic that is used to forward notifications when the status of a device job changes.

      Note

      The name of the instance migration task is the same as the name of the product whose data you want to migrate.

      Submit the status data of data migration tasks for instances

    2. In the Forward Data section, click Add Operation to specify a data forwarding destination. For more information about how to configure the data forwarding feature, see the related topics in the Data forwarding examples directory.

      Note

      You can create up to 10 data forwarding operations for each rule.

      If data forwarding fails due to exceptions in the destination cloud service, IoT Platform performs one of the following operations:

      • When IoT Platform forwards data to cloud services, such as Message Queue for Apache RocketMQ, ApsaraDB RDS, and ApsaraDB for Lindorm, access to the cloud services may fail due to resource changes. In this case, IoT Platform stops forwarding data and changes the status of the related rule to Abnormal. You must specify a new destination for data forwarding.

      • For other exceptions, IoT Platform retries three times at intervals of 1 second, 3 seconds, and 10 seconds. The retry policy may vary based on the scenario. If all retries fail, the message is discarded. If your business has high requirements for message reliability, you can add an error operation and forward error messages to other cloud services.

    3. In the Forward Error Data section, click Add Error Operation. Then, configure the parameters to forward error messages to the specified destination after all retries fail.

      Important
      • You can add only one error-handling operation for each rule.

      • A normal operation and an error-handling operation cannot forward messages to the same destination. For example, normal data and error data cannot be forwarded to OTS at the same time.

      • If an error message fails to be forwarded, no retry is performed.

      • Error messages are generated only if the rules engine fails to forward data due to issues of other cloud services.

      If a message fails to be forwarded to a cloud service, IoT Platform retries to forward the message. If the retry fails, an error message is forwarded based on the error operation that you specify for data forwarding.

      Sample error message:

      {
         "ruleName":"",
         "topic":"",
         "productKey":"",
         "deviceName":"",
         "messageId":"",
         "base64OriginalPayload":"",
         "failures":[
          {
           "actionType":"OTS",
           "actionRegion":"cn-shanghai",
           "actionResource":"table1",
           "errorMessage":""
          },
          {
           "actionType":"RDS",
           "actionRegion":"cn-shanghai",
           "actionResource":"instance1/table1",
           "errorMessage":""
          }
         ]
      }

      The following table describes the parameters of error messages.

      Parameter

      Description

      ruleName

      The name of the monitoring rule.

      topic

      The source topic of the message.

      productKey

      The ProductKey of the product.

      deviceName

      The DeviceName of the device.

      messageId

      The ID of the message that is sent from IoT Platform.

      base64OriginalPayload

      The Base64-encoded raw data.

      failures

      The error details. Multiple errors may occur.

      actionType

      The type of the failed operation.

      actionRegion

      The region where the error occurred.

      actionResource

      The destination service in which the error occurred.

      errorMessage

      The error message.

  7. Go to the Data Forwarding page. Find the rule that you configured and click Start in the Actions column. After the rule is enabled, data is forwarded based on the rule.

    You can also perform specific operations on the rule. The following table describes the operations.

    Operation

    Description

    View

    Click View in the Actions column to go to the Data Forwarding Rule page. Then, modify the settings of the rule. For example, you can change or delete the data source topic or data forwarding destination.

    Delete

    Click Delete in the Actions column to delete the rule.

    Important

    You cannot delete a rule that is in the Running state.

    Stopped

    Click Stop in the Actions column to disable the rule.

    Warning

    If one or more of your applications need to use the device data that is forwarded by using a rule but the rule is deleted or disabled, or the data forwarding destination of the rule is deleted, your services may become unavailable. As a result, your business may be interrupted. Proceed with caution.

What to do next

On the rule details page, you can view the status of the rule, check whether a data forwarding destination is configured, and check whether data is forwarded to a data forwarding destination as expected.Data forwarding destination