Forward data to DataHub
Use the Data Forwarding feature of IoT Platform to forward device data to DataHub. You can then use stream computing engines such as Realtime Compute for Apache Flink and MaxCompute to process the data and generate real-time results, including charts, alerts, and statistics.
How it works
The Data Forwarding feature forwards device messages from an IoT Platform topic to a topic within a DataHub project. You can then use DataHub to clean, analyze, and archive the data. For more information, see What is DataHub?.
In the preceding figure:
-
Data source: For information about the supported topic message types, see Data formats.
-
SQL expression: Write an SQL expression to parse and process JSON data reported by devices. For the syntax of SQL expressions, see SQL expression.
NoteBinary data is forwarded directly without being parsed.
-
Data destination: The DataHub project and topic that receive device data.
Scenarios
Forward data from IoT Platform to DataHub for unified management:
-
Deliver data to downstream systems for analysis and archiving to build a clear data pipeline and unlock data value.
-
Combine DataHub with Realtime Compute to clean heterogeneous data from multiple sources into structured data in real time for further analysis.
-
Build a real-time data warehouse with layers for raw data, real-time details, and real-time summaries.
For more information, see DataHub Scenarios.
Limitations
-
IoT Platform supports forwarding data to DataHub. For detailed information about the supported regions, see Features by region.
-
You can use both the new and previous versions of the Data Forwarding feature to forward data to DataHub. For an example of how to use the new version, see Forward data to DataHub (new version).
-
The previous version of Data Forwarding does not support cloud gateway products and devices, or products and devices in MQTT-compliant instances. To forward data for these products and devices, you must use the new version of Data Forwarding. For more information, see Data Forwarding (new version).
-
For more limitations on forwarding device messages from an IoT Platform instance, see Limitations.
Prerequisites
-
You have created a DataHub project and a topic to receive data. For more information about how to use DataHub, see Project operations.
ImportantFor an Enterprise instance, the DataHub project must be in the same region as the instance.
-
Create a data forwarding rule and an SQL expression to process data. For more information, see Create a data forwarding rule.
Configure a data destination
-
Log on to the IoT Platform console.
On the Overview page, find the instance that you want to manage and click the instance ID or instance name.
-
In the left-side navigation pane, choose .
-
Click View next to the target rule. The Data Forwarding Rule page appears.
ImportantIf the Data Forwarding (New Version) page is displayed, click Back to Previous Version in the upper-right corner, then click View next to the target rule.
-
In the Forward Data section, click Add Operation.
-
In the Add Operation dialog box, set the parameters and click OK.

Parameter
Description
Operation
Select Send data to DataHub.
Region
Select the region where your DataHub project is located.
Project
Select a DataHub project.
You can click Create Project to go to the DataHub console and create a project. For more information, see Project operations.
Topic
Select the DataHub topic that receives data.
After you select a topic, the rules engine automatically obtains the topic's schema and maps the filtered data to it.
Note-
When you map data to the schema, use
${}. Otherwise, the system stores a constant in the table. -
The schema's data types must match the data types that the rules engine processes. Otherwise, the system cannot store the data.
You can click Create Topic to go to the DataHub console and create a DataHub topic.
Role
Select the RAM role that grants IoT Platform permission to write data to DataHub.
If no RAM role exists, click Create RAM Role to create a RAM role and authorization policy in the RAM console. Create a RAM role.
-
-
On the Data Forwarding page, find the rule and click Start in the Actions column.
Example
Build a device monitoring dashboard by using a big data platform
What to do next
-
View the operational logs of Device-to-Cloud Messages and Data Forwarding in the IoT Platform console. Navigate to the IoT Platform Logs tab on the page for your instance. For more information, see IoT Platform logs.
-
Log on to the DataHub console to view the data written to DataHub. For more information, see Quick Start (Synchronization example).
-
Synchronize data from the topic to other Alibaba Cloud products in real time or near real time to connect data across services. Supported products include MaxCompute (formerly ODPS), AnalyticDB for MySQL (ADS), ApsaraDB RDS, Tablestore, Object Storage Service (OSS), Elasticsearch, and Function Compute. For more information, see Data synchronization.
References
-
To connect a device to IoT Platform and report data, see Device connection guide.
-
On the instance details page, view the available resources for message forwarding TPS to ensure that device messages can be forwarded to DataHub. For more information, see View instance information and operational data. If the available resources are insufficient, upgrade the instance. For more information, see Upgrade an instance.
-
For frequently asked questions about DataHub, see DataHub FAQ.