The data push node enables the results of data queries from other nodes in the Data Studio workflow to be sent to DingTalk groups, Lark groups, WeCom groups, Teams, and emails by setting up a new data push destination. This feature allows team members to stay updated with the latest data insights promptly.
Principle introduction
The data push node retrieves output parameters from ancestor nodes using Node Context Parameters. You can incorporate these output parameters into the push content using placeholders and send them to the specified destination. The data push node can source output parameters from various ancestor nodes, including SQL query nodes and assignment nodes.
-
After completing a query, an SQL query node produces an output parameter named
outputs, which can transmit the query or output results to descendant nodes. For details on configuring node context parameters, see Configure Node Context. -
To display the data in the push content using a Markdown component, use the format
${parameter name}within the Markdown content to reference the output parameters of ancestor nodes. -
To display the data in the push content using a table component, select the SQL query field name from the ancestor nodes in the table component's associated parameters to access the data results.
Prerequisites
-
Ensure that the DataWorks service is active. For more information, see Activate DataWorks Service.
-
Ensure that a DataWorks workspace has been created. For more information, see Create Workspace.
-
A workflow has been created in the DataWorks workspace.
-
Ensure access to a Serverless resource group, as only Serverless resource groups can execute this node task. To purchase and utilize a Serverless resource group, see Add and Use Serverless Resource Group.
Limits
-
There are data size limits when using the data push feature to send data to various destinations:
-
For DingTalk, the data size must not exceed 20KB.
-
For Lark, the data size must not exceed 20KB, and images must be smaller than 10MB.
-
For WeCom, each robot is limited to sending no more than 20 messages/minute.
-
For Teams, the push size must not exceed 28KB.
-
For email, each data push task supports adding only one email body. Once an email body is added, it cannot be added again. Additional restrictions are subject to the SMTP limitations of the email service provider.
-
-
Only DataWorks workspaces in the following regions can use the data push feature: China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), China (Chengdu), Hong Kong (China), Singapore, Malaysia (Kuala Lumpur), US (Silicon Valley), US (Virginia), Germany (Frankfurt).
Procedure
Step 1: Create ancestor nodes for the data push node
The data push node requires output parameters from the Context Parameters of ancestor nodes to push data. Therefore, ensure that the ancestor nodes include query nodes or assignment nodes before creating a data push node.
-
To push MaxCompute data, use an assignment node to query the data, generate output parameters in the scheduling configuration, and pass them to the data push node. For more information, see: MaxCompute Data Push
-
If the ancestor query node is from another data source, generate a node output parameter named
outputsin the ancestor node and pass it to the data push node. For more information, see: Best Practices: Configure Data Push Nodes in Data Development Workflow.
Create an SQL query node
Go to the DataStudio page.
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, select the desired workspace from the drop-down list and click Go to Data Development.
-
Double-click the workflow to access the canvas page, then click
to create an SQL node. -
Double-click the SQL node to edit the query.
NoteThe data push node does not support direct retrieval of query data from ODSP SQL. Please create an assignment node and configure the query SQL within it. For more information, see Configure Push Flow.
-
Click Schedule Configuration to open the schedule configuration panel and set up the schedule according to your business needs.
For detailed instructions on configuring Schedule Configuration, refer to Configure Basic Properties, Time Property Configuration Description, Configure Resource Properties, Configure Recurring Schedule Dependencies, and Configure Node Context.
-
In the Schedule Configuration panel, click , and then click Add Assignment Parameter to output the SQL query results as output parameters.
-
Click the save icon
to save the SQL query node.
Create an assignment node
Go to the DataStudio page.
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, select the desired workspace from the drop-down list and click Go to Data Development.
-
Double-click the created workflow to enter the canvas page of the workflow, and click
to create an assignment node. -
Double-click the assignment node to access its editing page.
The assignment node supports calculations using ODPS SQL, SHELL, and Python. For more information, see Assignment Node.
-
After configuring the assignment node, click
to save it.
Step 2: Create a data push node
-
Double-click the workflow to open the canvas page, then click
to create a data push node.Configuration Item
Configuration Content
Node Type
Select Data Push from the drop-down list.
Path
Select the same path as the node created in Step 1.
Name
Specify the name of the data push node based on your business requirements.
-
Double-click the data push node to enter its editing page.
-
Click Schedule Configuration. In , click Add to include the node created in Step 1 as a dependent ancestor node.
-
In the schedule configuration panel, under , select the Serverless resource group created after the data push node feature release on June 28, 2024. If the resource group was created before this date, submit a ticket to upgrade the resource group.
-
In the schedule configuration panel, after , click Add to include the output parameters from the query node created in Step 1 as the source for the data push node's input parameters. Then close the Schedule Configuration panel.
-
Set up the destination, title, and content for the data push on the editing page.
-
Choose the desired data push target from the Data Push Target dropdown list. If it's not listed, click Create Data Push Target in the bottom right corner of the dropdown to set up a new target.
Parameter
Description
Type
Supports DingTalk, Lark, WeCom, Teams, and email.
Object Name
Specify the name based on your business requirements.
Webhook
The webhook of DingTalk, Lark, WeCom robots, or Teams, and the SMTP of email need to be obtained on the corresponding target platform.
NoteFor more information about how to obtain the webhook URL of Lark, see Use the webhook trigger.
For more information about how to obtain the webhook URL of Teams, see Create incoming webhooks with Workflows for Microsoft Teams.
To create and manage a destination in DataService Studio:
-
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, select the desired workspace from the drop-down list and click Go to DataService Studio.
-
On the Service Development page, click
in the lower left corner to access the settings page. Switch to the Push Target Management tab, click Create Data Push Target to establish the target, and manage existing targets. For more information, see Create Webhook Data Push Target.
-
Title: Define the title according to your business requirements.
-
Content: You can include Markdown and tables, aligning with the data push service's capabilities. For more information, see Configure Push Content.
Note-
When the ancestor node is an SQL query node:
-
In the data push content, Markdown can directly use the SQL query field name as a placeholder
${field name}to access the output parameters of the ancestor node. -
Tables in the data push content can select the field name of the ancestor SQL query as an Associated Parameter to retrieve data.
-
-
When the ancestor node is an assignment node, the data push content should use the parameter name from the Input Parameters Of This Node as a placeholder
${input parameter name of this node}to access the output parameters of the ancestor node.
-
-
-
Click
to save the data push node.
Step 3: debug, run tasks, and publish
Once the data push flow is configured, test the workflow on the canvas page to ensure it runs correctly. After successful testing, submit and publish the task.
-
On the data push flow canvas editing page, click
to execute the workflow. -
Once all nodes in the data push flow display
, click 
to submit the successfully executed data push flow. -
In the Submit dialog box, select the nodes to be submitted and choose Ignore Warnings Of Inconsistent Input And Output.
-
Click Submit.
-
After successful submission, publish the flow nodes on the publish page. For more information, see Publish Task.
Best Practices
The data push node can be utilized in various scenarios such as simple push, merge push, script push, conditional push, and with MaxCompute. For more information, see Best Practices: Configure Data Push Nodes in Data Development Workflow.
What to do next
After successfully publishing the data push task, you can carry out maintenance operations on the task from the maintenance page. For more information, see Basic Maintenance Operations for Recurring Tasks.