Platform For AI:Global variables

Procedure

1. Configure a global variable.

   Open an existing pipeline, click a blank area of the canvas, and then create a global variable in the right-side panel.

   To reference a global variable, use the format ${variable_name}. The variable name must be 1 to 20 characters long, start with a letter, and contain only letters, underscores, and numbers. When you use a global variable for offline scheduling, you must define a scheduling parameter with the same name in DataWorks. The value you configure in PAI Designer applies only to online experiment runs.

2. In the parameter field where you want to use the global variable, enter ${variable_name}.

Example 1: Online pipeline

This example shows how to set a global variable para1 to rename the table column status to ifHealth and reference it in an SQL script.

1. Create a pipeline.

   After creating the pipeline, keep only the first two nodes and delete the others.

2. Click a blank area of the canvas and create a global variable in the right-side panel.

   Set the global variable name to para1 and its value to ifHealth.

3. Use the global variable.

   Modify the SQL Script component to reference the variable using ${para1}.

   In the SQL editor for the SQL Script-1 component, use ${para1} as the output column alias at the end of the SELECT statement. In the FROM clause, use ${t1} to reference the input source from the Read Table-1 component. Also, select the Automatically add CREATE TABLE statement checkbox. When this option is selected, the last statement must be a SELECT statement.

4. Run the pipeline.

   After the run is complete, right-click the SQL Script component and select View Data > Output of SQL Script. The column name status is changed to ifHealth.

Example 2: DataWorks offline scheduling

This example shows how to use the global variable gDate to filter data for a specific date in a pipeline. You then configure a matching scheduling parameter in DataWorks to run the pipeline as a scheduled job.

1. Create a test data table.

   In this example, create a table named dwtest in the MaxCompute console. For more information, see SQL Reference.

   The dwtest table has four columns: t1 (numeric, example values: 1.0, 0.0, 1.2, 0.1), t2 (numeric, example values: 1.1, 1.0, 0.0, 0.1), label (string, example values: A, B, C, D, E, F), and ds (date partition, example values: 20190519, 20190520). The table contains 10 rows of test data.

2. Create a pipeline and configure a global variable.

   1. Click a blank area of the canvas and create a global variable named gDate in the right-side panel.

      Set the variable value to 20190520.

   2. Configure the pipeline components.

      

      • Read Table: Set Table Name to the dwtest table.

      • SQL Script: Use the gDate global variable in the query.

        select * from ${t1} where ds=${gDate}

3. Run the pipeline.

   After the run is complete, right-click the SQL Script component and select View Data > Output of SQL Script. The output data is filtered based on the value of the gDate global variable.

   In the query results, the value in the ds column is 20190520, which matches the value of the gDate global variable. This confirms the global variable was passed successfully.

4. In the upper-left corner of the canvas, click Periodic Scheduling and then click Create Scheduling Node. This redirects you to DataWorks to configure offline scheduling. In the Create Node dialog box, specify a node name and click Confirm.

5. Select the PAI Designer experiment, and then click Properties in the right-side panel. For more information about the configuration, see node scheduling configuration.

   For this example, configure the following parameters and leave the others at their default settings:

   • Scheduling Parameters: Add a scheduling parameter named gDate and set its value to $bizdate.

   • Schedule: For the rerun property, specify that the node can be rerun regardless of the previous run's outcome.

   • Dependencies: Set the upstream dependency to the workspace root node.

6. Click the and icons in the toolbar. Follow the on-screen instructions to save and submit the node.

7. At the top of the page, click Operation Center to view the running status and operation logs of the task.

   You can also backfill data and run trial pipelines. For more information, see manage auto-triggered tasks.

Related documents

• Create a Pipeline

• node scheduling configuration

• manage auto-triggered tasks