The workflow feature, also called service orchestration, provides a composite API service. This topic describes the benefits of workflows and how to use workflows.

Prerequisites

  • DataWorks Enterprise Edition or higher is activated. For more information, see DataWorks advanced editions.
  • The DataWorks workspace is created in the China (Shanghai) region.

Background information

In DataService Studio, a workflow is represented as a directed acyclic graph (DAG). By dragging nodes to a DAG, you can arrange APIs and functions in a serial, parallel, or branch structure based on the business logic.

When you run a workflow to call APIs, DataWorks runs the nodes in the workflow in sequence, passes parameters among the nodes, and automatically changes the status of each node. The workflow feature simplifies the process of calling multiple APIs or functions and reduces the cost of development and maintenance. This allows you to focus on business development.

The workflow feature provides the following benefits:
  • Reduced cost of combining multiple APIs

    By dragging nodes to a DAG, you can arrange APIs and functions in a serial, parallel, or branch structure without writing code. This reduces the cost of developing APIs.

  • Higher performance in calling APIs and functions

    A workflow allows you to call multiple APIs and functions in a container. Compared with writing code to call APIs and functions, the workflow feature reduces the latency of calling APIs and functions.

  • Serverless architecture

    The workflow feature adopts a serverless architecture. A serverless architecture supports automatic resource scaling based on business needs. You can focus on the business logic, without the need to worry about the runtime environment.

Obtain values of request and response parameters

DataService Studio uses JSONPath to obtain parameter values. JSONPath is a query language that allows you to extract data from JSON files. For more information, see JSONPath.

For example, three nodes are run in the following order: A, B, and then C. Node C needs to use the response parameters of nodes A and B.
  • Response parameter of node A: {"namea":"valuea"}

    Expression for obtaining the value of the response parameter of node A: ${A.namea}

  • Response parameter of node B: {"nameb":"valueb"}

    Expression for obtaining the value of the response parameter of node B: $.nameb or ${B.nameb}

The built-in start node provides request parameters for the whole workflow. Assume that a request parameter of a workflow is {"namewf":"valuewf"}. All nodes of the workflow can obtain the value of the request parameter by using the ${START.namewf} expression.

Set parameters

Request parameters:
  • If you do not specify a value for a request parameter of a node, DataService Studio obtains the value of the same parameter in the first layer of the JSON string returned by the parent node, and assigns the value to the request parameter. If no value is specified for a request parameter of the first node, DataService Studio obtains the value of the same parameter in the request parameters of the workflow.
  • If you specify a value for a request parameter, DataService Studio uses the value that you set.
  • If you need to use the value of the specified parameter returned by a specified ancestor node, obtain the value by using a JSONPath expression.
Common JSONPath expressions for obtaining parameter values:
  • $.: obtains response parameters of the parent node.
  • $.param: obtains the value of the param parameter returned by the parent node. To allow you to obtain response parameters of any ancestor nodes, DataService Studio enhances JSONPath expressions.
  • ${NODEID1}: obtains response parameters of the node whose ID is NODEID1.
  • ${START}: obtains request parameters of the workflow, which are response parameters of the start node.
  • ${NODEID1.param}: obtains the value of the param parameter returned by the node whose ID is NODEID1.
JSONPath expressions for setting response parameters of a node:
  • $.: sets response parameters of the current node.
  • $.param: sets the param parameter to be returned by the current node.
  • ${NODEID1.param}: sets the param parameter returned by the node whose ID is NODEID1.

Example

Add a connection before you create and use a workflow. For more information, see Configure a connection. In this example, a MySQL connection is used.

  1. Go to the DataService Studio page.
    1. Log on to the DataWorks console.
    2. In the left-side navigation pane, click Workspaces.
    3. In the top navigation bar, select the region where the target workspace resides. Find the target workspace and click DataService Studio in the Actions column.
  2. Register an API.
    In this example, create an API by using the registration method.
    1. On the Service Development tab, move the pointer over Create icon and choose API > Register API.
      You can also click the target business process, right-click API, and then choose New > Register API.
    2. In the Register API dialog box, set the parameters as required. For more information, see Register an API.
    3. Click OK.
  3. Register a function.
    1. On the Service Development tab, move the pointer over Create icon and choose Function > Create Python Function.
      You can also click the target business process, right-click Function, and then choose New > Create Python Function.
    2. In the Create Python Function dialog box, set the parameters as required. For more information, see Manage functions.
    3. Click OK.
    4. On the configuration tab of the function, enter the following code in the Edit Code section:
      # -*- coding: utf-8 -*-
      # event (str) : in filter it is the API result, in other cases, it is your param
      # context : some environment information, temporarily useless
      # import module limit: json,time,random,pickle,re,math
      import json
      def handler(event,context):
          # load str to json object
          obj = json.loads(event)
          # add your code here
          # end add
          return obj
    5. In the Environment Configuration section, set the Memory and Function Timeout parameters.
    6. Click Save icon in the toolbar.
  4. Create a workflow.
    1. On the Service Development tab, move the pointer over the Create icon and select Service Orchestration.
      You can also click the target business process, right-click Service Orchestration, and then select Service Orchestration.
    2. In the Service Orchestration dialog box, set the parameters as required.
      Service Orchestration dialog box
      Parameter Description
      API Name The name of the workflow. The name must be 4 to 50 characters in length and can contain letters, digits, and underscores (_). It must start with a letter.
      API Path The path for storing the workflow, such as /user.
      Note The path can be up to 200 characters in length and can contain letters, digits, underscores (_), and hyphens (-). It must start with a forward slash (/).
      Protocol The protocol used by the API to be arranged in the workflow. Valid values: HTTP and HTTPS.

      If you need to call the API by using HTTPS, you must bind an independent domain name to the API in the API Gateway console after the API is published to API Gateway. In addition, you must upload a Secure Sockets Layer (SSL) certificate in the API Gateway console. For more information, see Enable HTTPS for an API operation.

      Request Method The request method used by the API. Valid values: GET and POST.
      Response Content Type The format of the data returned by the API. Set the value to JSON.
      Visible Range The range of users to whom the workflow is visible. Valid values:
      • Work Space: The workflow is visible to all members in the current workspace.
      • Private: The workflow is visible only to its owner and permissions on the API cannot be granted to other users.
        Note If you set this parameter to Private, other members in the workspace cannot view the workflow in the workflow list.
      Label The tags to be attached to the workflow. For more information, see Manage API tags.
      Note A tag can be up to 20 characters in length and can contain letters, digits, and underscores (_). You can set at most five tags for a workflow.
      Description The description of the workflow, which can be up to 2,000 characters in length.
      Target Folder The folder for storing the workflow.
    3. Click OK.
  5. Configure the workflow.
    1. On the configuration tab of the workflow, drag nodes to the DAG and connect them as required.
      Connect the nodes
    2. Click the API1 node. In the API1 pane, select the API that you registered earlier from the Select API drop-down list, select set output results, and then enter {"user_id":"$.data[0].id"}.
      Set output results

      Use JSONPath expressions to set response parameters. The syntax for obtaining the value of a parameter is ${NodeA.namea}, which is the same as that for setting request parameters. {" user_id":"$.data[0].id"} assigns the value of the id parameter of the first element in the data array to the user_id parameter. Then, the API1 node returns {"user_id":"value"} in JSON format.

    3. Click the PYTHON1 node. In the PYTHON1 pane, select the function that you registered earlier from the Select Function drop-down list.
    4. Click the SWITCH1 node. In the SWITCH1 pane, click Set branch conditions.
      In the Set branch conditions dialog box, enter conditional expressions based on the response parameter of the parent node. For example, you can enter expressions in the ${Node ID. Parameter}>1 or $. Parameter>1 format. Conditional expressions support the following operators: ==, !=, >=, >, <=, <, &&, !, (), +, -, *, /, and %.
      In this example, the user_id parameter is the response parameter of the API1 node and is used as the request parameter of the SWITCH1 node.
      Branch node 1: $.user_id !=1, indicating that the branch node 1 is run if the value of the user_id parameter is not 1.
      Branch node 2: $.user_id==1, indicating that the branch node 2 is run if the output value of the user_id parameter is 1.
    5. Click the The end node, and then click Response Parameters in the right-side navigation pane to configure the response parameters.
  6. Test the workflow.
    1. On the configuration tab of the workflow, click Test in the upper-right corner.
    2. In the Test APIs dialog box, click Determine.
    3. View the operational logs and execution results on the Operation Log and Execution results tabs in the lower part of the configuration tab.