CloudFlow:Inputs and outputs

Flows and steps

Typically, data needs to be passed between flows and steps, and between multiple steps of a flow. Steps of the Flow Definition Language (FDL) are similar to functions in functional programming languages. These steps accept inputs and produce outputs, and the outputs are stored in the local variable of the parent step (caller). The inputs and outputs must be JSON object structures, and the types of local variables vary with steps. For example, a task step uses the returned result of calling a function of Function Compute as the local variable, whereas a parallel step uses the outputs (arrays) of all branches as the local variable. The total size of inputs, outputs, and local variable in a step cannot exceed 32 KiB. Otherwise, the flow execution fails.

If a step contains another step, the outer step is called a parent step, and the included step is called a child step. The parent step of the outermost step is the flow. If the parent steps of two steps are the same, the two steps are of the same level.

Each flow and step contain inputs, outputs, and a local variable. Their mappings are listed in the following content:

• The inputMappings of a step maps the input and local variable of the parent step to the inputs of child steps.
• The outputMappings of a step maps the input and local variable of the current step to the output of the current step.
• The inputMappings of a flow maps the input of a flow execution to the input of the flow.
• The outputMappings of a flow maps the input and local variable of the flow to the output of the flow.

The local variable of a parent step contains a union set of the outputs of all its child steps. If the outputs contain repeated key values, the result of a later step overwrites that of an earlier step. In most cases, you can use the default mappings without specifying the input and output mappings.

• When no input mappings are specified, the input of a child step is the combination of the input and local variable of its parent step (if the local variable and the input have the same key value, the local variable will overwrite the input).
• When no output mappings are specified, the local variable is used as the output in all steps except parallel steps and foreach steps.

If you want to better control the input and output, you need to understand detailed mapping rules.

The following figure shows the input and output mappings of the example flow. In the flow, step1 is the parent step of step2 and step3, and step1 and step4 are the outermost steps.

version: v1
type: flow
steps:
  - type: parallel
    name: step1
    branches:
      - steps:
        - type: pass
          name: step2
      - steps:
        - type: pass
          name: step3
  - type: pass
    name: step4      

The following code can be used to describe mappings can be described, so that they can be easier to understand:

func flow(input0 Input) (output0 Output) {
  local0 := {}
  input1 := buildInput(step1InputMappings, input0, local0)
  output1 := step1(input1)
  save(local0, output1)
  input4 := buildInput(step4InputMappings, input0, local0)
  output4 := step4(input4)
  save(local0, output4)
  return buildOutput(flowOutputMappings, input0, local0)
}
func step1(input1 Input) (output1 Output) {
  local10 := {}
  input2 := buildInput(step2InputMappings, input1, local10)
  output2 := step2(input2)
  save(local10, output2)
  local11 := {}
  input3 := buildInput(step3InputMappings, input1, local11)
  output3 := step3(input3)
  save(local11, output3)
  return buildOutput(step1OutputMappings, [local10, local11])
}
func step2(input2 Input) (output2 Output) {
}
func step3(input3 Input) (output3 Output) {
}
func step4(input4 Input) (output4 Output) {
}       

In this example, the flow consists of two child steps: step1 and step4. step1 is a parallel step that contains step2 and step3.

1. When the system starts to execute the flow, it converts the StartExecution input into the flow input (input0) based on the input mappings of the flow.
2. When the flow execution starts, the local0 is empty.
3. The system prepares the input1 input for step1 based on the input mappings (step1InputMappings) of step1. The mapping sources are the input0 input and the local0 local variable of the flow.
4. The system calls step1 to load input1. step1 returns output1.
   • When the system starts to execute step1, its local10 is empty. step1 is a parallel step, so each branch corresponds to a local variable, avoiding concurrent access.
   • The system prepares the input2 input for step2 based on the input mappings of step2 (step2InputMappings). The mapping sources are the input1 input and the local10 local variable of step1.
   • The system calls step2 to load input2. step2 returns output2.
   • The system saves the output of step2 to the local10 local variable of step1.
   • Similarly, the system calls step3 and saves the result to the local11 local variable of step1.
5. The system saves the output of step1 to the local0 local variable of the flow.
6. Similarly, the system prepares the input4 input for step4 based on the input mappings of step4. The mapping sources are the input0 input and the local0 local variable of the flow.
   Note At this point, the local0 local variable may contain the output of step1. This achieves data transfer between step1 and step4.
7. The system calls step4 to load input4. step4 returns output4.
8. The system saves the output of step4 to the local0 local variable of the flow.
9. Finally, the system converts local0 into the flow output based on the output mappings of the flow.

Types

Both input and output mappings are arrays composed of target and source. The source defines the parameter source and is set to different values for different mappings. For example, $input.key indicates that the parameter source is the value of $.key in input. The target defines the names of target parameters. If the value of source starts with $, the value is specified in JSON path format (you can use JSONPath Online Evaluator to debug the JSON path), and the system parses the source into a specific value based on the path. Otherwise, the value is considered a constant.

• Source

  The source can be set to a constant, such as a value of the number, string, boolean, array, object, or null type.

  The source in the following example uses constants of different types. The information following the example shows the output.

  outputMappings:
    - target: int_key
      source: 1
    - target: bool_key
      source: true
    - target: string_key
      source: abc
    - target: float_key
      source: 1.234
    - target: null_key
      source: null
    - target: array1
      source: [1, 2, 3]
    - target: array2
      source:
        - 1
        - 2
        - 3
    - target: object1
      source: {a: b}
    - target: object2
      source:
        a:           

  {
    "array1": [1, 2, 3],
    "array2": [1, 2, 3],
    "bool_key": true,
    "float_key": 1.234,
    "int_key": 1,
    "null_key": null,
    "object1": {
      "a": "b"
    },
    "object2": {
      "a": "b"
    },
    "string_key": "abc"
  }            

• Target

  The target can only be a constant of the string type.

Input mappings

Input mappings convert the input ($input) of a parent step, the local variable ($local) of a parent step, or constants into the input of child steps. If no input mappings are specified, the input and local variable of the parent step are combined and used as the input of child steps. If the input and local variable of the parent step have the same name, the new input uses the name and value in the local variable.

inputMappings:
  - target: key1
    source: $input.key1
  - target: key2
    source: $local.key2
  - target: key3
    source: literal         

{
"key1":"value1"
}

{
"key2":"value2"
}

inputMappings:
  - target: key1
    source: $input.key1
  - target: key2
    source: $local.key2
  - target: key3
    source: literal

{
"key1":"value1"
"key2":"value2"
"key3":"literal"
}

{
"key1":"value1"
}

{
"key2":"value2"
}

{
"key1":"value1"
"key2":"value2"
}

{
"key1":"value1"
}

{
"key1":"value2"
}

{
"key1":"value2"
}

Output mappings

Output mappings convert the input ($input) of the current step, the local variable ($local) of the current step, or constants into the output of this step. If no output mappings are specified, choice steps and foreach steps use their local variables as outputs, whereas task steps uses task execution results as outputs. The local variables of parallel and foreach steps are arrays. Therefore, you must define output mappings to convert the arrays into JSON objects. By default, their local variables are not output. For more information, see the step description.

outputMappings:
  - target: key1
    source: $input.key1
  - target: key2
    source: $local.key2
  - target: key3
    source: literal          

{
"key1":"value1"
}

{
"key2":"value2"
}

outputMappings:
  - target: key1
    source: $input.key1
  - target: key2
    source: $local.key2
  - target: key3
    source: literal

{
"key1":"value1"
"key2":"value2"
"key3":"literal"
}

{
"key1":"value1"
}

[
  {
    "key2":"value2.1"
  },
  {
    "key2":"value2.2"
  }
]

outputMappings:
  - target: key1
    source: $input.key1
  - target: key2
    source: $local[*].key2
  - target: key3
    source: literal

{
  "key1":"value1",
  "key2":["value2.1","value2.2"],
  "key3":"literal"
}

{
"key1":"value1"
}

{
"key2":"value2"
}

{
"key2":"value2"
}

Save outputs to local variables of the parent steps

Child step outputs ($output) will be saved to local variables of the parent steps. If they contain the same name, the name and value in the outputs will overwrite the corresponding name and value in the local variables.

{
"key1":"value1"
}

{
"key2":"value2"
}

  {
    "key1":"value1"
  },
  {
    "key2":"value2"
  }

{
"key1":"value11"
}

  {
    "key1":"value1"
  },
  {
    "key2":"value2"
  }

  {
    "key1":"value11"
  },
  {
    "key2":"value2"
  }