Merge Columns - Platform For AI - Alibaba Cloud Documentation Center

The Merge Columns component merges two tables by column. If this component is used, the two tables must have the same number of rows. If one of the two tables has partitions, the partitioned table must connect to the second input port.

Configure the component

You can use one of the following methods to configure the Merge Columns component.

Method 1: Configure the component on the pipeline page

After you select the columns to be merged from the left table, the generated result is saved in the specified columns of the right table. Component configurations

Method 2: Use PAI commands

Configure the component parameters by using PAI commands. You can use the SQL Script component to call PAI commands. For more information, see SQL Script.

PAI -name appendColumns
    -project algo_public
    -DinputTableNames=maple_test_appendcol_basic_input1,maple_test_appendcol_basic_input2
    -DoutputTableName=maple_test_appendcol_setOutCol_output
    -DoutputTableColNames=x0,x1,x2,x3,x4,x5,x6,x7,x8,x9;


Parameter	Required	Description	Default value
inputTableNames	Yes	The names of two input tables. Separate two table names with a comma (,).	No default value
outputTableName	Yes	The name of the output table.	No default value
selectedColNamesList	No	The columns that are selected from the input tables. Separate the columns in a table with commas (,). Separate the columns of different tables with semicolons (;). Note If you select all columns in the two tables, the parameter value must be enclosed in double quotation marks ("). Otherwise, the semicolon (;) is processed as the terminator. If all columns in a table are selected, all column names can be omitted, but the semicolons (;) must be retained.	No default value
inputPartitionsInfoList	No	The partitions that are selected from the input tables. Separate the partitions in a table with forward slashes (/). Separate the partitions of different tables with semicolons (;). Note If you select all partitions, the parameter value must be enclosed in double quotation marks ("). Otherwise, the semicolon (;) is processed as the terminator. If a table is not partitioned, all partition names can be omitted, but the semicolons (;) must be retained.	No default value
autoRenameCol	No	Specifies whether to automatically rename the columns in the output table.	false
outputTableColNames	No	The new names of the columns in the output table. If this parameter is not specified, the column names in the original tables are used. Important If the autoRenameCol parameter is set to true, this parameter is negligible.	No default value
lifecycle	No	The lifecycle of the output table. The value must be a positive integer.	No default value
coreNum	No	The number of cores. The value must be a positive integer. This parameter must be used together with the memSizePerCore parameter. Valid values: [1,9999].	Determined by the system
memSizePerCore	No	The memory size of each core. The value must be a positive integer. Valid values: [1024,64 x 1024]. Unit: MB.	Determined by the system

Example

In this example, the following tables are merged:

Source Table 1

Table 1. maple_test_appendcol_basic_input1
col0:bigint	col1:double	col2:string	col3:Datetime	col4:Boolean
10	0.0	aaaa	2015-10-01 00:00:00	TRUE
11	1.0	aaaa	2015-10-01 00:00:00	FALSE
12	2.0	aaaa	2015-10-01 00:00:00	TRUE
13	3.0	aaaa	2015-10-01 00:00:00	TRUE
14	4.0	aaaa	2015-10-01 00:00:00	TRUE

Source Table 2

Table 2. maple_test_appendcol_basic_input2
col10:bigint	col11:double	col12:string	col13:Datetime	col14:Boolean
110	10.0	2aaaa	2015-10-01 00:00:00	TRUE
111	11.0	2aaaa	2015-10-01 00:00:00	FALSE
112	12.0	2aaaa	2015-10-01 00:00:00	TRUE
113	13.0	2aaaa	2015-10-01 00:00:00	TRUE
114	14.0	2aaaa	2015-10-01 00:00:00	FALSE

Run the following PAI command to merge the columns of the preceding two source tables:

PAI -name appendColumns
    -project algo_public
    -DinputTableNames=maple_test_appendcol_basic_input1,maple_test_appendcol_basic_input2
    -DoutputTableName=maple_test_appendcol_setOutCol_output
    -DoutputTableColNames=x0,x1,x2,x3,x4,x5,x6,x7,x8,x9;

The following output table is generated.

Table 3. maple_test_appendcol_setOutCol_output
x0	x1	x2	x3	x4	x5	x6	x7	x8	x9
10	0	aaaa	2015-10-01 00:00:00	true	110	10	2aaaa	2015-10-01 00:00:00	true
11	1	aaaa	2015-10-01 00:00:00	false	111	11	2aaaa	2015-10-01 00:00:00	false
12	2	aaaa	2015-10-01 00:00:00	true	112	12	2aaaa	2015-10-01 00:00:00	true
13	3	aaaa	2015-10-01 00:00:00	true	113	13	2aaaa	2015-10-01 00:00:00	true
14	4	aaaa	2015-10-01 00:00:00	true	114	14	2aaaa	2015-10-01 00:00:00	false