Merge Table Columns Using the PAI Merge Columns Component - Platform for AI

The Merge Columns component combines two tables side by side, appending all columns from the second table to the right of the first. The row count stays the same — columns are joined horizontally, not stacked.

Prerequisites

Before you begin, make sure that:

Both tables have the same number of rows. If the row counts differ, the component returns an error.
If one table has partitions, connect the partitioned table to the second input port. Connecting it to the first port causes the operation to fail.

Configure the component

Use one of the following methods.

Method 1: Configure the component on the pipeline page

After you select the columns to merge from the left table, the result is saved to the specified columns of the right table.

Method 2: Use PAI commands

Pass configuration through the appendColumns PAI command. To call PAI commands in a pipeline, use the SQL Script component.

PAI -name appendColumns
    -project algo_public
    -DinputTableNames=maple_test_appendcol_basic_input1,maple_test_appendcol_basic_input2
    -DoutputTableName=maple_test_appendcol_setOutCol_output
    -DoutputTableColNames=x0,x1,x2,x3,x4,x5,x6,x7,x8,x9;

Parameters

Parameter	Required	Default	Description
`inputTableNames`	Yes	—	Names of the two input tables, separated by a comma (`,`).
`outputTableName`	Yes	—	Name of the output table.
`selectedColNamesList`	No	—	Columns to select from each input table. Separate columns within a table with commas (`,`); separate the two tables with a semicolon (`;`). If you select all columns from both tables, wrap the value in double quotation marks (`"`) — otherwise the semicolon is treated as a command terminator. If you select all columns from one table, you can omit its column names but must keep the semicolon.
`inputPartitionsInfoList`	No	—	Partitions to read from each input table. Separate partitions within a table with forward slashes (`/`); separate the two tables with a semicolon (`;`). If you select all partitions, wrap the value in double quotation marks (`"`). If a table is not partitioned, omit its partition names but keep the semicolon.
`autoRenameCol`	No	`false`	Specifies whether to automatically rename columns in the output table. When set to `true`, the `outputTableColNames` parameter is ignored.
`outputTableColNames`	No	—	New column names for the output table, applied in order. If not specified, the original column names from both tables are used. Ignored when `autoRenameCol` is `true`.
`lifecycle`	No	—	Lifecycle of the output table. Must be a positive integer.
`coreNum`	No	System default	Number of cores to allocate. Must be a positive integer in the range [1, 9999]. Must be set together with `memSizePerCore`.
`memSizePerCore`	No	System default	Memory allocated per core, in MB. Must be a positive integer in the range [1024, 65536]. Must be set together with `coreNum`.

Handle column name conflicts

When both input tables share column names, the output table will contain duplicate column names unless you resolve the conflict. Two options are available:

Automatic renaming: Set autoRenameCol=true. PAI assigns unique names automatically. Use this when you don't need control over the output column names.
Manual naming: Set outputTableColNames to a comma-separated list of new names covering all output columns. Use this when downstream steps depend on specific column names.

If autoRenameCol=true, any value set for outputTableColNames is ignored.

Example

This example merges the following two tables.

Source table 1: `maple_test_appendcol_basic_input1` (5 columns, 5 rows)

col0:bigint	col1:double	col2:string	col3:Datetime	col4:Boolean
10	0.0	aaaa	2015-10-01 00:00:00	TRUE
11	1.0	aaaa	2015-10-01 00:00:00	FALSE
12	2.0	aaaa	2015-10-01 00:00:00	TRUE
13	3.0	aaaa	2015-10-01 00:00:00	TRUE
14	4.0	aaaa	2015-10-01 00:00:00	TRUE

Source table 2: `maple_test_appendcol_basic_input2` (5 columns, 5 rows)

col10:bigint	col11:double	col12:string	col13:Datetime	col14:Boolean
110	10.0	2aaaa	2015-10-01 00:00:00	TRUE
111	11.0	2aaaa	2015-10-01 00:00:00	FALSE
112	12.0	2aaaa	2015-10-01 00:00:00	TRUE
113	13.0	2aaaa	2015-10-01 00:00:00	TRUE
114	14.0	2aaaa	2015-10-01 00:00:00	FALSE

Run the following command to merge the two tables and rename all output columns:

PAI -name appendColumns
    -project algo_public
    -DinputTableNames=maple_test_appendcol_basic_input1,maple_test_appendcol_basic_input2
    -DoutputTableName=maple_test_appendcol_setOutCol_output
    -DoutputTableColNames=x0,x1,x2,x3,x4,x5,x6,x7,x8,x9;

The output table maple_test_appendcol_setOutCol_output contains all 10 columns (x0–x4 from source table 1, x5–x9 from source table 2) and the same 5 rows:

x0	x1	x2	x3	x4	x5	x6	x7	x8	x9
10	0	aaaa	2015-10-01 00:00:00	true	110	10	2aaaa	2015-10-01 00:00:00	true
11	1	aaaa	2015-10-01 00:00:00	false	111	11	2aaaa	2015-10-01 00:00:00	false
12	2	aaaa	2015-10-01 00:00:00	true	112	12	2aaaa	2015-10-01 00:00:00	true
13	3	aaaa	2015-10-01 00:00:00	true	113	13	2aaaa	2015-10-01 00:00:00	true
14	4	aaaa	2015-10-01 00:00:00	true	114	14	2aaaa	2015-10-01 00:00:00	false