All Products
Search
Document Center

MaxCompute:Column operations

Last Updated:Jan 06, 2025

MaxCompute supports column modification operations on existing tables, such as adding and deleting columns, and modifying column data types. You can perform the corresponding operations based on actual business scenarios.

The column operation commands in MaxCompute SQL are as follows:

Operation

Feature

Role

Operation entry

Add columns or comments

Add columns or comments to an existing partitioned or non-partitioned table.

Users who have the Alter permission on tables

You can execute the statements that are described in this topic on the following platforms:

Drop columns

Drop columns from an existing partitioned or non-partitioned table.

Modify column data types

Modify the data type of an existing column.

Modify column order

Modify the order of a specified column in a table.

Modify the name of a column

Modify the name of a column in an existing partitioned or non-partitioned table.

Modify column comments

Modify the comment of a column in an existing partitioned or non-partitioned table.

Modify the names and comments of columns

Simultaneously modify the name and comment of a column in an existing partitioned or non-partitioned table.

Modify the non-nullable properties of non-partition columns

Modify the non-nullable property of a non-partition key column in a table.

Limits

Schema evolution includes adding columns of complex data types to existing tables, deleting columns, and modifying the order and data types of columns. In the following scenarios, if you add or delete columns or change their order, the read and write behavior of the table will change, and the following limits apply:

  • If your job is a MapReduce job and the version of MapReduce is 1.0, you cannot use Graph jobs to read data from or write data to the table.

  • If your job uses the CUstomized PUshdown In Data (CUPID) computing framework, only the following Spark versions can be used to read data from the table, but cannot write data to the table:

    • Spark-2.3.0-odps0.34.0

    • Spark-3.1.1-odps0.34.0

  • If your job is a Machine Learning Platform for AI (PAI) job, you can use your job to read data from the table, but cannot write data to the table.

  • If your job is a Hologres job and the version of Hologres is earlier than 1.3, data cannot be read from and written to the table when Hologres references the table as an external table.

  • After you perform schema evolution operations on a table, the CLONE TABLE feature is not supported.

  • If you write data to the table by using MaxCompute Steaming Tunnel, you cannot perform schema evolution. If you try to do so, an error is reported.

Add columns or comments

Add columns or comments to an existing non-partitioned table or partitioned table.

Note
  • MaxCompute supports adding columns of the STRUCT type. For example struct<x: string, y: bigint> or map<string, struct<x: double, y: double>>. If you need to enable this feature, please set the following parameter and be aware of the limits. It takes approximately 10 minutes for the parameter to take effect.

    setproject odps.schema.evolution.enable=true;

    To set the project-level properties of a project, the account owner must be the owner of the project or be granted the Super_Administrator or Admin role at the project level. For more information, see Assign a built-in administrator role to a user.

  • Syntax

    ALTER TABLE <table_name> 
          ADD columns [if NOT EXISTS]
              (<col_name1> <type1> comment ['<col_comment>']
               [, <col_name2> <type2> comment '<col_comment>'...]
              );
  • Parameters

    Parameter

    Required

    Description

    table_name

    Yes

    The name of the table to which you want to add columns. You cannot specify the position of a new column in the table. By default, the new column is added as the last column.

    col_name

    Yes

    The name of the column that you want to add to the table.

    type

    Yes

    The data type of the column that you want to add to the table.

    col_comment

    No

    The comment of the column that you want to add to the table.

  • Examples

    • Example 1: Add two columns to the sale_detail table.

      ALTER TABLE sale_detail ADD columns if NOT EXISTS(customer_name STRING, education BIGINT);
    • Example 2: Add two columns and comments to the sale_detail table.

      ALTER TABLE sale_detail ADD columns (customer_name STRING comment 'customer', education BIGINT comment 'education' );
    • Example 3: Add a column of a complex data type to the sale_detail table.

      ALTER TABLE sale_detail ADD columns (region struct<province:string, area:string>);
    • Example 4: Add columns to a Delta table.

      ALTER TABLE mf_tt ADD columns (val2 bigint);
    • Example 5: Add the ID column to the sale_detail table. If the ID column already exists in the sale_detail table, the SQL statement is executed successfully but the ID column is not repeatedly added to the table.

      --If the ID column already exists in the sale_detail table, the ID column is not repeatedly added to the table.
      ALTER TABLE sale_detail ADD columns if NOT EXISTS(id bigint);

Drop columns

Drop one or more specified columns from an existing partitioned or non-partitioned table.

Note

If you need to enable this feature, please set the following parameter and be aware of the limits. It takes approximately 10 minutes for the parameter to take effect.

setproject odps.schema.evolution.enable=true;

To set the project-level properties of a project, the account owner must be the owner of the project or be granted the Super_Administrator or Admin role at the project level. For more information, see Assign a built-in administrator role to a user.

  • Syntax

    ALTER TABLE <table_name> DROP columns <col_name1>[, <col_name2>...];
  • Parameters

    Parameter

    Required

    Description

    table_name

    Yes

    The name of the table from which you want to drop columns.

    col_name

    Yes

    The name of the column you want to drop.

  • Example: Drop columns from a table

    --Drop the customer_id column from the sale_detail table. After you input 'yes' for confirmation, the column will be deleted.
    ALTER TABLE sale_detail DROP columns customer_id;
    
    --Drop the shop_name and customer_id columns from the sale_detail table. After you input 'yes' for comfirmation, the column will be deleted.
    ALTER TABLE sale_detail DROP columns shop_name, customer_id;

Modify column data types

Modify the data type of an existing column.

Note

If you need to enable this feature, please set the following parameter and be aware of the limits. It takes approximately 10 minutes for the parameter to take effect.

setproject odps.schema.evolution.enable=true;

To set the project-level properties of a project, the account owner must be the owner of the project or be granted the Super_Administrator or Admin role at the project level. For more information, see Assign a built-in administrator role to a user.

  • Syntax.

    ALTER TABLE <table_name> change [COLUMN] <old_column_name> <new_column_name> <new_data_type>;
  • Parameters

    Parameter

    Required

    Description

    table_name

    Yes

    The name of the table in which you want to modify the data type of a column.

    old_column_name

    Yes

    The name of the column whose data type you want to modify.

    new_column_name

    Yes

    The new name of the column whose data type you want to modify.

    Note

    old_column_name can be the same as new_column_name, which indicates that the column name is not modified. However, new_column_name cannot match any column name other than old_column_name.

    new_data_type

    Yes

    The new data type of the column whose data type you want to modify.

  • Examples.

    --Modify the id field of the mf_evol_t3 table from int to bigint. 
    ALTER TABLE mf_evol_t3 change id id bigint;
    --Modify the id field of the mf_evol_t3 table from bigint to string. 
    ALTER TABLE mf_evol_t3 change COLUMN id id string;
  • Data type conversion.

    Note

    Y indicates that the conversion is supported. N indicates that the conversion is not supported. - indicates that the conversion is not involved. Y() indicates that the conversion is supported if the conditions in the parentheses are met.

    数据类型支持转换表

Modify column order

Modify the order of a column in an existing partitioned or non-partitioned table.

Note

If you need to enable this feature, please set the following parameter and be aware of the limits. It takes approximately 10 minutes for the parameter to take effect.

setproject odps.schema.evolution.enable=true;

To set the project-level properties of a project, the account owner must be the owner of the project or be granted the Super_Administrator or Admin role at the project level. For more information, see Assign a built-in administrator role to a user.

  • Syntax

    ALTER TABLE <table_name> change <old_column_name> <new_column_name> <column_type> after <column_name>;
  • Parameters

    Parameter

    Required

    Description

    table_name

    Yes

    The name of the table in which you want to modify the column order.

    old_column_name

    Yes

    The original name of the column whose order you want to modify.

    new_col_name

    Yes

    The new name of the column whose order you want to modify.

    Note

    new_col_name can be the same as old_column_name, which indicates that the column name is not modified. However, new_col_name cannot match any column name other than old_column_name.

    column_type

    Yes

    The original data type of the column whose order you want to modify. You cannot modify the parameter.

    column_name

    Yes

    Place the column whose order you want to modify after column_name.

  • Examples

    --Modify the table sale_detail to change the column name customer_id to customer and place it after the total_price column.
    ALTER TABLE sale_detail change customer_id customer string after total_price;
    --Modify the table sale_detail to move the column customer_id after the total_price column without changing its name.
    ALTER TABLE sale_detail change customer_id customer_id string after total_price;

Modify the name of a column

Modify the name of a column in an existing partitioned or non-partitioned table.

  • Syntax

    ALTER TABLE <table_name> change COLUMN <old_col_name> rename TO <new_col_name>;
  • Parameters

    Parameter

    Required

    Description

    table_name

    Yes

    The name of the table in which you want to modify the name of a column.

    old_col_name

    Yes

    The name of the column that you want to modify.

    Note

    The column must exist.

    new_col_name

    Yes

    The new name of the column. The column name cannot be duplicated.

  • Examples

    --Modify the table sale_detail to change the column_name from customer_name to customer.
    ALTER TABLE sale_detail change COLUMN customer_name rename TO customer;

Modify column comments

Modify the comment of a column in an existing partitioned or non-partitioned table.

  • Syntax

    ALTER TABLE <table_name> change COLUMN <col_name> comment '<col_comment>';
  • Parameters

    Parameter

    Required

    Description

    table_name

    Yes

    The name of the table in which you want to modify the column comment.

    col_name

    Yes

    The name of the column whose comment you want to modify.

    Note

    The column must exist.

    col_comment

    Yes

    The new comment of the column. The comment must be a valid string that does not exceed 1,024 bytes in length. If the value of this parameter does not meet the requirements, an error is returned.

  • Examples

    --Modify the comment for the customer column in sale_detail table. 
    ALTER TABLE sale_detail change COLUMN customer comment 'customer';

Modify the names and comments of columns

Modify the name or comment of a column in a partitioned or non-partitioned table.

  • Syntax

    ALTER TABLE <table_name> change COLUMN <old_col_name> <new_col_name> <column_type> comment '<col_comment>';
  • Parameters

    Parameter

    Required

    Description

    table_name

    Yes

    The name of the table in which you want to modify the name and comment of a column.

    old_col_name

    Yes

    The name of the column whose name and comment you want to modify.

    Note

    The column must exist.

    new_col_name

    Yes

    The new name of the column. The column name cannot be duplicated.

    column_type

    Yes

    Data type of the column.

    col_comment

    Optional

    The new comment of the column. The comment must be a valid string that does not exceed 1,024 bytes in length.

  • Examples

    --Modify the table sale_detail to rename the column customer_name to customer_newname and add its comment 'customer'.
    ALTER TABLE sale_detail change COLUMN customer_name customer_newname STRING comment 'customer';

Modify the non-nullable properties of non-partition columns

Modify the Nullable property of a non-partition column in a table. If the values of a non-partition column in a table cannot be null, you can execute the statement to allow the column values to be null.

You can run the desc extended table_name; command to view the value of the Nullable property. If Nullable is true, the column values can be null. If Nullable is false, the column values cannot be null.

  • Limits

    After you modify the value of the Nullable property to true, you cannot revert the property setting.

  • Syntax

    ALTER TABLE <table_name> change COLUMN <old_col_name> NULL;
  • Parameters

    Parameter

    Required

    Description

    table_name

    Yes

    The name of the table in which you want to modify the value of the Nullable property.

    old_col_name

    Yes

    The name of the non-partition column whose Nullable property you want to modify.

    Note

    The column must be an existing non-partition column.

  • Examples

    --Create a partition table and the column id cannot be null.
    CREATE TABLE null_test(id int NOT NULL, name string) partitioned BY (ds string);
    --Modify the id column in the table to allow null values.
    ALTER TABLE null_test change COLUMN id NULL;

References

For more information about table operation commands, see the following: