All Products
Search
Document Center

MaxCompute:Partition and column operations

Last Updated:Dec 26, 2023

MaxCompute allows you to perform operations on partitions or columns in a MaxCompute table. For example, you can add a column, drop a column, or change the data type of a column based on your business requirements.

The following table describes the statements that can be used for operations on partitions or columns in a MaxCompute table.

Type

Operation

Description

Authorized user

Operation platform

Partition operations

Add partitions

Adds partitions to an existing partitioned table.

Users who have the ALTER permission on tables

You can execute the statements that are described in this topic on the following platforms:

Drop partitions

Drops partitions from an existing partitioned table.

Change the value of LastModifiedTime for a partition

Changes the value of LastModifiedTime for a partition in a partitioned table.

Change a value in a partition key column

Changes a value in a partition key column of a partitioned table.

Merge partitions

Merges multiple partitions of a partitioned table into one partition. This operation deletes the dimension information about the partitions that are merged and transfers the partition data to a specified partition.

Clear data from a partition

Clears data from a specified partition.

Column operations

Add columns or column comments

Adds columns or column comments to an existing non-partitioned table or partitioned table.

Drop columns

Drops columns from an existing non-partitioned table or partitioned table.

Change the data type of a column

Changes the data type of an existing column.

Change the sequence of a column

Changes the sequence of a specified column in a table.

Change the name of a column

Changes the name of a column in an existing non-partitioned table or partitioned table.

Modify the comment of a column

Modifies the comment of a column in an existing non-partitioned table or partitioned table.

Change the name and comment of a column

Changes the name and comment of a column in an existing non-partitioned table or partitioned table at the same time.

Change the non-nullable property of a non-partition key column in a table

Changes the non-nullable property of a non-partition key column in a table.

Note

Transaction Table 2.0 is in invitational preview. By default, you cannot directly use Transaction Table 2.0. To use this feature, submit a ticket. If examples provided in this topic involve Transaction Table 2.0, you can apply for a trial on the application page for new features and then execute the related statements. For more information about Transaction Table 2.0, see Overview of Transaction Table 2.0.

Limits

  • Only columns of the TINYINT, SMALLINT, INT, BIGINT, CHAR, VARCHAR, and STRING types can be configured as partition key columns in a MaxCompute table.

  • A MaxCompute table can contain a maximum of six levels of partitions.

  • A MaxCompute table can contain a maximum of 60,000 partitions.

  • MaxCompute allows you to query data from a maximum of 10,000 partitions at a time.

  • When you use schema evolution, take note of the following limits:

    If you use the schema evolution feature, you can add columns of complex data types, drop columns, change the column sequence, and change the data types of columns in an existing table. If you change the sequence of a column in a table, add a column to a table and then change the sequence of the new column, or drop a column, the read and write operations on the table are affected. The actual impact varies based on the scenario.

    • If your job is a MapReduce job and the version of MapReduce is V1.0, you cannot use Graph jobs to read data from or write data to the table.

    • If your job is a Spark job that runs on the Cupid console and one of the following Spark versions is used, you can use your job to read data from the table. You cannot use your job to write data to the table.

      • Spark-2.3.0-odps0.34.0

      • Spark-3.1.1-odps0.34.0

    • If your job is a Machine Learning Platform for AI (PAI) job, you can use your job to read data from the table. You cannot use your job to write data to the table.

    • If your job is a Hologres job and the version of Hologres is earlier than V1.3, you cannot use your job to read data from or write data to the table when Hologres references the table as an external table.

    • You cannot execute the CLONE TABLE statement for the table on which schema evolution is performed.

    • If you write data to the table by using MaxCompute Steaming Tunnel, you cannot perform schema evolution. If you perform schema evolution, an error is reported.

  • You cannot merge partitions of a partitioned transactional table.

  • You cannot clear data in a Transaction Table 2.0 non-partitioned table or change the write.bucket.num property of a Transaction Table 2.0 non-partitioned table.

Add partitions

Adds partitions to an existing partitioned table.

  • Limits

    • A MaxCompute table can have a maximum of 60,000 partitions.

    • To add the values of partition key columns to a table that has multi-level partitions, you must specify all the partitions.

    • This operation can add only the values of partition key columns. The names of partition key columns cannot be added.

  • Syntax

    alter table <table_name> add [if not exists] partition <pt_spec> [partition <pt_spec> partition <pt_spec>...];
  • Parameters

    • table_name: required. The name of the partitioned table to which you want to add partitions.

    • if not exists: optional. If you do not configure the if not exists parameter and a partition with the same name already exists, this operation fails and an error is returned.

    • pt_spec: required. The partitions that you want to add. The value of this parameter is in the (partition_col1 = partition_col_value1, partition_col2 = partition_col_value2, ...) format. partition_col indicates the column name, and partition_col_value indicates the column value. The names of partition key columns are not case sensitive, but their values are case sensitive.

  • Examples:

    • Example 1: Add a partition to the sale_detail table. The partition stores the sales records in the China (Hangzhou) region in December 2013.

      alter table sale_detail add if not exists partition (sale_date='201312', region='hangzhou');
    • Example 2: Add two partitions to the sale_detail table. The partitions store the sales records in the China (Beijing) and China (Shanghai) regions in December 2013.

      alter table sale_detail add if not exists partition (sale_date='201312', region='beijing') partition (sale_date='201312', region='shanghai');
    • Example 3: Add one or more partitions to the sale_detail table and specify only the partition key column sale_date for the partitions. An error is returned because you must specify the two partition key columns sale_date and region for the partitions.

      alter table sale_detail add if not exists partition (sale_date='20111011');
    • Example 4: Add partitions to a Transaction Table 2.0 table.

      -- Create a Transaction Table 2.0 table.
      create table mf_tt (pk bigint not null primary key, 
                          val bigint not null) 
                   partitioned by (dd string, hh string) 
                   tblproperties ("transactional"="true"); 
                   
      -- Add a partition to the Transaction Table 2.0 table.           
      alter table mf_tt add partition (dd='01', hh='01');
    • Example 5: Change the properties of a Transaction Table 2.0 table.

      -- Change the write.bucket.num property of a Transaction Table 2.0 table. You can change only the property of a Transaction Table 2.0 partitioned table. You cannot change the property of a Transaction Table 2.0 non-partitioned table.
      alter table mf_tt3 set tblproperties("write.bucket.num"="64");
      
      -- Change the acid.data.retain.hours property of a Transaction Table 2.0 table.
      alter table mf_tt3 set tblproperties("acid.data.retain.hours"="60");

Drop partitions

Drops partitions from an existing partitioned table.

MaxCompute allows you to drop partitions based on a specified filter condition. If you want to drop multiple partitions that meet a specified filter condition at a time, you can use an expression to specify the filter condition, use the filter condition to match partitions, and drop the partitions at a time.

  • Limits

    • You can specify the information of only one partition key column in a PARTITION (<partition_filtercondition>) clause.

    • If you use an expression to specify PARTITION (<partition_filtercondition>), the function used in the expression must be a built-in scalar function.

  • Precautions

    • After you drop a partition from a table in your MaxCompute project, the volume of stored data in your MaxCompute project decreases.

    • You can specify a lifecycle for a partitioned table. This way, MaxCompute automatically reclaims partitions whose data is not updated within the time specified by the lifecycle. For more information about the lifecycle, see Lifecycle.

  • Syntax

    • The filter condition is not specified.

      -- Drop one partition at a time. 
      alter table <table_name> drop [if exists] partition <pt_spec>;
      -- Drop multiple partitions at a time. 
      alter table <table_name> drop [if exists] partition <pt_spec>,partition <pt_spec>[,partition <pt_spec>....];
    • The filter condition is specified.

      alter table <table_name> drop [if exists] partition <partition_filtercondition>;
  • Parameters

    • table_name: required. The name of the partitioned table from which you want to drop partitions.

    • if exists: optional. If you do not configure the if exists parameter and the partition that you want to drop does not exist, an error is returned.

    • pt_spec: required. The partitions that you want to drop. The value of this parameter is in the (partition_col1 = partition_col_value1, partition_col2 = partition_col_value2, ...) format. partition_col indicates the column name, and partition_col_value indicates the column value. The names of partition key columns are not case sensitive, but their values are case sensitive.

    • partition_filtercondition: the filter condition. This parameter is required when you specify the filter condition. This parameter is not case-sensitive. You must configure this parameter in the following format:

      partition_filtercondition
          : partition (<partition_col> <relational_operators> <partition_col_value>)
          | partition (scalar(<partition_col>) <relational_operators> <partition_col_value>)
          | partition (<partition_filtercondition1> AND|OR <partition_filtercondition2>)
          | partition (NOT <partition_filtercondition>)
          | partition (<partition_filtercondition1>)[,partition (<partition_filtercondition2>), ...]
      • partition_col: the name of the partition key column.

      • relational_operators: the relational operator. For more information, see Operator.

      • partition_col_value: a value in the partition key column. The value may be a comparison value or regular expression. The data type of this value must be the same as the data type of the partition key column.

      • scalar(): a scalar function. The scalar function generates a scalar based on the input value, processes the values in the column specified by partition_col, and uses relational operators specified by relational_operators to compare the processed values with the value specified by partition_col_value.

      • The filter conditions that are used to drop partitions support the logical operators NOT, AND, and OR. You can use PARTITION (NOT <partition_filtercondition>) to obtain the complementary set of the filter conditions that you specified. You can use PARTITION (<partition_filtercondition1> AND|OR <partition_filtercondition2>) to obtain the condition that is used to match the partitions you want to drop.

      • Multiple PARTITION (<partition_filtercondition>) clauses are supported. If these clauses are separated by commas (,), the logical relationship between the clauses is OR. The filter condition is obtained based on the OR logical relationship and used to match the partitions that you want to drop.

  • Examples:

    • The filter condition is not specified.

      -- Drop a partition from the sale_detail table. The partition stores the sales record in the China (Hangzhou) region in December 2013. 
      alter table sale_detail drop if exists partition(sale_date='201312',region='hangzhou'); 
      -- Drop two partitions from the sale_detail table. The partitions store the sales records in the China (Hangzhou) and China (Shanghai) regions in December 2013. 
      alter table sale_detail drop if exists partition(sale_date='201312',region='hangzhou'),partition(sale_date='201312',region='shanghai');
    • The filter condition is specified.

      -- Create a partitioned table named sale_detail. 
      create table if not exists sale_detail(
      shop_name     STRING,
      customer_id   STRING,
      total_price   DOUBLE)
      partitioned by (sale_date STRING);
      -- Add partitions to the table. 
      alter table sale_detail add if not exists
      partition (sale_date= '201910')
      partition (sale_date= '201911')
      partition (sale_date= '201912')
      partition (sale_date= '202001')
      partition (sale_date= '202002')
      partition (sale_date= '202003')
      partition (sale_date= '202004')
      partition (sale_date= '202005')
      partition (sale_date= '202006')
      partition (sale_date= '202007');
      -- Drop partitions from the table at a time. 
      alter table sale_detail drop if exists partition(sale_date < '201911');
      alter table sale_detail drop if exists partition(sale_date >= '202007');
      alter table sale_detail drop if exists partition(sale_date LIKE '20191%');
      alter table sale_detail drop if exists partition(sale_date IN ('202002','202004','202006'));
      alter table sale_detail drop if exists partition(sale_date BETWEEN '202001' AND '202007');
      alter table sale_detail drop if exists partition(substr(sale_date, 1, 4) = '2020');
      alter table sale_detail drop if exists partition(sale_date < '201912' OR sale_date >= '202006');
      alter table sale_detail drop if exists partition(sale_date > '201912' AND sale_date <= '202004');
      alter table sale_detail drop if exists partition(NOT sale_date > '202004');
      -- Drop partitions by using multiple PARTITION (<partition_filtercondition>) clauses. The logical relationship between these clauses is OR. 
      alter table sale_detail drop if exists partition(sale_date < '201911'), partition(sale_date >= '202007');
      -- Add partitions in other formats. 
      alter table sale_detail add IF NOT EXISTS
      partition (sale_date= '2019-10-05') 
      partition (sale_date= '2019-10-06') 
      partition (sale_date= '2019-10-07');
      -- Use regular expressions to match the partitions that you want to drop and drop these partitions at a time. 
      alter table sale_detail drop if exists partition(sale_date RLIKE '2019-\\d+-\\d+');
      -- Create a table named region_sale_detail. The table contains multi-level partitions. 
      create table if not exists region_sale_detail(
      shop_name     STRING,
      customer_id   STRING,
      total_price   DOUBLE)
      partitioned by (sale_date STRING , region STRING );
      -- Add partitions to the region_sale_detail table. 
      alter table region_sale_detail add IF NOT EXISTS
      partition (sale_date= '201910',region = 'shanghai')
      partition (sale_date= '201911',region = 'shanghai')
      partition (sale_date= '201912',region = 'shanghai')
      partition (sale_date= '202001',region = 'shanghai')
      partition (sale_date= '202002',region = 'shanghai')
      partition (sale_date= '201910',region = 'beijing')
      partition (sale_date= '201911',region = 'beijing')
      partition (sale_date= '201912',region = 'beijing')
      partition (sale_date= '202001',region = 'beijing')
      partition (sale_date= '202002',region = 'beijing');
      -- Execute the following statement to drop multi-level partitions at a time. The logical relationship between the two PARTITION (<partition_filtercondition>) clauses is OR. After the statement is executed, the partitions in which the value of the sale_date column is earlier than 201911 and the partitions in which the value of the region column is beijing are dropped. 
      alter table region_sale_detail drop if exists partition(sale_date < '201911'),partition(region = 'beijing');
      -- Execute the following statement to drop a partition in which the value of the sale_date column is earlier than 201911 and the value of the region column is beijing. 
      alter table region_sale_detail drop if exists partition(sale_date < '201911', region = 'beijing');

      When you drop multi-level partitions at a time, you cannot specify the information of multiple partition key columns in one PARTITION (<partition_filtercondition>) clause. Otherwise, the following error is returned: FAILED: ODPS-0130071:[1,82] Semantic analysis exception - invalid column reference region, partition expression must have one and only one column reference.

      -- If you specify the information of multiple partition key columns in one PARTITION (<partition_filtercondition>) clause, an error is returned. Example of incorrect usage: 
      alter table region_sale_detail drop if exists partition(sale_date < '201911' AND region = 'beijing');

Change the value of LastModifiedTime for a partition

Changes the value of LastDataModifiedTime for a partition in a partitioned table by executing the TOUCH statement that is provided by MaxCompute SQL. This statement changes the value of LastModifiedTime to the current time. In this case, MaxCompute considers that data is updated, and a new lifecycle of the partition starts from the time specified by LastModifiedTime.

  • Limits

    If a table contains multi-level partitions, you must specify all partition levels when you change the value of LastModifiedTime for a partition.

  • Syntax

    alter table <table_name> touch partition (<pt_spec>);
  • Parameters

    • table_name: required. The name of the partitioned table for which you want to change the value of LastModifiedTime. If the table does not exist, an error is returned.

    • pt_spec: required. The partition for which you want to change the value of LastModifiedTime. The value of this parameter is in the (partition_col1 = partition_col_value1, partition_col2 = partition_col_value2, ...) format. partition_col indicates the column name, and partition_col_value indicates the column value. If the specified column name or column value does not exist, an error is returned.

  • Example

    -- Change the value of LastModifiedTime for the partition in which the value of the sale_date column is 201312 and the value of the region column is shanghai in the sale_detail table. 
    alter table sale_detail touch partition (sale_date='201312', region='shanghai');

Change a value in a partition key column

Changes a value in a partition key column. MaxCompute SQL allows you to execute the RENAME statement to change values in partition key columns.

  • Limits

    • The RENAME statement can change values in partition key columns but cannot change the column names.

    • If a table contains multi-level partitions, you must specify all partition levels when you change the value of LastModifiedTime for a partition.

  • Syntax

    alter table <table_name> partition (<pt_spec>) rename to partition (<new_pt_spec>);
  • Parameters

    • table_name: required. The name of the table in which you want to change column values in a partition.

    • pt_spec: required. The partition in which you want to change column values. The value of this parameter is in the (partition_col1 = partition_col_value1, partition_col2 = partition_col_value2, ...) format. partition_col indicates the column name, and partition_col_value indicates the column value. If the specified column name or column value does not exist, an error is returned.

    • new_pt_spec: required. The new partition information. The value of this parameter is in the (partition_col1 = new_partition_col_value1, partition_col2 = new_partition_col_value2, ...) format. partition_col indicates the column name, and new_partition_col_value indicates the new column value.

  • Example

    -- Change column values in a partition of the sale_detail table. 
    alter table sale_detail partition (sale_date = '201312', region = 'hangzhou') rename to partition (sale_date = '201310', region = 'beijing');

Merge partitions

Merges multiple partitions of a partitioned table into one partition. MaxCompute SQL allows you to execute the MERGE PARTITION statement to merge multiple partitions of a table into one partition. This operation deletes the dimension information about the merged partitions and migrates the partition data to a specified partition.

  • Limits

    • Partitions of external tables cannot be merged. After partitions of a clustered table are merged into one partition, the partition does not have the clustering attribute.

    • You can merge a maximum of 4,000 partitions at a time.

  • Syntax

    alter table <table_name> merge [if exists] partition (<predicate>) [, partition(<predicate2>) ...] overwrite partition (<fullpartitionSpec>) [purge];
  • Parameters

    • table_name: required. The name of the partitioned table whose partitions you want to merge.

    • if exists: optional. If you do not configure the if exists parameter and the partitions that you want to merge do not exist, an error is returned when you merge partitions. If you configure the if exists parameter but no partitions meet the merge condition, a new partition cannot be generated after merging. If you merge partitions and modify source data by using a statement at the same time, an error is returned even if you specify if exists. The statement may be INSERT, RENAME, or DROP.

    • predicate: required. The condition that is used to match the partitions that you want to merge.

    • fullpartitionSpec: required. The partition that is generated after you merge partitions.

    • purge: optional. If you configure this parameter, the session directory is cleared. By default, logs that are generated in the last three days are deleted. For more information, see Purge.

  • Examples

    • Example 1: Merge the partitions that meet a specified condition into the destination partition.

      -- View the partitions in a partitioned table. 
      show partitions intpstringstringstring;
      
      ds=20181101/hh=00/mm=00
      ds=20181101/hh=00/mm=10
      ds=20181101/hh=10/mm=00
      ds=20181101/hh=10/mm=10
      
      -- Merge all partitions that meet the hh='00' condition into the ds='20181101', hh='00', mm='00' partition. 
      alter table intpstringstringstring merge partition(hh='00') overwrite partition(ds='20181101', hh='00', mm='00');
      
      -- View the partition that is generated after you merge partitions. 
      show partitions intpstringstringstring;
      
      ds=20181101/hh=00/mm=00
      ds=20181101/hh=10/mm=00
      ds=20181101/hh=10/mm=10                    
    • Example 2: Merge specified partitions into the destination partition.

      -- Merge specified partitions into one partition. 
      alter table intpstringstringstring merge if exists partition(ds='20181101', hh='00', mm='00'), partition(ds='20181101', hh='10', mm='00'),  partition(ds='20181101', hh='10', mm='10') overwrite partition(ds='20181101', hh='00', mm='00') purge;
      -- View the partition that is generated in the partitioned table after you merge partitions. 
      show partitions intpstringstringstring;
      
      ds=20181101/hh=00/mm=00

Clear data from a partition

Clears the data from a specified partition of a partitioned table.

MaxCompute allows you to clear the data in partitions that meet a specified filter condition. If you want to drop one or more partitions that meet a filter condition at a time, you can use an expression to specify the condition, use the condition to match partitions, and then clear data from the matched partitions.

  • Syntax

    • The filter condition is not specified.

      truncate table <table_name> partition <pt_spec>[, partition <pt_spec>....];
    • The filter condition is specified.

      truncate table <table_name> partition <partition_filtercondition>;
  • Parameters

    • table_name: required. The name of the partitioned table of which you want to clear partition data.

    • pt_spec: required. The partition in which you want to clear data. The value of this parameter is in the (partition_col1 = partition_col_value1, partition_col2 = partition_col_value2, ...) format. partition_col indicates the column name, and partition_col_value indicates the column value. The names of partition key columns are not case sensitive, but their values are case sensitive.

    • partition_filtercondition: the filter condition. This parameter is required when you specify the filter condition. This parameter is not case-sensitive. You must configure this parameter in the following format:

      partition_filtercondition
          : partition (<partition_col> <relational_operators> <partition_col_value>)
          | partition (scalar(<partition_col>) <relational_operators> <partition_col_value>)
          | partition (<partition_filtercondition1> AND|OR <partition_filtercondition2>)
          | partition (NOT <partition_filtercondition>)
          | partition (<partition_filtercondition1>)[,partition (<partition_filtercondition2>), ...]
      • partition_col: the name of the partition key column.

      • relational_operators: the relational operator. For more information, see Operator.

      • partition_col_value: a value in the partition key column. The value may be a comparison value or regular expression. The data type of this value must be the same as the data type of the partition key column.

      • scalar(): a scalar function. The scalar function generates a scalar based on the input value, processes the values in the column specified by partition_col, and uses relational operators specified by relational_operators to compare the processed values with the value specified by partition_col_value.

      • The filter conditions that are used to drop partitions support the logical operators NOT, AND, and OR. You can use PARTITION (NOT <partition_filtercondition>) to obtain the complementary set of the filter conditions that you specified. You can use PARTITION (<partition_filtercondition1> AND|OR <partition_filtercondition2>) to obtain the condition that is used to match the partitions you want to drop.

      • Multiple PARTITION (<partition_filtercondition>) clauses are supported. If these clauses are separated by commas (,), the logical relationship between the clauses is OR. The filter condition is obtained based on the OR logical relationship and used to match the partitions that you want to drop.

  • Examples

    • Clear the data from a partition with the filter condition not specified.

      -- Clear the data from a partition of the sale_detail table. The partition stores the sales record in the China (Hangzhou) region in December 2013. 
      truncate table sale_detail partition(sale_date='201312',region='hangzhou');
      
      -- Clear the data from two partitions of the sale_detail table. The partitions store the sales records in the China (Hangzhou) and China (Shanghai) regions in December 2013. 
      truncate table sale_detail partition(sale_date='201312',region='hangzhou'),  partition(sale_date='201312',region='shanghai');
    • Clear the data from a partition with the filter condition specified.

      -- Clear the data from multiple partitions of the sale_detail table. The partitions store the sales records whose sales dates start with 2013 in the China (Hangzhou) region. 
      truncate table sale_detail partition(sale_date like '2013%' and region='hangzhou');
    • Clear the data from a Transaction Table 2.0 non-partitioned table.

      -- Clear the data from a Transaction Table 2.0 non-partitioned table. The table must be a non-partitioned table. Otherwise, an error is reported.
      truncate table mf_tt2; 

Add columns or column comments

Adds columns or column comments to an existing non-partitioned table or partitioned table.

Note
  • MaxCompute allows you to add columns of the STRUCT type, such as struct<x: string, y: bigint> and map<string, struct<x: double, y: double>>. If you need to enable this feature, add the following configuration and take note of limits described in Limits. The configuration takes effect 10 minutes after you add it.

    setproject odps.schema.evolution.enable=true;

    To configure the project-level properties of a project, you must use the account of the project owner or the account that is assigned the project-level role Super_Administrator or Admin. For more information about how to assign roles, see Assign a built-in administrator role to a user.

  • You cannot perform the following operations on a Transaction Table 2.0 table by using the schema evolution feature: column dropping, changing of the column sequence, addition of columns of complex data types, and changing of the data type of a column.

  • Syntax

    alter table <table_name> add columns (<col_name1> <type1> comment ['<col_comment>'][, <col_name2> <type2> comment '<col_comment>'...]);
  • Parameters

    • table_name: required. The name of the table to which you want to add columns. You cannot specify the position of a new column in the table. By default, the new column is added as the last column.

    • col_name: required. The name of the column that you want to add to the table.

    • type: required. The data type of the column that you want to add to the table.

    • col_comment: optional. The comment of the column that you want to add to the table.

  • Examples

    • Example 1: Add two columns to the sale_detail table.

      alter table sale_detail add columns (customer_name STRING, education BIGINT);
    • Example 2: Add two columns and their comments to the sale_detail table.

      alter table sale_detail add columns (customer_name STRING comment 'Customer', education BIGINT comment 'Education' );
    • Example 3: Add a column of a complex data type to the sale_detail table.

      alter table sale_detail add columns (region struct<province:string, area:string>);
    • Example 4: Add columns to a Transaction Table 2.0 table.

      alter table mf_tt add columns (val2 bigint);

Drop columns

Drops one or more specified columns from an existing non-partitioned table or partitioned table.

Note

If you need to enable this feature, add the following configuration and take note of limits described in Limits. The configuration takes effect 10 minutes after you add it.

setproject odps.schema.evolution.enable=true;

To configure the project-level properties of a project, you must use the account of the project owner or the account that is assigned the project-level role Super_Administrator or Admin. For more information about how to assign roles, see Assign a built-in administrator role to a user.

  • Syntax

    alter table <table_name> drop columns <col_name1>[, <col_name2>...];
  • Parameters

    • table_name: required. The name of the table from which you want to drop columns.

    • col_name: required. The name of the column that you want to drop.

  • Example: Drop columns from a table

    -- Drop the customer_id column from the sale_detail table. Enter yes on the MaxCompute client to drop the column. 
    alter table sale_detail drop columns customer_id;
    
    -- Drop the shop_name and customer_id columns from the sale_detail table. Enter yes on the MaxCompute client to drop the column. 
    alter table sale_detail drop columns shop_name, customer_id;

Change the data type of a column

Changes the data type of an existing column.

Note

If you need to enable this feature, add the following configuration and take note of limits described in Limits. The configuration takes effect 10 minutes after you add it.

setproject odps.schema.evolution.enable=true;

To configure the project-level properties of a project, you must use the account of the project owner or the account that is assigned the project-level role Super_Administrator or Admin. For more information about how to assign roles, see Assign a built-in administrator role to a user.

  • Syntax

    alter table <table_name> change [column] <old_column_name> <new_column_name> <new_data_type>;
  • Parameters

    • table_name: required. The name of the table in which you want to change the column data type.

    • old_column_name: required. The name of the column whose data type you want to change.

    • new_column_name: required. The new name of the column. The value of new_column_name can be the same as the value of old_column_name. If the values of the parameters are the same, the name of the column is not changed. The name specified by the new_column_name parameter cannot be the same as the name of a column except the name specified by the old_column_name parameter.

    • new_data_type: required. The new data type of the column whose data type you want to change.

  • Examples

    -- Change the data type of the id column in the mf_evol_t3 table from INT to BIGINT.
    alter table mf_evol_t3 change id id bigint;
    -- Change the data type of the id column in the mf_evol_t3 table from BIGINT to STRING.
    alter table mf_evol_t3 change column id id string;
  • Supported data type conversions

    Note

    The following table describes the conversion between data types. Y indicates that the conversion between the data types is supported. N indicates that the conversion between the data types is not supported. - indicates that the conversion between the data types is not involved. Y() indicates that the conversion between the data types is supported only if the conditions enclosed in parentheses () are met.

    数据类型支持转换表

Change the sequence of a column

Changes the sequence of a column in an existing non-partitioned table or partitioned table.

Note

If you need to enable this feature, add the following configuration and take note of limits described in Limits. The configuration takes effect 10 minutes after you add it.

setproject odps.schema.evolution.enable=true;

To configure the project-level properties of a project, you must use the account of the project owner or the account that is assigned the project-level role Super_Administrator or Admin. For more information about how to assign roles, see Assign a built-in administrator role to a user.

  • Syntax

    alter table <table_name> change <old_column_name> <new_column_name> <column_type> after <column_name>;
  • Parameters

    • table_name: required. The name of the table in which you want to change the column sequence.

    • old_column_name: required. The original name of the column whose sequence you want to change.

    • new_col_name: required. The new name of the column whose sequence you want to change. The value of new_col_name can be the same as the value of old_column_name. If the values of the parameters are the same, the name of the column is not changed. However, the name specified by the new_col_name parameter cannot be the same as the name of a column except the name specified by the old_column_name parameter.

    • column_type: required. The original data type of the column whose sequence you want to change. The value of this parameter cannot be changed.

    • column_name: required. The column after which you want to place the specified column.

  • Examples

    -- Change the name of the customer_id column to customer and place the customer column after the total_price column in the sale_detail table. 
    alter table sale_detail change customer_id customer string after total_price;
    -- Place the customer_id column after the total_price column in the sale_detail table. The name of the customer_id column is not changed. 
    alter table sale_detail change customer_id customer_id string after total_price;

Change the name of a column

Changes the name of a column in an existing non-partitioned table or partitioned table.

  • Syntax

    alter table <table_name> change column <old_col_name> rename to <new_col_name>;
  • Parameters

    • table_name: required. The name of the table in which you want to change the name of a column.

    • old_col_name: required. The name of the column that you want to change. The column specified by old_col_name must already exist in the table.

    • new_col_name: required. The new name of the column whose sequence you want to change. The new name that you specify cannot be the same as an existing column name in the table.

  • Examples

    -- Change the name of the customer_name column to customer in the sale_detail table. 
    alter table sale_detail change column customer_name rename to customer;

Modify the comment of a column

Modifies the comment of a column in an existing non-partitioned table or partitioned table.

  • Syntax

    alter table <table_name> change column <col_name> comment '<col_comment>';
  • Parameters

    • table_name: required. The name of the table in which you want to modify the column comment.

    • col_name: required. The name of the column whose comment you want to modify. The column specified by col_name must already exist in the table.

    • col_comment: required. The new comment of the column. The new comment must be a valid string that does not exceed 1,024 bytes in length. If the value of this parameter does not meet the requirements, an error is returned.

  • Example

    -- Modify the comment of the customer column in the sale_detail table. 
    alter table sale_detail change column customer comment 'customer';

Change the name and comment of a column

Changes the name or comment of a column in a non-partitioned table or partitioned table.

  • Syntax

    alter table <table_name> change column <old_col_name> <new_col_name> <column_type> comment '<col_comment>';
  • Parameters

    • table_name: required. The name of the table in which you want to change the name and comment of a column.

    • old_col_name: required. The original name of the column whose name and comment need to be changed. The column specified by old_col_name must already exist in the table.

    • new_col_name: required. The new name of the column. The new name that you specify cannot be the same as an existing column name in the table.

    • column_type: required. The data type of the column.

    • col_comment: optional. The new comment of the column. The maximum size of a comment is 1,024 bytes.

  • Example

    -- Change the name of the customer_name column in the sale_detail table to customer_newname and the comment of the column to customer. 
    alter table sale_detail change column customer_name customer_newname STRING comment 'customer';

Change the non-nullable property of a non-partition key column in a table

Changes the non-nullable property of a non-partition key column in a table. If the values of a non-partition key column in a table cannot be null, you can execute the statement to allow the column values to be null.

You can execute the DESC EXTENDED table_name; statement to view the value of the Nullable property. This property determines whether the value of a non-partition key column can be null. If the value of the Nullable property is true, the column value can be null. If the value of the Nullable property is false, the column value must not be null.

  • Limits

    After you change the value of the Nullable property to true, you cannot restore the property setting. Proceed with caution.

  • Syntax

    alter table <table_name> change column <old_col_name> null;
  • Parameters

    • table_name: required. The name of the table in which you want to change the value of the Nullable property.

    • old_col_name: required. The name of the non-partition key column whose Nullable property you want to change. The column specified by old_col_name must already exist in the table.

  • Example

    -- Create a partitioned table. The values in the id column must not be null. 
    create table null_test(id int not null, name string) partitioned by (ds string);
    -- Change the values in the id column to null. 
    alter table null_test change column id null;

References

For more information about table operations, see Table operations.