Schema change lets you modify the structure of an existing table in ApsaraDB for SelectDB to meet evolving business requirements. All schema change operations are asynchronous — the system returns a confirmation when you submit the job, and you track progress separately with SHOW ALTER TABLE COLUMN.
Supported operations:
| Operation | Description |
|---|---|
| Add a column | Insert a single column at a specified position in a specified index |
| Add multiple columns | Insert several columns into a specified index in one statement |
| Drop a column | Remove a column from a specified index |
| Modify a column type or position | Change a column's data type, position, or both |
| Reorder columns | Change the column order in a specified index |
| Create or modify a Bloom filter index | — |
| Create or delete a bitmap index | — |
Key concepts
| Term | Definition |
|---|---|
| Base table | The primary table created when you run CREATE TABLE. Each table corresponds to a base table upon creation. |
| Rollup table | A pre-aggregated table derived from a base table or another rollup table. |
| Materialized index | The internal storage structure. Both base tables and rollup tables are materialized indexes. |
| Load job | A data import operation. Each load job is a transaction with a unique, incrementally assigned transaction ID. |
How it works
A schema change generates new index data from the existing index data. Two data streams are converted during the process: historical data and data loaded while the job is running.
+----------+
| Load job |
+----+-----+
|
| Load job writes to both the original and new indexes
|
| +------------------+ +---------------+
| | Original index | | Original index|
+------> New incoming data| | History data |
| +------------------+ +------+--------+
| |
| | Convert history data
| |
| +------------------+ +------v--------+
| | New index | | New index |
+------> New incoming data| | History data |
+------------------+ +---------------+
Before converting historical data, SelectDB records the latest transaction ID as the watershed. It then waits for all load jobs with an ID below the watershed to complete. Load jobs submitted after the watershed write to both the original and new indexes simultaneously. This guarantees the new index contains complete data when historical data conversion finishes.
Usage notes
-
Only one schema change job can run on a table at a time.
-
Data imports and queries continue uninterrupted while a schema change runs.
-
Partition key columns and bucket columns cannot be modified.
-
If a table has value columns aggregated with REPLACE, you cannot drop key columns — SelectDB cannot determine the correct aggregated values without them. All non-key columns in the UNIQUE KEY model use REPLACE aggregation.
-
Adding a value column aggregated with SUM or REPLACE does not backfill meaningful values for historical data, because historical data no longer contains the original detail records.
-
When modifying a column type, preserve all other column attributes (aggregation method, nullability, default value) in the
MODIFY COLUMNstatement. -
Column name, aggregation method, nullability, default value, and column comment cannot be changed.
Submit a schema change job
ALTER TABLE [database.]table <alter_clause>;
Add a column
Inserts a single column at a specified position in a specified index.
ADD COLUMN column_name column_type [KEY | agg_type] [DEFAULT "default_value"]
[AFTER column_name | FIRST]
[TO rollup_index_name]
[PROPERTIES ("key"="value", ...)]
For an AGGREGATE KEY model value column, specifyagg_type. For a key column in a non-aggregate model (e.g., DUPLICATE KEY), specify theKEYkeyword. Columns that already exist in a base index cannot be added to a rollup index — create a new rollup index instead.
Example: Add a key column to a rollup index (DUPLICATE KEY model)
ALTER TABLE example_db.my_table
ADD COLUMN new_col INT KEY DEFAULT "0" AFTER col1
TO example_rollup_index;
Example: Add a value column to a rollup index (DUPLICATE KEY model)
ALTER TABLE example_db.my_table
ADD COLUMN new_col INT DEFAULT "0" AFTER col1
TO example_rollup_index;
Example: Add a value column with SUM aggregation (AGGREGATE KEY model)
ALTER TABLE example_db.my_table
ADD COLUMN new_col INT SUM DEFAULT "0" AFTER col1
TO example_rollup_index;
Add multiple columns
Inserts several columns into a specified index in a single statement.
ADD COLUMN (column_name1 column_type [KEY | agg_type] DEFAULT "default_value", ...)
[TO rollup_index_name]
[PROPERTIES ("key"="value", ...)]
For AGGREGATE KEY model value columns, specifyagg_type. For AGGREGATE KEY model key columns, specify theKEYkeyword. Columns already in a base index cannot be added to a rollup index.
Example: Add a key column and a value column in one statement
ALTER TABLE example_db.my_table
ADD COLUMN (col1 INT DEFAULT "1", col2 FLOAT SUM DEFAULT "2.3")
TO example_rollup_index;
Example: Add columns to multiple rollup tables
The original schemas:
+-----------+-------+------+------+------+---------+-------+
| IndexName | Field | Type | Null | Key | Default | Extra |
+-----------+-------+------+------+------+---------+-------+
| tbl1 | k1 | INT | No | true | N/A | |
| | k2 | INT | No | true | N/A | |
| | k3 | INT | No | true | N/A | |
| | | | | | | |
| rollup2 | k2 | INT | No | true | N/A | |
| | | | | | | |
| rollup1 | k1 | INT | No | true | N/A | |
| | k2 | INT | No | true | N/A | |
+-----------+-------+------+------+------+---------+-------+
Add k4 to rollup1, and add k4 and k5 to rollup2:
ALTER TABLE tbl1
ADD COLUMN k4 INT DEFAULT "1" TO rollup1,
ADD COLUMN k4 INT DEFAULT "1" TO rollup2,
ADD COLUMN k5 INT DEFAULT "1" TO rollup2;
The updated schemas:
+-----------+-------+------+------+------+---------+-------+
| IndexName | Field | Type | Null | Key | Default | Extra |
+-----------+-------+------+------+------+---------+-------+
| tbl1 | k1 | INT | No | true | N/A | |
| | k2 | INT | No | true | N/A | |
| | k3 | INT | No | true | N/A | |
| | k4 | INT | No | true | 1 | |
| | k5 | INT | No | true | 1 | |
| | | | | | | |
| rollup2 | k2 | INT | No | true | N/A | |
| | k4 | INT | No | true | 1 | |
| | k5 | INT | No | true | 1 | |
| | | | | | | |
| rollup1 | k1 | INT | No | true | N/A | |
| | k2 | INT | No | true | N/A | |
| | k4 | INT | No | true | 1 | |
+-----------+-------+------+------+------+---------+-------+
Columns added to a rollup table are automatically added to the base table (tbl1 in this example).
Drop a column
Removes a column from a specified index.
DROP COLUMN column_name [FROM rollup_index_name]
Partition key columns cannot be dropped. Dropping a column from a base index also drops it from any rollup index that contains that column.
Example: Drop a column from a rollup index
ALTER TABLE example_db.my_table
DROP COLUMN col2
FROM example_rollup_index;
Modify a column type or position
Changes a column's type, position, or both.
MODIFY COLUMN column_name column_type [KEY | agg_type] [NULL | NOT NULL] [DEFAULT "default_value"]
[AFTER column_name | FIRST]
[FROM rollup_index_name]
[PROPERTIES ("key"="value", ...)]
For AGGREGATE KEY model value columns, specifyagg_type. For non-aggregate model key columns, specify theKEYkeyword. Partition key columns and bucket columns cannot be modified. Always include all original column attributes (aggregation method, nullability, default value) in the statement — only the type changes.
Supported type conversions
| From | To |
|---|---|
| TINYINT, SMALLINT, INT, BIGINT, LARGEINT, FLOAT, DOUBLE | Any larger numeric type |
| TINYINT, SMALLINT, INT, BIGINT, LARGEINT, FLOAT, DOUBLE, DECIMAL | VARCHAR |
| VARCHAR | Larger VARCHAR (extend max length) |
| VARCHAR, CHAR | TINYINT, SMALLINT, INT, BIGINT, LARGEINT, FLOAT, DOUBLE |
| VARCHAR, CHAR | DATE (formats: %Y-%m-%d, %y-%m-%d, %Y%m%d, %y%m%d, %Y/%m/%d, %y/%m/%d) |
| DATETIME | DATE (retains year, month, day only; e.g., 2019-12-09 21:47:05 → 2019-12-09) |
| DATE | DATETIME (fills hours, minutes, seconds with 0; e.g., 2019-12-09 → 2019-12-09 00:00:00) |
| FLOAT | DOUBLE |
| INT | DATE (fails if value is not a valid date; original data is preserved) |
| All types except DATE and DATETIME | STRING |
STRING columns cannot be converted to any other type.
Example: Change a key column type and move it
Change col1 in the base index from INT to BIGINT and move it after col2:
ALTER TABLE example_db.my_table
MODIFY COLUMN col1 BIGINT KEY DEFAULT "1" AFTER col2;
Example: Extend a VARCHAR column's max length
The original schema:
+-----------+-------+-------------+------+-------+---------+-------+
| IndexName | Field | Type | Null | Key | Default | Extra |
+-----------+-------+-------------+------+-------+---------+-------+
| tbl1 | k1 | INT | No | true | N/A | |
| | k2 | INT | No | true | N/A | |
| | k3 | varchar(20) | No | true | N/A | |
| | k4 | INT | No | false | N/A | |
+-----------+-------+-------------+------+-------+---------+-------+
Extend k3 from VARCHAR(20) to VARCHAR(50):
ALTER TABLE example_tbl MODIFY COLUMN k3 varchar(50) key null COMMENT 'to 50';
The updated schema:
+-----------+-------+-------------+------+-------+---------+-------+
| IndexName | Field | Type | Null | Key | Default | Extra |
+-----------+-------+-------------+------+-------+---------+-------+
| tbl1 | k1 | INT | No | true | N/A | |
| | k2 | INT | No | true | N/A | |
| | k3 | varchar(50) | No | true | N/A | |
| | k4 | INT | No | false | N/A | |
+-----------+-------+-------------+------+-------+---------+-------+
Example: Extend a value column's max length
ALTER TABLE example_db.my_table
MODIFY COLUMN val1 VARCHAR(64) REPLACE DEFAULT "abc";
Reorder columns
Changes the column order in a specified index.
ORDER BY (column_name1, column_name2, ...)
[FROM rollup_index_name]
[PROPERTIES ("key"="value", ...)]
All columns in the index must be listed. Value columns must follow key columns.
Example: Reorder columns in a rollup index
Original order: k1, k2, k3, v1, v2
ALTER TABLE example_db.my_table
ORDER BY (k3, k1, k2, v2, v1)
FROM example_rollup_index;
Monitor a schema change job
Use SHOW ALTER TABLE COLUMN to check the status of a running or completed schema change job. If the job involves multiple indexes, one row is returned per index.
SHOW ALTER TABLE COLUMN;
Sample output:
SHOW ALTER TABLE COLUMN\G;
*************************** 1. row ***************************
JobId: 20021
TableName: tbl1
CreateTime: 2019-08-05 23:03:13
FinishTime: 2019-08-05 23:03:42
IndexName: tbl1
IndexId: 20022
OriginIndexId: 20017
SchemaVersion: 2:792557838
TransactionId: 10023
State: FINISHED
Msg:
Progress: NULL
Timeout: 86400
1 row in set (0.00 sec)
Output fields:
| Field | Description |
|---|---|
JobId |
Unique ID of the schema change job |
TableName |
Name of the base table |
CreateTime |
Time the job was created |
FinishTime |
Time the job completed; N/A if not yet complete |
IndexName |
Name of the index involved in the change |
IndexId |
Unique ID of the new index |
OriginIndexId |
Unique ID of the original index |
SchemaVersion |
Schema version in M:N format, where M is the version number and N is the hash value; increments with each schema change |
TransactionId |
Watershed transaction ID |
State |
Job state: PENDING, WAITING_TXN, RUNNING, FINISHED, or CANCELLED |
Msg |
Error message if the job fails |
Progress |
Conversion progress in M/N format (M = replicas converted, N = total replicas); displayed only when State is RUNNING |
Timeout |
Timeout period in seconds |
Job states:
| State | Meaning |
|---|---|
PENDING |
Waiting to be scheduled |
WAITING_TXN |
Waiting for all load jobs below the watershed transaction ID to complete |
RUNNING |
Historical data is being converted |
FINISHED |
Job completed successfully |
CANCELLED |
Job failed |
Cancel a schema change job
Cancel a schema change job that is not in the FINISHED or CANCELLED state.
CANCEL ALTER TABLE COLUMN FROM <tbl_name>;
Example:
CANCEL ALTER TABLE COLUMN FROM example_db.my_table;
Related configurations
Frontend
| Parameter | Default | Description |
|---|---|---|
alter_table_timeout_second |
86400 seconds | Timeout period for schema change jobs |
Backend
| Parameter | Default | Description |
|---|---|---|
alter_tablet_worker_count |
3 | Number of threads used to convert historical data. Increasing this value speeds up schema change jobs but raises I/O pressure. Shared with rollup jobs. |
alter_index_worker_count |
3 | Number of threads used to build indexes for historical data. Increasing this value speeds up schema change jobs but raises I/O pressure. Applies to inverted indexes only. |
To apply changes, restart the backend after updating these parameters.
FAQ
How fast does a schema change job run?
Schema change throughput is approximately 10 MB/s under worst-case conditions. Use this estimate to set an appropriate timeout period for your job.
What does the `Table xxx is not stable` error mean?
Schema change requires all data shard replicas to be complete and balanced. If some replicas are incomplete or balancing is in progress, SelectDB rejects the job submission.
To diagnose the issue, check replica health:
ADMIN SHOW REPLICA STATUS FROM tbl WHERE STATUS != "OK";
If results are returned, replicas have issues. SelectDB usually fixes them automatically. To repair manually:
ADMIN REPAIR TABLE tbl1;
To check for active balancing operations:
SHOW PROC "/cluster_balance/pending_tablets";
Wait for balancing to complete, or temporarily disable it:
ADMIN SET FRONTEND CONFIG ("disable_balance" = "true");