This topic provides answers to some frequently asked questions about the cold data archiving feature in PolarDB.
What do I do if the "[Data Lifecycle Management] DLM storage engine is not support. The value of polar_dlm_storage_mode is OFF." error message appears?
You must enable the cold data archiving feature in the PolarDB console. For more information, see Enable cold data archiving.
Why is the Cold Data Archiving menu not available in the navigation bar of the PolarDB console?
After you enable the cold data archiving feature and refresh the page in a few minutes, the Cold Data Archiving menu appears.
What formats are cold data files archived in?
Cold data is archived in the CSV or ORC format.
How do I determine whether a table or partition is archived?
You can use one of the following methods to check whether a table or partition is archived:
Execute the
SHOW CREATE TABLE table_name;statement to check whether the table or partition is archived.For a common table, if STORAGE OSS or
CONNECTION='default_oss_server'is displayed, the data is archived.For a partitioned table, if ENGINE=CSV/ORC is displayed for partitions in the table, the data is archived.
Check in the console whether the data is archived.
Log on to the PolarDB console. Find the cluster that you want to manage and click the ID of the cluster. In the left-side navigation pane, choose > Cold Data Archiving. Check whether the table name exists. If the table name exists, the data is archived to OSS.
The error message "[Data Lifecycle Management] errmsg: Not allowd archive the archived table." is returned after I execute statements to archive data. Why?
You cannot archive a table that is already archived. Make sure that the table in the statements is not archived and try again.
How do I download archived cold data?
You cannot directly download archived cold data. To download archived cold data, go to Quota Center. Enter the polardb_mysql_download_oss quota ID to find the quota name. In
the Actions column, click Apply.
How do I view the size of archived cold data?
You can view the size of archived cold data in the PolarDB console, or directly query the cold data tables on a cluster. For more information, see View information about the cold data archived in OSS.
How do I query archived cold data?
After you archive cold data for common tables and partitioned tables, you can use one of the following methods to query the archived cold data:
Common tables: Use the same method as querying hot data. You do not need to modify the access mode.
Partitioned tables: For more information about how to query archived cold data, see Query data in a hybrid partitioned table.
How do I modify archived cold data?
You cannot directly modify cold data archived to OSS in the CSV or ORC format. To modify archived cold data, execute the ALTER TABLE table_name ENGINE=InnoDB statement to import the table data from OSS to PolarStore and modify the data in PolarStore. After you modify the data, execute the ALTER TABLE t ENGINE = CSV STORAGE OSS; statement to archive the modified table data to OSS. For more information, see Import data from OSS tables to PolarStore.
How do I delete archived cold data?
Use a privileged account to delete archived cold data.
Execute the DROP TABLE statement to delete the archived table, and then execute the CALL dbms_oss.delete_table_file('database_name', 'table_name'); to delete the archived cold data.
How is automatic data archiving triggered?
Automatic data archiving is triggered by events. For more information, see Data lifecycle management (DLM).
Are logs available for automatically archived data?
Logs for automatically archived data are stored in the mysql.dlm_progress table. You can view the mysql.dlm_progress table by using an account that has permissions on the MySQL databases. For more information, see mysql.dlm_progress table.
How do I view the current policy for automatic data archiving?
Policies for automatic data archiving are stored in the mysql.dlm_policies table. You can view the mysql.dlm_policies table by using an account that has permissions on the MySQL databases. For more information, see mysql.dlm_policies table.
How do I archive data to my own OSS engine?
You cannot archive data to your own OSS engine. You can execute the select into ossoutfile statement to export data to the OSS engine. For more information, see Serially export local tables to OSS.
After cold data is archived, the original table still exists in the database. Do I need to manually delete the table to save storage space?
After you archive cold data, PolarDB stores only the metadata of the table. The actual data is stored in OSS and does not consume the storage space of your cluster.
What do I do if the "OSS error: error message: The OSS Access Key Id you provided does not exist in our records" error message appears?
Error messages that start with OSS error: indicate errors that occurred on OSS. You can view the OSS error codes to identify the possible causes. The following items describe the cause of and solution to the preceding error message:
Cause: The AccessKey pair becomes invalid after the temporary access credential expires.
Solution: Use the AccessKey pair to apply for new temporary access credentials from the app server. For more information, see Use temporary credentials provided by STS to access OSS.
Why am I unable to view information about the archived table in the console after I archive cold data?
For an empty table, no CSV file is generated in OSS after the table is archived, and therefore no information about the table is displayed in the console.
How long does it take to rename a table that is archived to OSS?
If you rename a table that is archived to OSS, the file information in OSS is modified at a rate of about 100 Mbit/s. You can estimate the duration based on the total size of data.
What do I do if the "ERROR 8185 (HY000): Change partition not support archive to engine CSV" error message appears?
The cold data archiving feature for partitioned tables is in canary release. To use the feature, go to Quota Center. Enter the polardb_mysql_hybrid_partition quota ID to find the quota name. In
the Actions column, click Apply.
Is the size of archived cold data larger or smaller than that of the original table?
The occupied space changes because archived cold data is in different formats from the original data. The following changes may occur:
If the original table uses the InnoDB engine, the archived cold data may be smaller in size.
If the original table uses X-Engine, the archived cold data may be larger in size.
The archived cold data in the CSV format may be larger in size than the archived cold data in the ORC format.
Why am I unable to view archived cold data and objects in the OSS of my Alibaba Cloud account?
PolarDB cold data is stored in the default OSS, instead of in your own OSS. As a result, you cannot view archived cold data in your own OSS. You can view archived cold data only in the PolarDB console.
Why does the "Statement violates GTID consistency: Updates to non-transactional tables can only be done in either autocommitted statements or single-statement transactions, and never in the same statement as updates to transactional tables." error message appear when a stored procedure involves an archived table?
Archived tables in the CSV or ORC format do not support transactions. If a stored procedure involves an archived table, global transaction identifiers (GTID) inconsistency may occur even if no actual write operations are performed. To resolve the issue, you can change the storage engine to X-Engine to reduce storage costs and ensure that all tables support transactions.
Why does the "The last packet successfully received from the server was 15,287 milliseconds ago. The last packet sent successfully to the server was 15,287 milliseconds ago.”error message appear when I restore data from OSS to my cluster?
The message appears because the restoration process exceeds the timeout threshold of the client. To resolve the issue, you can increase the minimum PolarDB Capacity Unit (PCU) specification of your PolarDB for MySQL cluster and then execute the restoration task.
What do I do if the version of my PolarDB for MySQL cluster is earlier than required by the cold data archiving feature?
Upgrade your PolarDB for MySQL cluster to the latest version. For more information, see Version management.
How does the size of a table in a PolarDB for MySQL cluster change after the table is archived to OSS?
After you archive a table in a PolarDB for MySQL cluster to OSS, the size of the table may change.
For InnoDB tables that do not have indexes, the data is compressed during the archiving process. As a result, the size of the archived table in OSS is smaller than the size of the original InnoDB table.
For InnoDB tables that have indexes, the indexes are removed during the archiving process. As a result, the size of the archived table in OSS is significantly smaller compared to the size of the original InnoDB table.
For X-Engine tables, the size of the archived table in OSS may be larger than the size of the original X-Engine table because the compression efficiency achieved by OSS may not match the high compression already applied by X-Engine.
PolarDB for MySQL supports archiving data in the CSV or ORC format. ORC files provide higher compression ratio than CSV files. As a result, the size of the archived data in the CSV format may be relatively larger. For more information, see Manually archive cold data in the CSV format.