Cold data tiered storage
This topic describes the benefits of implementing tiered storage for hot and cold data and how this feature works in . Implementing tiered storage by dumping data that is infrequently accessed or updated to OSS can result in significant storage savings.
Benefits
When you , per-unit storage costs drop about 90% compared to ESSD PL1. .
Cold data tiered storage in provides:
-
Easy to use
-
SQL transparency: All SQL operations work without code changes. OSS tables support federated queries and full CRUD operations.
-
Index transparency: Archiving policies for indexes and materialized views work transparently.
-
-
Highly flexible
-
Archive data by table (including indexes and materialized views), by partition, or by Large Object (LOB) field. Combine policies for flexible configuration.
-
-
Excellent performance
-
Three-layer caching — UDF-level logical object cache, shared page cache, and file persistence cache — reduces OSS access requests and minimizes read/write latency impact.
-
-
Wide range of scenarios
-
Supports archiving general-purpose, spatiotemporal, and time series data such as spatiotemporal trajectories and high-precision maps, significantly reducing storage costs.
-
-
Secure and reliable
-
Cold data in OSS supports backup and recovery, reducing backup costs while maintaining high availability (HA).
-
Access latency increases after data is archived to OSS. Frequent updates or writes to archived data are not recommended. You can determine which data to store in OSS.
Applicability
Version requirements
Supported regions
|
Area |
Region |
|
China |
China (Hangzhou) |
|
China (Shanghai) |
|
|
China (Shenzhen) |
|
|
China (Beijing) |
|
|
Other |
Singapore |
How tiered storage works
Tiered storage results in cost savings because it uses the cost-effective OSS as a data storage option. can also be used with Elastic Block Storage (EBS) and OSS to achieve automatic storage tiering for hot and cold data based on specific usage patterns. This way, PolarDB maintains transparency during SQL CRUD operations and minimizes performance degradation by leveraging a multi-level caching system. The tiered storage architecture is as follows:
Cold storage modes
Cold storage lets you dump data tables, indexes, or materialized views into OSS. It minimizes disk usage, thereby resulting in significant storage savings. After cold storage, all SQL statements that specify CRUD operations are transparent without additional modifications.
PolarDB supports the following cold storage modes:
Store data within an entire table in OSS and retain indexes in cloud disks. This reduces storage costs while keeping access performance high.
Store the columns of the LOB type and secondary columns in OSS.
Store expired partitions in OSS and retain hot partitions in cloud disks. This is a typical tiered storage mode.
Use cases
The access latency of OSS is hundreds of times higher than that of cloud disks. Consequently, once data is dumped to OSS, its access performance is reduced. However, there are still customers who require fairly high performance when querying or updating their cold storage data. To meet this requirement, supports two types of storage tiering as follows:
Dump expired partitions into OSS while retaining hot partitions in cloud disks. This minimizes the impact on the query performance and can reduce storage costs. For more information, see .
Provide a materialized cache for frequently accessed and updated data in cloud disks and store the entirety of data in OSS. The lifecycle of data in the materialized cache is determined by their access frequency. This can achieve excellent performance and reduce storage costs. For more information, see .