Upgrading a major version by using blue-green deployment - ApsaraDB RDS

This topic describes how to upgrade the major version of an ApsaraDB RDS for PostgreSQL database using the blue-green deployment method.

Prerequisites

The instance runs RDS PostgreSQL 16 or an earlier version.
The instance is not a read-only instance or a cluster instance.
Babelfish is not enabled for the instance. This means that the minor version number does not contain the babelfish suffix.

Background information

In a blue-green deployment, the source instance is restored to a new instance. Then, pg_upgrade is used to upgrade the new instance to the target version. If you perform a cutover, the endpoint of the source instance is automatically switched to the new instance.

The ApsaraDB RDS for PostgreSQL console also supports upgrading the major database version using the zero-downtime mode and local upgrade mode. For a comparison of the different modes, see Introduction to major version upgrade solutions.

Upgrade fees

When you upgrade a major version using blue-green deployment, the system creates a new instance based on the source instance. After the upgrade is complete:

The source instance and the new instance are both billed.
The billing method and fees for the source instance remain unchanged.
The billing method for the new instance may be different from that of the source instance.
Billing method of the source instance
Billing method of the new instance
Subscription/Pay-as-you-go
Pay-as-you-go
Serverless
Serverless
Note
To save costs, after the services on the new instance are running stably, we recommend that you convert the new instance to a subscription and release or unsubscribe from the source instance. However, note the following:
- If your source instance is a subscription instance that has not expired, the new instance cannot inherit the remaining subscription duration. Releasing the source instance may result in a financial loss. For more information about unsubscription rules, see the Refund policy.
- The new instance does not inherit any discounts that were applied to the source instance. You can view the specific refund amount on the instance unsubscription page before you decide whether to upgrade.
- The refund amount and timing for a subscription instance are subject to the actual unsubscription bill. The refund is not processed in real time.

Precautions

Service impact: When you upgrade using blue-green deployment with a cutover, the source instance is set to read-only during the cutover process. This causes a transient connection interruption that lasts for several minutes. We recommend that you perform the upgrade during off-peak hours. If you choose not to perform a cutover, your services are not affected.
- The duration for which the source instance is read-only depends on the number of database objects. The more database objects the instance has, the longer the read-only duration. If the number of database objects reaches millions, the read-only duration can be tens of minutes or even hours. You can run the SELECT count(1) FROM pg_class; command to view the number of database objects.
- The transient connection interruption duration as perceived by the client depends on the DNS cache refresh time. You can switch the virtual switch and use the service interruption duration to estimate the client's DNS cache refresh time.
- The duration of the upgrade process depends on the number of database objects in the instance. The more database objects an instance has, the longer the upgrade takes. For major version upgrades, you can view the task progress in the Task Hub.
- For blue-green deployments, if you do not want the source instance to be set to read-only after the switchover, you must set the rds_force_trans_ro_non_sup parameter to off after the upgrade. For more information, see Set instance parameters.
Cross-version upgrades: ApsaraDB RDS for PostgreSQL 9.4 and 10 instances that use high-performance local disks can only be upgraded to instances that use cloud disks. You can upgrade them to a maximum version of ApsaraDB RDS for PostgreSQL 14. To upgrade to PostgreSQL 15 or a later version, you must first upgrade to an intermediate version (PostgreSQL 9.4 can be upgraded to 10, 11, 12, 13, or 14; PostgreSQL 10 can be upgraded to 11, 12, 13, or 14), and then upgrade to PostgreSQL 15 or a later version.
Replication Slots:
- If the source instance is a publisher with replication slots, the replication slots are lost after the upgrade.
- If the source instance is a replication slot subscriber, an upgrade may cause data synchronization issues due to replication slot preemption. To resolve this issue, see How to prevent data inconsistency caused by replication slot preemption during an upgrade.
Virtual IP: For a blue-green deployment with a cutover, the virtual IP address of the new instance changes after the upgrade. You must check the network connectivity, such as firewall configurations.
- Impact of virtual IP change: If you have configured the virtual IP in your application, you must modify the application configuration to point to the new instance's virtual IP.
- Recommendation: To avoid the complexity of manual configuration, we recommend that you directly configure the connection address of the instance in your application. For more information about how to obtain the connection address, see View or modify connection addresses and ports.
Parameter changes:
- If the original instance uses parameters that are not supported by the new version, these parameters are automatically deleted in the new version.
- If the value of a parameter in the original instance is not within the valid range of the corresponding parameter in the new version, the parameter is set to the default value in the parameter template of the new version.
- During the upgrade, the system temporarily changes the value of the statement_timeout parameter to 0 and restores the parameter to its original value after the upgrade is complete.
DTS tasks: If the instance to be upgraded is the source or destination instance of a Data Transmission Service (DTS) task, you must recreate the DTS task after the upgrade.
Plugin compatibility issues: When you perform a major version upgrade, the system automatically updates to the latest minor engine version, and you may encounter plugin compatibility issues.
A new instance does not inherit the Instance Name, tags, CloudMonitor alert rules, or backup data of the source instance.

Step 1: Pre-upgrade check

Log on to the ApsaraDB RDS console and go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the ID of the instance.
(Optional) If a read-only instance exists for the source instance, change the read-only instance endpoint in your application to the primary instance endpoint, and then delete the read-only instance.
Note
To ensure service stability, change the application endpoint during off-peak hours.
In the navigation pane on the left, click Major Version Upgrade.
Note
If Major Version Upgrade is not displayed in the navigation pane on the left, check the version and configuration of your ApsaraDB RDS for PostgreSQL instance. For more information, see Prerequisites.
On the Upgrade Check tab, click Create upgrade check report.
Select the upgrade version, set Upgrade Mode to Blue-green Deployment, and then click OK.
The instance status changes to Maintaining Instance. After the upgrade check is complete, the instance status changes to Running.
If the result of the upgrade check report is Success or Warning, you can proceed with the subsequent steps of the major engine version upgrade. If the result is Failed, click View Information, fix the abnormal check items based on the report content, and then perform the pre-upgrade check again. For more information about common errors and causes, see Understand the major engine version upgrade check report of an RDS PostgreSQL instance.
Important
- To ensure a smooth upgrade, if the result of the upgrade check report is Warning, we recommend that you fix the abnormal check items based on the report content and then perform the pre-upgrade check again until the check result is Success.
- If you create an extension in the primary instance after the upgrade check is successful, you must perform the check again.

Step 2: Upgrade major version

Click the Upgrade Instance tab, read the warning message, select a version under Select upgrade version, and then click Create Upgrade Task.
In the dialog box that appears, read the message and click OK.

In the Create Major Engine Version Upgrade Task section, Upgrade Mode is set to Blue-green Deployment. Configure the upgrade parameters. The following table describes only the key parameters.

Configuration	Description
Storage type	The new instance supports only enterprise SSDs (ESSDs) and premium performance disks. If you select ESSD, you can change the performance level (PL). The storage class of the new instance must be the same as that of the source instance. Note The minimum storage capacity of an ESSD varies based on the PL. When you change the PL, the minimum storage capacity of the instance is automatically adjusted based on the selected PL. Make sure that the new value meets the requirements. If the source instance uses a high-performance local disk, the new instance can only use ESSDs.
The Available Zone	The system lets you configure the new primary and secondary instances in different zones after the upgrade. Set these parameters as needed.
For Available Zone
Primary Instance Switch
Standby Instance Switch
Cutover configuration	Select whether to switch traffic to the new instance based on your requirements. No cutting: The traffic is not automatically switched. This option is typically used to test the compatibility of your services with the new version before the official upgrade. Cutover: The traffic is automatically switched. This option is typically used for the official upgrade after you confirm that your services can run stably on the new version. After the cutover, your application is automatically connected to the new instance. You do not need to change the database endpoint in your application. Note You cannot roll back after a cutover. Select this option with caution. During the cutover, the source instance is set to read-only, which causes a transient connection that lasts for minutes. Perform the upgrade during off-peak hours. If you choose not to perform a cutover, your services are not affected. We recommend that you select No cutting for your first attempt. After you fully test and verify the application layer, release the new instance, repeat the upgrade operation, and then select Cutover to begin the formal upgrade.
Storage space	Select the storage capacity for the new instance. When you upgrade the major version of an instance that uses a high-performance local disk, you can scale in the storage capacity. The minimum selectable storage capacity must meet the following conditions: The smaller of the following two values: Used storage of the source instance × 120%. If the result (in GB) is not a multiple of 5, it is rounded up to the nearest multiple of 5. Note You can view the used storage space of the source instance using the Monitoring And Alarms feature to check the Disk Storage (MB) metric. For more information, see View Enhanced Monitoring. Storage capacity of the source instance. Greater than or equal to the minimum purchasable storage capacity of an ESSD. If not, the following values are used as the minimum purchasable storage capacity: PL1 ESSD: 20 GB. PL2 ESSD: 500 GB. PL3 ESSD: 1500 GB. Note Examples: Your source instance has a storage capacity of 100 GB, and 70 GB is used. When you upgrade the major version, you select PL1 ESSD for the storage type. Calculation: 70 × 120% = 84. Rounded up to the nearest multiple of 5, the value is 85 GB. Therefore, the minimum purchasable storage capacity is 85 GB. Your source instance has a storage capacity of 700 GB, and 350 GB is used. When you upgrade the major version, you select PL2 ESSD for the storage type. Calculation: 350 × 120% = 420. 420 GB is less than the minimum purchasable storage capacity of a PL2 ESSD, which is 500 GB. Therefore, the minimum purchasable storage capacity is 500 GB.
Instance specification	Specify the instance type of the new instance. For more information, see List of Primary Instance Types.

Click Create now.
When the instance status changes to Migration in Progress, the upgrade task has started.
The time required for an upgrade is closely related to the number of database objects in the instance. The more database objects there are, the longer the upgrade takes. During a major version upgrade, you can view the upgrade progress in the Task Hub.
Important
- After an upgrade task is created, it cannot be modified or deleted.
- When the source instance is in the Migration in Progress state, you cannot perform O&M operations such as modifying parameters, restarting, or releasing the instance.
- If the Insufficient Resources message appears when you create the task, switch the Target Primary Zone.
View the upgrade result.
When the status of both the source and new instances is Running, the upgrade is successful.
Note
- For blue-green deployment with cutover, traffic is automatically switched to the new (higher-version) instance after the upgrade.
- For blue-green deployment without cutover, traffic is not switched to the new (higher-version) instance after the upgrade.
- After the upgrade is complete, on the Upgrade History tab, click View Information in the Upgrade Log column for the target upgrade task. You can view the read-only duration of the instance and the detailed upgrade process. The read-only duration is the period between the Cutover Time and the Cutover End Time. This period does not include the time during which the instance is unreachable because the DNS cache is not refreshed.
- For blue-green deployment without cutover, the system also displays the Cutover Time and Cutover End Time for your reference and estimation in a cutover scenario.

What to do next

If you upgrade an instance using the blue-green deployment (cutover) pattern, promptly release the source instance after you confirm that your services are running stably on the new instance.
We also recommend that you change the billing method for the new instance to subscription for cost savings.
Note
- If you release a subscription instance before its expiration date, you may incur a loss.
- If you received a discount when you purchased the source instance, the new instance does not inherit this discount. The unsubscription of the source instance is subject to the actual unsubscription page.
- The refund amount and timing for a subscription instance are subject to the actual unsubscription bill. The refund is not processed in real time.
(Optional) The new instance does not include read-only instances. If you deleted a read-only instance before the upgrade, perform the following steps:
1. On the new instance, create a PostgreSQL read-only instance again.
2. In your application, change the endpoint that you previously set to the primary instance endpoint back to the new read-only instance endpoint.

Upgrade result description

The Upgrade Result column on the Upgrade History tab displays one of the following statuses for the upgrade task.

Upgrade result	Instance status	Meaning	Available actions
Running	Migration in Progress	The upgrade task is running.	None.
Succeeded	Running	The upgrade task is successful.	None.

Related API operations

API	Description
UpgradeDBInstanceMajorVersionPrecheck	Performs a pre-upgrade check on an RDS PostgreSQL instance.
DescribeUpgradeMajorVersionPrecheckTask	Queries the pre-upgrade check report of an RDS PostgreSQL instance.
UpgradeDBInstanceMajorVersion	Upgrades the major engine version of an RDS PostgreSQL instance.
DescribeUpgradeMajorVersionTask	Queries the historical major engine version upgrade tasks of an RDS PostgreSQL instance.

References

FAQ

Can I change the configuration such as the instance type of my RDS instance during the major engine version upgrade of the RDS instance?

No, you cannot change the instance during the major engine version upgrade. You must wait until the major engine version upgrade is complete before you can perform other operations.

Can the major engine version of an RDS instance be automatically upgraded?

No, the major engine version of an RDS instance cannot be automatically upgraded.

Can I downgrade the major engine version?

After the upgrade, you cannot downgrade the major engine version. If you need to downgrade, purchase a low-version instance and use DTS to migrate the instance to the low version.

After a major version upgrade, a raster_overviews conflict error occurs when I create a raster_overviews view in the new instance. How do I fix this?

If the PostGIS version is earlier than 2.5.2 and the RDS PostgreSQL version is 10 or 11, after you upgrade the extension and then upgrade the major engine version to PostgreSQL 12, a view incompatibility issue may occur when you create the raster_overviews view in the new instance.

You can use one of the following methods to address this issue:

Update the version of PostGIS extensions in the original RDS instance.
Execute the following statements twice to ensure that the update is successful:
```
SELECT PostGIS_Extensions_Upgrade();
SELECT PostGIS_Extensions_Upgrade();
```

Determine whether the postgis_raster extension is used and select an update method based on your business requirements.

The postgis_raster extension is used

Execute the following statements in the original RDS instance to modify the raster_overviews view:

ALTER EXTENSION PostGIS_Raster DROP VIEW raster_overviews;
CREATE OR REPLACE VIEW raster_overviews AS SELECT 1;

Upgrade the major version of the RDS instance to PostgreSQL 12 or later.

After the upgrade is complete, re-create the view in the new RDS instance.

CREATE 
OR REPLACE VIEW raster_overviews AS 
SELECT 
  current_database() AS o_table_catalog, 
  n.nspname AS o_table_schema, 
  c.relname AS o_table_name, 
  a.attname AS o_raster_column, 
  current_database() AS r_table_catalog, 
  split_part(
    split_part(s.consrc, '''::name', 1), 
    '''', 
    2
  ): :name AS r_table_schema, 
  split_part(
    split_part(s.consrc, '''::name', 2), 
    '''', 
    2
  ): :name AS r_table_name, 
  split_part(
    split_part(s.consrc, '''::name', 3), 
    '''', 
    2
  ): :name AS r_raster_column, 
  trim(
    both 
    from 
      split_part(s.consrc, ',', 2)
  ): :integer AS overview_factor 
FROM 
  pg_class c, 
  pg_attribute a, 
  pg_type t, 
  pg_namespace n, 
  (
    SELECT 
      connamespace, 
      conrelid, 
      conkey, 
      pg_get_constraintdef(oid) As consrc 
    FROM 
      pg_constraint
  ) AS s 
WHERE 
  t.typname = 'raster' : :name 
  AND a.attisdropped = false 
  AND a.atttypid = t.oid 
  AND a.attrelid = c.oid 
  AND c.relnamespace = n.oid 
  AND c.relkind = ANY(
    ARRAY[ 'r' : :char, 'v' : :char, 'm' : :char, 
    'f' : :char ]
  ) 
  AND s.connamespace = n.oid 
  AND s.conrelid = c.oid 
  AND s.consrc LIKE '%_overview_constraint(%' 
  AND NOT pg_is_other_temp_schema(c.relnamespace) 
  AND has_table_privilege(c.oid, 'SELECT' : :text); ALTER EXTENSION PostGIS_Raster 
ADD 
  VIEW raster_overviews;

The postgis_raster extension is not used

Execute the following statement in the original RDS instance to delete the postgis_raster extension:
```
DROP EXTENSION PostGIS_Raster;
```
Upgrade the major version of the RDS instance to PostgreSQL 12 or later.

How can I prevent data inconsistency caused by replication slot preemption during an upgrade?

To keep the subscription data on the source (lower-version) instance, make sure that the source instance does not fail due to an excessive payload during the upgrade. Otherwise, the replication slot may be preempted by the destination (higher-version) instance, which leads to data inconsistency.
After the upgrade is complete, run the following SQL statement to disable the subscription on the database of the destination instance.
```
\c your_database
ALTER SUBSCRIPTION your_subscription_name DISABLE;
```
To preserve the subscription data on the destination instance, disable the subscription on the source instance before the major version upgrade. After the upgrade is complete, enable the subscription on the destination instance. The following are sample SQL statements:
- Disable the subscription on the source instance.
```
\c your_database
ALTER SUBSCRIPTION your_subscription_name DISABLE;
```
- Enable the subscription on the destination instance.
```
\c your_database
ALTER SUBSCRIPTION your_subscription_name ENABLE;
```

Note

For more information about change tracking with replication slots, see Logical Subscription and Change Tracking.
If you do not perform the preceding operations, your subscribed data may become inconsistent. For more information about how to resolve this issue, see How do I handle inconsistent subscription data after an upgrade?.

How do I handle the issue that the subscribed data is inconsistent after an upgrade?

After the upgrade is successful, clear the table data in the new RDS instance, re-create the subscription, and set copy_data=true. For more information, see ALTER SUBSCRIPTION.

Use the CONFLICT keyword to import data consumed on the original RDS instance to the new RDS instance. Example:

CREATE TABLE my_tbl(id INT PRIMARY KEY, t TIMESTAMP, val TEXT);

INSERT INTO my_tbl VALUES (1, CURRENT_TIMESTAMP, 'a');
INSERT INTO my_tbl VALUES (2, CURRENT_TIMESTAMP, 'b');
INSERT INTO my_tbl VALUES (3, CURRENT_TIMESTAMP, 'c');

-- in case with newer timestamp: do update
INSERT INTO my_tbl VALUES (1, CURRENT_TIMESTAMP, 'd') ON CONFLICT(id) DO UPDATE
	SET t = excluded.t,
		val = excluded.val
	WHERE my_tbl.t < excluded.t;

-- in case with older timestamp: do nothing
INSERT INTO my_tbl VALUES (2, CURRENT_TIMESTAMP - '10 hours'::interval, 'e') ON CONFLICT(id) DO UPDATE
	SET t = excluded.t,
		val = excluded.val
	WHERE my_tbl.t < excluded.t;

-- in case with new val: just insert
INSERT INTO my_tbl VALUES (5, CURRENT_TIMESTAMP - '10 hours'::interval, 'f') ON CONFLICT(id) DO UPDATE
	SET t = excluded.t,
		val = excluded.val
	WHERE my_tbl.t < excluded.t;

Why do I need to delete read-only instances before a major version upgrade?

After the upgrade, read-only nodes and replication slots remain on the source instance and are not automatically transferred to the new instance. Therefore, before you perform a major version upgrade, you must change the read-only instance endpoint in your application to the primary instance endpoint and delete the existing read-only instance. After the upgrade is complete, create and configure a new read-only instance.

Billing method of the source instance	Billing method of the new instance
Subscription/Pay-as-you-go	Pay-as-you-go
Serverless	Serverless