All Products
Search
Document Center

ApsaraDB RDS:Migrate the data of a PostgreSQL instance that has a public IP address to an ApsaraDB RDS for PostgreSQL instance

Last Updated:Sep 22, 2023

This topic describes how to use the cloud migration feature of ApsaraDB RDS for PostgreSQL to migrate the data of a PostgreSQL instance that has a public IP address to an ApsaraDB RDS for PostgreSQL instance. The PostgreSQL instance can be a self-managed PostgreSQL instance or a PostgreSQL instance of a third-party cloud service provider.

Background information

Your RDS instance resides in a virtual private cloud (VPC). You can use the cloud migration feature to migrate the data of a PostgreSQL instance to the RDS instance over the internal network. If you want to use the cloud migration feature to migrate the data of a PostgreSQL instance that has a public IP address to the RDS instance, you must configure an Internet NAT gateway for the RDS instance and associate an elastic IP address (EIP) with the Internet NAT gateway.

This topic also describes how to configure an Internet NAT gateway for an RDS instance and associate an EIP with the Internet NAT gateway.

This configuration enables migration over the Internet helps ensure the network security of the RDS instance during the cloud migration.

Note

Source network address translation (SNAT) entries that are created for an Internet NAT gateway allow an RDS instance to access the Internet but does not allow the RDS instance to provide services over the Internet. For more information about Internet NAT gateways and SNAT, see Use the SNAT feature of an Internet NAT gateway to access the Internet.

Scenarios

  • You want to migrate the data of a self-managed PostgreSQL instance that has a public IP address to an RDS instance.

  • You want to migrate the data of a PostgreSQL instance that is provided by a third-party cloud service provider, such as Google or Amazon, to an RDS instance.

    Note

    Third-party cloud service providers may use self-developed or custom PostgreSQL extensions. If an extension incompatibility issue occurs during the cloud migration, you can submit a ticket.

Prerequisites

The following requirements are met:

  • The major engine versions of the PostgreSQL instance and the RDS instance are the same. PostgreSQL 10 or later is supported.

    Note
    • If the major engine version of the PostgreSQL instance is earlier than PostgreSQL 10, you must upgrade the major version before you perform the cloud migration.

    • If the major engine version of the PostgreSQL instance is different from that of the RDS instance, the cloud migration assessment fails, and you cannot perform the cloud migration.

  • The RDS instance is a primary RDS instance. Read-only RDS instances do not support the cloud migration feature.

  • The RDS instance uses cloud disks.

  • The RDS instance is empty. The available storage of the RDS instance is greater than or equal to the size of the data in the PostgreSQL instance.

    Note

    If data exists in the RDS instance, the cloud migration assessment fails, and you cannot perform the cloud migration. You must back up the data and clear the data on the RDS instance or create another RDS instance before you perform the cloud migration.

Limits

None.

Impacts

None.

Usage notes

During the cloud migration, make sure that the PostgreSQL instance can be accessed. Do not perform operations such as instance restart. If transient connections or switchovers that are required to achieve high availability occur during the cloud migration, the cloud migration fails.

Billing rules

You must configure an Internet NAT gateway for the RDS instance and associate an EIP with the Internet NAT gateway. You are charged for the Internet NAT gateway. For more information, see Billing of Internet NAT gateways.

Procedure

Step 1: Configure an Internet NAT gateway and associate an EIP with the Internet NAT gateway

This step describes how to associate a public IP address with the VPC of your RDS instance. After a public IP address is associated, your RDS instance can access a self-managed PostgreSQL instance that has a public IP address or a PostgreSQL instance of a third-party cloud service provider.

Important

Before the configuration, make sure no NAT gateways are configured for the VPC in which the RDS instance resides and no other public IP addresses are associated. This helps prevents connectivity issues.

  1. Go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the ID of the instance.
  2. In the left-side navigation pane, click Database Connection. In the Database Connection section of the page that appears, click the VPC link to the right of the Network Type parameter.

    image..png
  3. In the VPC console, click the Resources tab. In the Access to Internet section, click Create Now.

    image..png
  4. On the Internet NAT Gateway page, configure the following parameters and retain the default values for the parameters that are not described in the following table.

    Parameter

    Description

    Example

    EIP

    If an EIP has been associated with the VPC, set this parameter to Select EIP. In this topic, Purchase EIP is selected.

    Purchase EIP

    Maximum Bandwidth

    You must specify the maximum bandwidth value based on your business requirements. To facilitate the cloud migration, you can set this parameter to the maximum value to prevent slow cloud migration issues due to bandwidth limits.

    200Mbps

  5. Click Buy Now.

  6. Confirm the parameter settings, read and select Terms of Service, and then click Confirm.

    If the page that is shown in the following figure appears, the Internet NAT gateway is created, an EIP is associated with the Internet NAT gateway, and SNAT entries are created for the Internet NAT gateway.

    image..png
  7. Click the resource ID of the EIP. On the Instance Information tab of the EIP, view the public IP address.

    image..png

Step 2: Configure the self-managed PostgreSQL instance or the PostgreSQL instance of the third-party cloud service provider

Configure the self-managed PostgreSQL instance

  1. Configure the postgresql.conf file of the self-managed PostgreSQL instance.

  2. Create an account for cloud migration on the self-managed PostgreSQL instance.

  3. Update the pg_hba.conf file of the self-managed PostgreSQL instance.

    Important

    The IP address configured in the pg_hba.conf file is a CIDR block of EIP that is associated with your RDS instance in Step 1/32.

  4. Configure the firewall of the server on which the self-managed PostgreSQL instance resides.

Configure the PostgreSQL instance of a third-party cloud service provider

  1. Change the value of the wal_keep_segments or wal_keep_size parameter. For more information, see the official documentation of each cloud service provider.

    View the minimum size of log files that can be retained in the pg_wal directory. If the minimum size exceeds, WAL logs of the PostgreSQL instance are deleted after a full backup. In this case, you must back up the PostgreSQL instance again.

    • wal_keep_segments: This parameter is applicable to instances that run PostgreSQL 10, PostgreSQL 11, and PostgreSQL 12. We recommend that you set this parameter to a value that is greater than or equal to 4096.

    • wal_keep_size: This parameter is applicable to instances that run PostgreSQL 13, PostgreSQL 14, and PostgreSQL 15. We recommend that you set this parameter to a value that is greater than or equal to 65536.

  2. Create an account for cloud migration.

    CREATE USER migratetest CREATEROLE REPLICATION LOGIN PASSWORD '123456';
    GRANT pg_monitor TO migratetest;
    Note

    In the preceding statements, migratetest and 123456 are examples. You need to replace them with actual values.

  3. Enable the public IP address and configure a security group or whitelist to allow the RDS instance to access the PostgreSQL instance of the third-party cloud service provider by using the EIP that is associated with the RDS instance in Step 1. For more information, see the official documentation of the cloud service provider.

    Important

    You must make sure that the Internet Control Message Protocol (ICMP) protocol is enabled for the PostgreSQL instance of the third-party cloud service provider. You can run the ping <Public endpoint of the PostgreSQL instance of the third-party cloud service provider> command to check whether the ICMP protocol is enabled.

Step 3: Assess a cloud migration task

  1. Go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the ID of the instance.
  2. In the left-side navigation pane, click Migrate to Cloud. On the page that appears, click the Migration Assessment tab.

  3. In the Select Migration Source step of the configuration wizard, select an option and click Next.

  4. In the Configure Destination Database step of the configuration wizard, click Next.

  5. In the Configure Source Database step of the configuration wizard, select all options and click Next.

  6. In the Initiate Migration Assessment step of the configuration wizard, configure the following parameters.

    Parameter

    Description

    Migration Task Name

    The system automatically generates a name for the cloud migration task. You do not need to modify the generated name.

    Source VPC/DNS IP

    • If the source instance is a self-managed PostgreSQL instance that has a public IP address, set this parameter to the public IP address of the server on which the self-managed PostgreSQL instance is deployed.

    • If the source instance is a PostgreSQL instance of a third-party cloud service provider, set this parameter to the public endpoint of the PostgreSQL instance.

    Source Port

    • If the source instance is a self-managed PostgreSQL instance that has a public IP address, you can run the netstat -a | grep PGSQL command to view the port.

    • If the source instance is a PostgreSQL instance of a third-party cloud service provider, you can go to the console of the third-party cloud service provider to view the port.

    Username

    The username and password that are created for the cloud migration task in Step 2.

    Password

  7. Click Create Migration Assessment Task.

    After the migration assessment task is complete, you can view the status of the task on the Migration Assessment tab.

    • If Successful is displayed in the Status column of the migration assessment task, you can Migration to Cloud.

    • If Failed is displayed in the Status column of the migration assessment task, you can click View Report in the Actions column to view and handle the reported errors. For more information about common errors, see Introduction to cloud migration assessment reports.

      Note

      Third-party cloud service providers may use self-developed or custom PostgreSQL extensions. If an extension incompatibility issue occurs during the migration, you can submit a ticket.

    After you handle the reported errors, you can click Re-assess in the Actions column to run the cloud migration assessment task again.

Step 4: Start the cloud migration

Important

You can start the cloud migration only when the status of the migration assessment task indicates a success.

  1. Go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the ID of the instance.
  2. In the left-side navigation pane, click Migrate to Cloud. On the page that appears, click the Migration to Cloud tab. On the tab that appears, click Create Cloud Migration Task.

  3. In the Create Cloud Migration Task dialog box, select the migration assessment task whose status indicates a success in Step 3 from the Associated Assessment Task drop-down list.

    After you select a migration assessment task from the Associated Assessment Task drop-down list, the system automatically obtains the values of the required parameters.

  4. Click Initiate Migration to Cloud. The system automatically starts the cloud migration task.

    Warning

    During the cloud migration, make sure that the PostgreSQL instance can be accessed. Do not perform operations such as instance restart. If transient connections or switchovers that are required to achieve high availability occur during the cloud migration, the cloud migration fails.

  5. Switch the workloads.

    1. Click the link in the Cloud Migration Phase column to view the progress of the cloud migration task.

    2. When the cloud migration task enters the phase of Incremental Data Synchronization, click Switchover in the Actions column of the cloud migration task to switch the workloads of the PostgreSQL instance to the RDS instance.

    3. In the Switchover dialog box, configure the PostgreSQL instance to process only read requests. Alternatively, stop the connected application from writing data to the PostgreSQL instance.

      You can configure the PostgreSQL instance to process only read requests.

      -- Configure the PostgreSQL instance to process only read requests.
      ALTER SYSTEM SET default_transaction_read_only=on;
      
      -- Reload the parameter configuration for the modification to take effect.
      SELECT pg_reload_conf();
      
      -- Terminate all existing sessions.
      SELECT pg_terminate_backend(pid) FROM pg_stat_activity 
      WHERE usename not in ('replicator', 'monitor', 'pgsql', 'aurora') AND pid != pg_backend_pid();
    4. Select all check boxes and click Switch Now. Then, wait until the cloud migration is complete.