Migration between PolarDB for PostgreSQL (Compatible with Oracle) clusters
Use Data Transmission Service (DTS) to migrate data between clusters.
Background
is a widely adopted, enterprise-grade relational database backed by a strong community. Each new version enhances performance, availability, and security with new features and improvements. Upgrading to 2.0 significantly improves your database performance and user experience.
Version 2.0 introduces a new storage engine and query optimization algorithms that increase query speed and concurrency. This enables your database to process large data volumes faster, boosting responsiveness and overall performance. In addition, Version 2.0 includes features that enhance the user experience and developer tooling. For example, it offers enhanced support for the JSON data type, making it easier to process and query JSON data. It provides comprehensive monitoring and diagnostics tools for understanding and optimizing database performance. It also enhances database security and reliability with stricter access control and permission management to protect your data from potential threats. Improved backup and recovery functions simplify data protection and restoration.
Upgrading to version 2.0 also helps you keep pace with the PostgreSQL community, providing access to more resources and support for resolving issues and learning best practices. We strongly recommend upgrading to 2.0 to leverage its full capabilities.
Migration assessment
If you have a version 1.0 cluster, you can use the migration assessment feature to run a compatibility pre-check before migrating. This helps you identify and address potential issues in advance, which reduces processing and resource costs during the migration.
Supported regions
Migration assessment is supported in the following regions:
China (Hangzhou), China (Shanghai), China (Shenzhen), China (Beijing), China (Zhangjiakou), China (Ulanqab), China (Chengdu), China (Hong Kong), Japan (Tokyo), Singapore, Indonesia (Jakarta), US (Silicon Valley), and US (Virginia).
Impact
Migration assessment does not affect your workloads.
Create a migration assessment task
-
Log on to the PolarDB Console.
-
In the upper-left corner, select the region where the cluster is located.
-
Create a migration assessment task. PolarDB provides two entry points:
-
From the Clusters page: In the upper-left corner of the Clusters page, click Migration/Upgrade Assessment.
In the right-side panel, set creation method to Upgrade from PolarDB and select Oracle 1.0 for source PolarDB version. For source PolarDB instance, select your source cluster. Then, for target database engine, select Oracle 2.0 and for database name, select the database to migrate (for example,
yushudb). Click Next. -
From the Migration/Upgrade page: In the upper-left corner of the Migration/Upgrade page, click Migration/Upgrade Assessment.
-
-
In the Migration/Upgrade Assessment dialog box, configure the migration assessment task parameters and click Next.
Parameter
Description
creation method
Select Upgrade from PolarDB.
source PolarDB version
Select Oracle 1.0.
source PolarDB instance
Select the source Oracle 1.0 cluster to migrate.
target database engine
Select Oracle 2.0.
database name
Select the source database to migrate.
-
In the Migration/Upgrade Assessment dialog box, you can view a high-level compatibility summary and detailed results.
-
View the summary.
The Overall Description tab displays a pie chart summarizing compatible and incompatible objects, and a table detailing compatibility by object type (view, function, index, constraint, and table). The table includes Number of Compatible Objects, Number of Incompatible Objects, and Total Number of Objects.
-
View details.
In the Compatible drop-down list, select Incompatible to filter the results. The table shows the schema, object type, object name, and DDL of incompatible objects.
NoteFocus on the incompatible objects in the assessment results.
-
Native Oracle database objects can be ignored.
-
Incompatible objects referenced in your application SQL must be modified for compatibility. If you cannot make these modifications, you can contact us. Our engineers will assist you.
-
-
Manage migration assessment tasks
You can view the details of an existing assessment task on the Migration/Assessment page.
A migration assessment task is valid for seven days and is automatically deleted upon expiration. If a task expires, create a new one.
Prerequisites
-
You have created the source and destination clusters. For details, see Create a PolarDB for PostgreSQL (Compatible with Oracle) cluster.
-
You have set the wal_level parameter of the source and destination clusters to logical. For instructions, see Configure cluster parameters.
-
The storage of the destination database must exceed that used by the source database.
Limitations
-
During schema synchronization, DTS synchronizes foreign keys from the source database to the destination database.
-
During full data synchronization and incremental data synchronization, DTS temporarily disables constraint checks and foreign key cascade operations at the session level. If cascade update or delete operations occur in the source database while the task is running, data inconsistency may occur.
|
Type |
Description |
|
Source database limitations (PolarDB 1.0) |
|
|
Other limitations |
|
Billing
|
Migration type |
Cost |
|
Schema migration and full data migration |
Free |
|
Incremental data migration |
Charged. See the billing overview for details. |
Migration types
|
Type |
Description |
|
Schema migration |
DTS migrates the schema definitions of objects to the destination. Supported objects include tables, views, synonyms, stored procedures, stored functions, packages, and user-defined types. Note
Triggers are not currently supported. Delete triggers from the source to prevent data inconsistency. For more information, see Configure a data synchronization or migration task for a source database that contains a trigger. |
|
Full data migration |
DTS migrates all existing data of the objects from the source to the destination. Note
Do not perform any DDL operation on the objects until the schema migration and full data migration are complete. Otherwise, the migration may fail. |
|
Incremental data migration |
After the full data migration is complete, DTS captures incremental updates from the WAL log of the source and applies them to the destination. This allows a smooth migration with minimal application downtime. |
Supported SQL operations
|
Operation type |
SQL statement |
|
DML |
INSERT, UPDATE, and DELETE |
|
DDL |
Important
If the source database account is a high-privilege account, the data synchronization task supports the following DDL operations:
Important
|
Database account permissions
|
Database |
Permissions |
|
source cluster |
A high-privilege account. |
|
target cluster |
An account with database owner permissions. Important
The database owner is specified when you create the database. See Create a database account for instructions on creating a database account and granting permissions. |
Procedure
-
Navigate to the migration task list page for the destination region using one of the following methods.
From the DTS console
-
Log on to the Data Transmission Service (DTS) console.
-
In the navigation pane on the left, click Data Migration.
-
In the upper-left corner of the page, select the region where the migration instance is located.
From the DMS console
NoteThe actual operations may vary based on the mode and layout of the DMS console. For more information, see Simple mode console and Customize the layout and style of the DMS console.
-
Log on to the Data Management (DMS) console.
-
In the top menu bar, choose .
-
To the right of Data Migration Tasks, select the region where the migration instance is located.
-
-
Click Create Task and configure the source and destination databases.
NoteFor Database type, select PolarDB for PostgreSQL (Oracle-compatible) for both the source and destination databases.
Category
Parameter
Description
N/A
Task name
DTS automatically generates a task name. Specify a descriptive name for easy identification. The name does not need to be unique.
Source database (PolarDB 1.0)
Select an existing DMS database instance
You can choose to use an existing instance.
-
If you use an existing instance, DTS automatically fills in the database information.
-
If not, enter the database information below.
Database type
Select PolarDB for PostgreSQL (Oracle-compatible).
Access method
Select Cloud Instance.
Instance region
Select the region of the source cluster.
Cross-Alibaba Cloud account
To synchronize data within the same Alibaba Cloud account, select No.
Instance ID
Select the instance ID of the source cluster.
Database name
Enter the name of the source database.
Database account
Enter the database account of the source cluster.
Database password
Enter the password for the database account.
Destination database (PolarDB 2.0)
Select an existing database instance
You can choose to use an existing instance.
-
If you use an existing instance, DTS automatically fills in the database information.
-
If not, enter the database information below.
Database type
Select PolarDB for PostgreSQL (Oracle-compatible).
Access method
Select Cloud Instance.
Instance region
Select the region of the destination cluster.
Instance ID
Select the instance ID of the destination cluster.
Database name
Enter the name of the destination database.
Database account
Enter a privileged database account for the destination cluster. For instructions on how to create an account and grant permissions, see Create a database account.
Database password
Enter the password for the database account.
-
-
After completing the configuration, click Test Connection and Proceed at the bottom of the page.
DTS automatically adds the IP addresses of the DTS servers in the corresponding region to the whitelist of the Alibaba Cloud database instance.
WarningAdding the public IP addresses of DTS servers to a whitelist, whether automatically by DTS or manually, may pose security risks. By using this product, you acknowledge and accept these potential risks and are responsible for implementing basic security measures. These measures include but are not limited to strengthening account passwords, restricting open ports for each CIDR block, using authentication for internal API communication, and periodically reviewing and restricting unnecessary network segments. Alternatively, you can connect to your database privately by using Express Connect, a VPN Gateway, or Smart Access Gateway (SAG).
-
Configure the synchronization objects and advanced settings.
NoteThe Source Objects box lists all namespaces. Select the namespaces that you want to migrate.
Parameter
Description
Synchronization type
Incremental Synchronization is selected by default. You must also select Schema Synchronization and Full Synchronization. After the pre-check is complete, DTS initializes the full data of the objects to be synchronized in the destination cluster. This data serves as the baseline data for subsequent incremental synchronization.
NoteYou must select all synchronization types.
Synchronization topology
-
One-way Topology: Creates a one-way synchronization link from the source to the destination database, supporting both full and incremental synchronization.
-
Two-way synchronization: Creates a two-way synchronization link between the source and destination databases. This link performs incremental synchronization from source to destination and reverse synchronization from destination to source.
NoteReverse synchronization supports only data synchronization, not DDL operations.
Processing mode for existing tables in destination
-
Pre-check and Report an Error: Checks for tables with matching names in the destination database. The pre-check passes if no matching table is found. If a match is found, the pre-check fails with an error, and the data synchronization task does not start.
NoteIf you cannot delete or rename the table with the same name in the destination database, you can change its name in the destination database. For more information, see Map schema, table, and column names.
-
Ignore and Continue: Skips the check for tables with the same name in the destination database.
WarningSelecting Ignore and Continue may lead to data inconsistency and pose business risks. For example:
-
If the schemas are identical and the destination table contains a record with the same primary key or unique key value as a source record:
-
During full synchronization, DTS retains the existing record in the destination cluster, and the source record is not synchronized.
-
During incremental synchronization, the source record overwrites the existing record in the destination database.
-
-
If the table structures are inconsistent, the task may fail to initialize data, synchronize only a subset of columns, or fail completely. Use this option with caution.
-
Case policy for destination object names
You can configure the case policy for the database, table, and column names of synchronized objects in the destination instance. By default, DTS Default Policy is selected. You can also choose to align with the default policy of the source or destination database. For more information, see Case policy for destination object names.
Source objects
In the Source Objects box, select the objects to synchronize and click
to move them to the Selected Objects box.NoteYou can select objects at the schema, table, or column level. If you select tables or columns, other objects such as views, triggers, and stored procedures are not synchronized to the destination database.
Selected objects
-
To change the name of an individual object in the destination instance, right-click the object in the Selected Objects box. For instructions, see Map schema, table, and column names.
-
To change the names of multiple objects in the destination instance in bulk, click Batch Edit in the upper-right corner of the Selected Objects box. For instructions, see Map schema, table, and column names.
Note-
To select SQL operations for synchronization at the schema or table level, right-click an object in the Selected Objects box and select the required SQL operations in the dialog box that appears. For a list of supported operations, see Supported SQL operations.
-
To filter data by using a WHERE clause, right-click a table in the Selected Objects box and set the filter condition in the dialog box that appears. For instructions, see Set filter conditions.
-
-
Click Next: Advanced Configuration to configure advanced settings.
Parameter
Description
Select dedicated cluster for this task
A DTS dedicated cluster is a group of ECS virtual machines of the same specification in a region that manages DTS migration, synchronization, and subscription tasks. Compared to shared DTS clusters, dedicated clusters offer exclusive resources, better stability, higher performance, and lower costs. For more information, see What is a DTS dedicated cluster?.
Retry duration after connection to source/destination database is lost
After a synchronization task starts, if the connection to the source or destination database fails, DTS reports an error and immediately starts retrying the connection. The default retry duration is 720 minutes. You can set a custom duration from 10 to 1,440 minutes. We recommend that you set it to 30 minutes or more. If DTS successfully reconnects within the specified duration, the task automatically resumes. Otherwise, the task fails.
Note-
If you have multiple DTS instances (for example, Instance A and Instance B) with the same source or destination, and you set the retry duration to 30 minutes for Instance A and 60 minutes for Instance B, the shorter duration of 30 minutes is used.
-
You are charged for the task running time during the connection retry period. We recommend that you customize the retry duration based on your business needs, or release the DTS instance as soon as the source and destination database instances are released.
Retry duration after other issues occur in source/destination database
After a synchronization task starts, if a non-connection issue (such as a DDL or DML execution exception) occurs in the source or destination database, DTS reports an error and immediately starts retrying the operation. The default retry duration is 10 minutes. You can set a custom duration from 1 to 1,440 minutes. We recommend that you set it to 10 minutes or more. If the operation succeeds within the specified duration, the task automatically resumes. Otherwise, the task fails.
ImportantThe value for Retry Duration After Other Issues Occur in Source/Destination Database must be less than the value for Retry Duration After Connection to Source/Destination Database Is Lost.
Limit full migration rate
Full migration initiates multiple concurrent reads from the source database and writes to the destination database, which can increase the load on your databases. To minimize the impact, you can limit the maximum migration rate.
Limit incremental migration rate
Incremental migration initiates multiple concurrent reads from the source database and writes to the destination database. High service load on one database can increase the write pressure on the other. To minimize the impact, you can limit the maximum migration rate.
Environment tag
Indicates the importance of the DTS task. Environment tags do not affect normal task operation.
Configure ETL function
Specify whether to configure the ETL function. For more information about ETL, see What is ETL?.
-
Yes: Configure the ETL function and enter the data processing statements in the text box. For more information, see Configure ETL for a DTS migration or synchronization task.
-
No: Do not configure the ETL function.
Monitoring and alarms
Specify whether to enable alarms. If the synchronization fails or latency exceeds the specified threshold, DTS notifies the alert contacts.
-
Disable: Do not set an alarm.
-
Enable: Set an alarm. You must also specify an alarm threshold and alert contacts. For more information, see Configure monitoring and alarms.
-
-
Click Next: Data Validation to configure data validation.
Parameter
Description
Data validation method
-
Full Data Validation: Validates all data. If you select Full Data Verification, you need to configure validation parameters and objects. For more information, see Configure data validation in a DTS synchronization or migration instance.
-
Incremental Data Validation: Selected by default. Validates only the incremental data being transferred.
-
Schema Validation: Selected by default. Validates the schemas of the objects that you specify for validation.
-
-
Save the task and run a pre-check.
-
To view the parameters for configuring this instance via an API call, hover your pointer over the Next: Save Task and Run Pre-check button and click Preview OpenAPI Parameters in the pop-up.
-
If you do not need to view the API parameters, or have finished viewing them, click Next: Save Task and Run Pre-check at the bottom of the page.
Note-
A pre-check runs before the synchronization task starts. The task starts only after it passes the pre-check.
-
If the pre-check fails, click View Details for the failed check item. Follow the instructions to fix the issue, and then run the pre-check again.
-
If the pre-check returns a warning:
-
For items that cannot be ignored, click View Details for the failed check item. Follow the instructions to fix the issue, and then run the pre-check again.
-
For items that can be ignored, click Confirm Alert Details, Acknowledge and Ignore, OK, and Rerun Pre-check to skip the alert item and run the pre-check again. If you choose to ignore an alert, you may cause data inconsistency or other business risks.
-
-
-
-
When the Pre-check Pass Rate is 100%, click Next: Purchase.
-
On the Purchase page, select the billing method and link specification for the data synchronization instance. The following table describes the parameters.
Category
Parameter
Description
Configuration
Billing method
-
Subscription: You pay when you create the instance. This is suitable for long-term needs and is more cost-effective than Pay-As-You-Go. The longer the subscription duration, the greater the discount.
-
Pay-As-You-Go: You are charged on an hourly basis. This is suitable for short-term needs. You can release the instance immediately after use to save costs.
Resource group configuration
The resource group to which the instance belongs. The default is default resource group. For more information, see What is Resource Management?.
Link specification
DTS offers various synchronization specifications with different performance levels. The link specification affects the synchronization speed. Select a specification that meets your needs. For more information, see Data synchronization link specifications.
Subscription duration
If you select the Subscription billing method, specify the instance duration and quantity. You can select a monthly subscription for 1 to 9 months, or a yearly subscription for 1, 2, 3, or 5 years.
NoteThis option is available only when Billing Method is set to Subscription.
-
-
After completing the configuration, read and select the Data Transmission (Pay-As-You-Go) Terms of Service check box.
-
Click Purchase and Start to begin the synchronization task. You can monitor the task's progress on the Data Synchronization page.
Procedure
Execution overview
-
pre-check: A preparatory step before a data migration or synchronization task. It verifies prerequisites such as network connectivity, system permissions, data format, and integrity.
-
pre-task: A task that preprocesses data, for example, by creating triggers for subsequent data operations.
-
incremental data capture: Captures data that has changed since the last operation, improving efficiency by not processing unchanged data.
-
schema migration: Transfers the database schema from a source to a target system while maintaining structural consistency. This is usually the initial migration step for setting up the data structure in the new environment. This step completes most of the migration work, including table structures, sequences, stored procedures and functions, views, and indexes.
-
full data migration: Migrates all data from a source system to a target system.
-
schema migration 2: A second round of schema adjustments after the initial migration. This step fine-tunes the data structure or resolves issues found during the initial migration. This step mainly creates foreign keys on tables.
-
incremental data write: Writes new and changed data from the source to the target system after the initial full data migration.
-
post-task: After the data migration or synchronization is complete, the post-task performs final activities, such as data cleanup, index creation, and performance optimization.
-
full data validation: A process to ensure data integrity and accuracy after migration. It validates all migrated data to confirm it was copied correctly to the target system.
-
incremental data validation: Similar to full data validation, incremental data validation focuses on validating only the incrementally migrated data. This ensures that recent changes have been accurately captured and applied.
Example
Sign in to the DTS synchronization task list page to view execution details.
-
Pre-check
The pre-check includes nine items for both the source and target databases, such as checks for connectivity, permissions, source database version, source kernel minor version, object name conflicts, complex topology, and PostgreSQL logical replication parameters. All check items have a status of Succeeded, indicating a 100% pass rate.
NoteThis step runs automatically. No action is required.
-
Pre-task
The task details page displays the task execution flow, which consists of 10 steps: pre-check, pre-task, incremental data capture, schema migration 1, full data migration, schema migration 2, incremental data write, post-task, full data validation, and incremental data validation. On the Basic Information tab, the task status is Completed and the progress is 100%.
NoteThis step runs automatically. No action is required.
-
Incremental data capture
The status of this step is Running, and the current checkpoint shows No lag.
NoteThis step runs automatically. No action is required.
-
Schema migration
The system automatically migrates most objects, such as tables, indexes, and sequences. For example, the
t1,empty, andt2tables have been migrated successfully.NoteIf a task fails during schema migration, you can use the schema correction feature to manually fix the migration SQL statements.
On the Task Details tab, the Sequence subtab shows that four sequences have also been migrated. The Actions column for each object provides View Creation Syntax and View Index and Foreign Keys links.
-
Full data migration
All data from the tables within the selected scope is migrated. You can view the data traffic and transfer speed during the migration on the Data synchronization page.
At the top of the task details page, the progress bar shows the status of each stage. For example, full data migration is complete, and incremental data write is running with a 1.8-second lag. Select the Basic Information tab to view the migration Status, Progress (e.g., 850,172 rows, 100%), Traffic (e.g., 411.27 MB), Creation Time, and Completion Time. You can also perform Restart or Pause operations.
-
Schema migration 2
This step migrates table foreign key constraints. Adding foreign keys after data migration is crucial for ensuring data consistency across tables.
On the Task Details tab, click the ForeignKey subtab to view the migration status of foreign key objects. In this example, no foreign key objects need to be migrated, so the list is empty.
-
Incremental data write
This step performs one-way synchronization of traffic from the source database. During this process, it subscribes to, forwards, and applies all source database traffic to the target database.
At the top of the migration task details page, the progress bar shows the status of each stage. The incremental data write stage is running with a 1.8-second lag. The Basic Information tab displays metrics such as the current checkpoint, lag, rate (RPS: 0.00), and current rate limit (5,000 RPS). You can manage the task using the Restart or Pause operations.
-
Post-task
The pipeline at the top of the data migration task details page shows the status of each step: pre-check, pre-task, schema migration 1, full data migration, schema migration 2, and full data validation are complete; incremental data capture (No lag), incremental data write (1.4-second lag), and incremental data validation are running; the post-task stage has not started. Below, the Basic Information tab shows the creation time and task status, and the actions bar provides Restart and Pause links.
-
Full data validation
Full data validation compares data between the source and target databases. In this example, the 835,266 rows in the
t1table of the source database are compared with those in thet1table of the target database. The data is consistent, so the validation is successful.The Validation Details tab shows that the empty table (0 rows) and t2 table (14,906 rows) also passed validation, with 0 inconsistent rows. A total of 850,172 rows were validated.
-
Incremental data validation
Once all prerequisite tasks are complete, the system automatically enters the incremental synchronization and validation phase, where it compares incremental data changes between the databases.
The progress bar at the top of the task details page shows the status of each stage (including pre-check, schema migration, full data migration, incremental data write, and incremental data validation). Select the Basic Information tab to view fields such as Start Time, Modification Time, Creation Time, Current Checkpoint, Status, and Incremental Validation Time Rule. You can also perform Restart, Pause, Stop, or Delete Validation Task operations.
Cutover and go-live
After the migration is complete, we recommend connecting the target database to a test environment to run comprehensive business tests. The data in the target database should be fully synchronized with the source database, although minor latency is normal during the data replay process. The testing duration varies based on your business needs, but we recommend testing for at least one week to ensure everything is running correctly before the cutover.
Before the cutover, plan a specific cutover window, for example, 10 hours. During this window, stop the services connecting to the source database. Once the target database completes the data replay, update the database connection strings in your applications and restart your business systems. Then, perform essential business testing. If the tests pass, you can go live with the new service.
We recommend retaining the source database for a period following the cutover. You can safely decommission the source database once you confirm that the target database is fully stable.
References
PolarDB for PostgreSQL (Oracle-compatible) 2.0 General Availability Announcement