Type | Description |
Source database limits | Bandwidth requirements: The server that hosts the source database must have sufficient outbound bandwidth. Otherwise, the data migration speed is affected. The tables to be migrated must have primary keys or UNIQUE constraints, and the fields must be unique. Otherwise, duplicate data may appear in the destination database. If you migrate table-level objects and need to edit them, such as by mapping table and column names, a single data migration task supports a maximum of 1,000 tables. If you exceed this limit, an error is reported after you submit the task. In this case, split the tables into multiple migration tasks or configure a task to migrate the entire database. A single data migration task supports a maximum of 10 databases. If you exceed this limit, stability and performance issues may occur. In this case, split the databases into multiple migration tasks. If you configure a task to migrate specific objects instead of an entire database, you cannot migrate tables that have the same name but different schema names to the same destination database. For incremental migration, data logs must meet the following requirements: Logs must be enabled. The backup mode must be set to Full. A full physical backup must have been successfully performed. For an incremental migration task, Data Transmission Service (DTS) requires that the data logs of the source database are retained for more than 24 hours. For a task that includes both full migration and incremental migration, DTS requires that the data logs of the source database are retained for at least 7 days. You can change the log retention period to more than 24 hours after the full migration is complete. Otherwise, the DTS task may fail because DTS cannot obtain the data logs. In extreme cases, data inconsistency or data loss may occur. Issues caused by a log retention period that is shorter than the required period are not covered by the DTS Service-Level Agreement (SLA).
To enable change data capture (CDC) for the tables to be migrated from the source database, the following conditions must be met. Otherwise, the precheck fails. The value of the srvname field in the sys.sysservers view must be the same as the return value of the SERVERPROPERTY function. If the source is a self-managed SQL Server, the database owner must be sa. If the source is an RDS for SQL Server instance, the database owner must be sqlsa. If the source database is Enterprise Edition, the version must be 2008 or later. If the source database is Standard Edition, the version must be 2016 SP1 or later. If the source database is SQL Server 2017 (Standard or Enterprise Edition), upgrade the version.
DTS uses the fn_log function to obtain source database logs. This function has performance bottlenecks. Do not clear the source database logs too early. Otherwise, the DTS task may fail. Source database operation limits: During initial schema synchronization and full data migration, do not perform DDL operations to change the schemas of databases or tables. Otherwise, the data migration task fails. If you perform only full data migration, do not write new data to the source instance. Otherwise, data inconsistency occurs between the source and destination databases. To ensure real-time data consistency, select Initial Schema Synchronization, Full Data Migration, and Incremental Data Migration.
If the source database is a read-only instance, DDL operations cannot be migrated. If the source database is an Azure SQL Database, a DTS instance can migrate only one database. If the source database is an RDS for SQL Server instance and the migration task includes incremental migration, disable transparent data encryption (TDE) to ensure the stability of the DTS instance. For more information, see Disable TDE. If you use the sp_rename command to rename objects such as stored procedures in the source database before the initial schema synchronization task runs, the task may not work as expected or may fail.
Note Use the ALTER command to rename objects in the database. In hybrid log parsing mode, you cannot consecutively perform multiple operations to add or remove columns in the source database within an interval of less than 10 minutes. For example, if you run the following SQL statements consecutively, the task reports an error. ALTER TABLE test_table DROP COLUMN Flag;
ALTER TABLE test_table ADD Remark nvarchar(50) not null default('');
If the source database is a Web Edition RDS for SQL Server instance, you must set SQL Server Incremental Synchronization Mode to Incremental Synchronization Based on Logs of Source Database (Heap tables are not supported) when you configure the task. During full data migration, make sure that the READ_COMMITTED_SNAPSHOT transaction processing mode parameter is enabled for the source database. This prevents shared locks from affecting data writes. Otherwise, exceptions such as data inconsistency and instance failures may occur. Exceptions caused by this issue are not covered by the DTS SLA.
|
Other limits | Only data of basic data types can be migrated. Data of the CURSOR, ROWVERSION, SQL_VARIANT, HIERARCHYID, POLYGON, GEOMETRY, GEOGRAPHY, and user-defined data types created using the CREATE TYPE command cannot be migrated. Objects of the following types cannot be migrated: INDEX, VIEW, PROCEDURE, FUNCTION, TRIGGER, FK, INDEX, FULL_TEXT_INDEX, DATATYPE, DEFAULT, SYNONYM, CATALOG, PLAN_GUIDE, DEFAULT_CONSTRAINT, UK, CK, and SEQUENCE. You can select tables to migrate. You can also modify column mappings. If you use column mapping for a non-full table migration or if the source and destination table schemas are inconsistent, data in the columns that exist in the source table but not in the destination table is lost. Append-optimized (AO) tables are not supported as destination tables. If a table to be migrated contain a primary key, the primary key column in the destination table must be the same as that of the source table. If a table to be migrated does not contain a primary key, the primary key column and the distribution key in the destination table must be the same. A unique key (primary key column contained) in the destination table must contain all columns of its distribution key. If you select Incremental Synchronization Based on Logs of Source Database (Heap tables are not supported) for SQL Server Incremental Synchronization Mode in the Configure Objects stage, the tables to be migrated must have a clustered index that contains primary key columns. The tables to be migrated cannot be heap tables, tables without primary keys, compressed tables, tables with computed columns, or tables with sparse columns. In mixed log parsing mode, these restrictions do not apply. If you set SQL Server Incremental Synchronization Mode to Log-based Parsing for Non-heap Tables and CDC-based Incremental Synchronization for Heap Tables (Hybrid Log-based Parsing) in the Configure Objects step, the following limits also apply: Incremental migration by DTS depends on the CDC component. Make sure that the CDC job in the source database is running. Otherwise, the DTS task fails. By default, the incremental data stored in the CDC component is retained for 3 days. We recommend that you use the exec console.sys.sp_cdc_change_job @job_type = 'cleanup', @retention= <time>; command to adjust the retention period.
Note <time> specifies the time in minutes.
If the number of incremental change SQL statements for a single table in the source database exceeds 10 million per day, we recommend that you set <time> to 1440.
In a single migration task, we recommend that you enable CDC for no more than 1,000 tables. Otherwise, task latency or instability may occur. The prerequisite module of an incremental migration task enables CDC for the source database. During this process, the source database may be briefly locked due to the limits of the SQL Server database kernel.
If you set SQL Server Incremental Synchronization Mode to Polling and querying CDC instances for incremental synchronization in the Configure Objects step, the following limits also apply: The source database account used by the DTS instance must have the permissions to enable CDC. To enable database-level CDC, you need an account with the sysadmin role. To enable table-level CDC, you need a privileged account.
Note The privileged account (server administrator) provided by the Azure SQL Database console meets the requirements. For vCore-based databases, all instance types support CDC. For DTU-based databases, only instance types of S3 and later support CDC. The privileged account of Amazon RDS for SQL Server meets the requirements and can be used to enable database-level CDC for stored procedures. Clustered columnstore index tables do not support CDC. The prerequisite module of an incremental migration task enables CDC for the source database. During this process, the source database may be briefly locked due to the limits of the SQL Server database kernel.
DTS polls the CDC instance of each table in the source database to obtain incremental data. Therefore, we recommend that you migrate no more than 1,000 tables from the source database. Otherwise, task latency or instability may occur. By default, the incremental data stored in the CDC component is retained for 3 days. We recommend that you use the exec console.sys.sp_cdc_change_job @job_type = 'cleanup', @retention= <time>; command to adjust the retention period.
Note <time> specifies the time in minutes.
If the number of incremental change SQL statements for a single table in the source database exceeds 10 million per day, we recommend that you set <time> to 1440.
You cannot consecutively perform operations to add or remove columns. For example, you cannot perform more than two DDL operations to add or remove columns within one minute. Otherwise, the task may fail. You cannot change the CDC instance of the source database. Otherwise, the task may fail or data may be lost.
To ensure the accuracy of incremental data migration latency, DTS creates the dts_cdc_sync_ddl trigger, the dts_sync_progress heartbeat table, and the dts_cdc_ddl_history DDL storage table in the source database in log parsing mode. In hybrid incremental synchronization mode, DTS creates the dts_cdc_sync_ddl trigger, the dts_sync_progress heartbeat table, and the dts_cdc_ddl_history DDL storage table, and enables database-level CDC and CDC for some tables. We recommend that the data change rate of tables with CDC enabled in the source database does not exceed 1,000 records per second (RPS). Before you migrate data, evaluate the performance of the source and destination databases. We recommend that you migrate data during off-peak hours. Otherwise, DTS consumes read and write resources on the source and destination databases during full data migration, which may increase the database load. Full data migration involves concurrent INSERT operations, which cause table fragmentation in the destination database. Therefore, after full data migration is complete, the table storage space in the destination database is larger than that in the source instance. Confirm whether the migration precision that DTS provides for columns of the FLOAT or DOUBLE data type meets your business requirements. DTS reads the values of these columns using ROUND(COLUMN,PRECISION). If you do not specify the precision, DTS migrates FLOAT values with a precision of 38 and DOUBLE values with a precision of 308. DTS attempts to resume a failed migration task within seven days. Therefore, before you switch your business to the destination instance, you must end or release the task, or use the revoke command to revoke the write permissions of the account that DTS uses to access the destination instance. This prevents the source data from overwriting the data in the destination instance after the task is automatically resumed. If a migration task includes incremental data migration, you cannot reindex. Otherwise, the task may fail or data may be lost.
Note You cannot change the primary keys of tables for which CDC is enabled. If the number of tables for which CDC is enabled in a single migration task is greater than the value of The maximum number of tables for which CDC is enabled that DTS supports., the precheck fails. If a task includes incremental migration and the data to be written to a single field of a table with CDC enabled exceeds 64 KB, you must run the exec sp_configure 'max text repl size', -1; command to adjust the configuration of the source database in advance.
Note By default, a CDC job can process a single field with a maximum length of 64 KB. If multiple DTS instances use the same SQL Server database as the source, their incremental data ingestion modules are independent of each other. If an instance fails, DTS helpdesk will try to recover the instance within 8 hours. During the recovery process, operations such as restarting the instance and adjusting parameters may be performed.
Note When parameters are adjusted, only the parameters of the DTS instance are modified. The parameters of the database are not modified. The parameters that may be modified include but are not limited to those described in Modify instance parameters. SQL Server is a commercial closed-source database. Due to known or unknown format-specific limits, issues may occur when DTS performs CDC and parsing on SQL Server logs. Therefore, before you enable incremental synchronization or migration for a SQL Server source in a production environment, we recommend that you perform a comprehensive proof of concept (POC) test. The test must cover all business change types, table schema changes, and business peak-hour stress tests. Due to the unpredictable nature of the SQL Server log format, you must ensure that the business logic in the production environment is consistent with that in the POC test. This is key to ensuring the high efficiency and stability of DTS.
|
Special cases | If the source instance is an RDS for SQL Server instance, DTS creates an rdsdt_dtsacct account in the source instance for data migration. Do not delete this account or change its password while the task is running. Otherwise, the task may fail. For more information, see System accounts. |