Source database limitations | Tables to be synchronized must have a primary key or a UNIQUE constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database. If you synchronize data at the table level, need to edit objects such as mapping column names, and the number of tables in a single task exceeds 5,000, split the tables into multiple tasks. You can also configure a task to synchronize the entire database. Otherwise, an error may be reported after you submit the task. A single sync task supports a maximum of 10 databases. If you exceed this limit, you risk stability and performance issues. In this case, split the tables and configure them in multiple tasks. When you configure a task to synchronize specific objects to the same destination database, you cannot select objects that have the same table name but different schema names. DTS uses the `fn_log` function to get logs from the source database. This function has performance bottlenecks. Do not clear the source database logs too early, or the task may fail. Data logs: Data logs must be enabled. The backup mode must be set to Full, and a full physical backup must have been successfully performed. For incremental synchronization tasks, DTS requires the source database to retain data logs for more than 24 hours. For tasks that include both full and incremental synchronization, DTS requires the source database to retain data logs for at least 7 days. After the full synchronization is complete, you can change the log retention period to more than 24 hours. If the retention period is too short, the DTS task may fail because it cannot get the data logs. In extreme cases, this can cause data inconsistency or data loss. Issues caused by setting a log retention period shorter than required by DTS are not covered by the DTS Service-Level Agreement (SLA).
If Change Data Capture (CDC) needs to be enabled for tables in the source database, the following conditions must be met. Otherwise, the precheck fails. The value of the `srvname` field in the `sys.sysservers` view must be the same as the return value of the `SERVERPROPERTY` function. If the source database is a self-managed SQL Server instance, the database owner must be `sa`. If the source database is an RDS for SQL Server instance, the database owner must be `sqlsa`. If the source database is Enterprise Edition, it must be SQL Server 2008 or later. If the source database is Standard Edition, it must be SQL Server 2016 SP1 or later. If the source database is SQL Server 2017 (Standard or Enterprise Edition), upgrade the version.
If the source instance is a read-only instance, DDL operations cannot be synchronized. If the source database is an Azure SQL Database, a single sync instance can synchronize only one database. If the source database is an RDS for SQL Server instance, ensure that the Transparent Data Encryption (TDE) feature is disabled to ensure the stability of the sync instance. For more information, see Disable TDE. If you use the sp_rename command to modify the names of objects, such as stored procedures, in the source database before a schema synchronization task runs, the task may produce unexpected results or fail.
Note We recommend using the ALTER command to rename database objects. In hybrid log parsing mode, you cannot consecutively run multiple operations to add or drop columns in the source database within a 10-minute interval. For example, running the following SQL statements consecutively causes the task to report an error. ALTER TABLE test_table DROP COLUMN Flag;
ALTER TABLE test_table ADD Remark nvarchar(50) not null default('');
During schema synchronization and initial full data synchronization, do not perform DDL operations to change the schemas of databases or tables. Otherwise, the data synchronization task fails. If the source database is a web-based RDS SQL Server, you must set SQL Server Incremental Synchronization Mode to Incremental Synchronization Based on Logs of Source Database (Heap tables are not supported) when you configure the task. We recommend that you keep the READ_COMMITTED_SNAPSHOT transaction processing mode parameter of the source database enabled while a full data sync task is running. This prevents shared locks from affecting data writes. Otherwise, issues such as data inconsistency or instance failures may occur. Such issues are not covered by the DTS Service-Level Agreement (SLA).
|
Other limitations | Requirements for synchronized objects: Objects supported for initial schema synchronization include Schema, Table, View, Function, and Procedure.
Warning This scenario involves data synchronization between heterogeneous databases, so data types cannot be mapped one-to-one. This may cause the task to fail or result in data loss. Carefully evaluate the business impact of data type mappings. For more information, see Data type mappings for initial schema synchronization. The following objects are not supported for initial schema synchronization: assemblies, service broker, full-text indexes, full-text catalogs, distributed schemas, distributed functions, CLR stored procedures, CLR scalar functions, CLR table-valued functions, internal tables, system objects, and aggregate functions. Data of the following data types cannot be synchronized: CURSOR, ROWVERSION, SQL_VARIANT, HIERARCHYID, POLYGON, GEOMETRY, GEOGRAPHY, and user-defined types created with the CREATE TYPE command. Synchronization of tables that contain computed columns is not supported. Synchronization of the following objects is not supported: INDEX, VIEW, PROCEDURE, FUNCTION, TRIGGER, FK, INDEX, FULL_TEXT_INDEX, DATATYPE, DEFAULT, SYNONYM, CATALOG, PLAN_GUIDE, DEFAULT_CONSTRAINT, UK, CK, and SEQUENCE.
If a table to be synchronized has a primary key, the primary key column of the destination table must be the same as that of the source table. If a table to be synchronized does not have a primary key, the primary key column of the destination table must be the same as the distribution key. The unique key (including the primary key column) of the destination table must contain all columns of the distribution key. If you set SQL Server Incremental Synchronization Mode to Incremental Synchronization Based on Logs of Source Database (Heap tables are not supported) in the Configure Objects stage, the tables to be synchronized must have a clustered index that contains the primary key column. Synchronization of heap tables, tables without a primary key, compressed tables, tables with computed columns, or tables with sparse columns is not supported. These restrictions do not apply in the hybrid log parsing mode. In the Configure Objects stage, if you set SQL Server Incremental Synchronization Mode to Log-based Parsing for Non-heap Tables and CDC-based Incremental Synchronization for Heap Tables (Hybrid Log-based Parsing), the following limitations also apply: The incremental synchronization of DTS depends on the CDC component. Ensure that the CDC job in the source database is running correctly. Otherwise, the DTS task will fail. By default, the incremental data stored by the CDC component is retained for 3 days. Adjust the retention period as needed using the exec console.sys.sp_cdc_change_job @job_type = 'cleanup', @retention= <time>; command.
Note <time> specifies the time in minutes.
If the average number of daily incremental change SQL statements for a single table in the source database exceeds 10 million, set <time> to 1440.
The prerequisite module for a DTS incremental synchronization task enables CDC at the database and table levels in the source database. During this process, the source database may be briefly locked due to limitations of the SQL Server database kernel. In a single sync task, do not enable CDC for more than 1,000 tables. Otherwise, the task may experience latency or become unstable.
If you set SQL Server Incremental Synchronization Mode to Polling and querying CDC instances for incremental synchronization in the Configure Objects stage, the following limitations also apply: The source database account used by the DTS instance must have the permission to enable CDC. Enabling database-level CDC requires an account with the sysadmin role permission, and enabling table-level CDC requires a privileged account.
Note The privileged account (server administrator) provided in the Azure SQL Database console meets the requirements. For databases that use the vCore-based purchasing model, all specifications support enabling CDC. For databases that use the DTU-based purchasing model, the specification must be S3 or higher to support enabling CDC. The privileged account for Amazon RDS for SQL Server meets the requirements and supports enabling database-level CDC for stored procedures. CDC cannot be enabled for tables with clustered columnstore indexes. The prerequisite module for a DTS incremental synchronization task enables CDC at the database and table levels in the source database. During this process, the source database may be briefly locked due to limitations of the SQL Server database kernel.
DTS polls the CDC instance of each table in the source database to get incremental data. Therefore, do not synchronize more than 1,000 tables from the source database. Otherwise, the task may experience latency or become unstable. By default, the incremental data stored by the CDC component is retained for 3 days. Adjust the retention period as needed using the exec console.sys.sp_cdc_change_job @job_type = 'cleanup', @retention= <time>; command.
Note <time> specifies the time in minutes.
If the average number of daily incremental change SQL statements for a single table in the source database exceeds 10 million, set <time> to 1440.
Running add or drop column operations consecutively (more than two add or drop DDL operations within one minute) is not supported. Otherwise, the task may fail. Do not modify the CDC instance in the source database. Otherwise, the task may fail or data may be lost.
To ensure accurate latency for incremental data synchronization, DTS performs the following actions: In the "parse source logs for incremental synchronization" mode, DTS creates the dts_cdc_sync_ddl trigger, the dts_sync_progress heartbeat table, and the dts_cdc_ddl_history DDL storage table in the source database. In hybrid incremental synchronization mode, DTS creates the dts_cdc_sync_ddl trigger, the dts_sync_progress heartbeat table, and the dts_cdc_ddl_history DDL storage table, and also enables database-level CDC and CDC for some tables. The data change volume for tables with CDC enabled in the source database should not exceed 1,000 records per second (RPS). Evaluate the performance of the source and destination databases before you synchronize data. Synchronize data during off-peak hours. Otherwise, the initial full data synchronization consumes read and write resources on both databases, which may increase the database load. Initial full synchronization runs concurrent INSERT operations, which causes table fragmentation in the destination database. As a result, the tablespace of the destination instance is larger than that of the source instance after the initial full synchronization is complete. During DTS synchronization, do not write data to the destination database from any source other than DTS. This will cause data inconsistency between the source and destination databases. For example, if you write data from another source and then use DMS to perform an online DDL operation, data may be lost in the destination database. You can select objects for synchronization at the table level and modify column mappings. If you use column mapping and do not synchronize the full table, or if the source and destination table structures are different, data in columns that exist in the source but not the destination will be lost. Reindexing is not supported for a sync instance. This operation can cause the task to fail or even lead to data loss.
Note Changes related to the primary key are not supported for tables with CDC enabled. If the number of tables with CDC enabled in a single sync task is greater than the value set for The maximum number of tables for which CDC is enabled that DTS supports., the precheck will fail. If a single field in a table with CDC enabled needs to store more than 64 KB of data, you must run the exec sp_configure 'max text repl size', -1; command in advance to adjust the configuration of the source database.
Note By default, a CDC job can process a maximum of 64 KB for a single field. To use the feature to modify synchronized objects, you cannot remove a database. If multiple sync instances use the same SQL Server database as the source, their incremental data ingestion modules are independent of each other. If an instance fails, DTS helpdesk will try to recover the instance within 8 hours. During the recovery process, operations such as restarting the instance and adjusting parameters may be performed.
Note When parameters are adjusted, only the parameters of the DTS instance are modified. The parameters of the database are not modified. The parameters that may be modified include but are not limited to those described in Modify instance parameters. SQL Server is a commercial, closed-source database. Its log format has characteristics that can cause unavoidable issues when DTS performs incremental CDC and parsing. Before you use DTS for incremental or migration synchronization from a SQL Server source in a production environment, perform a comprehensive proof of concept (POC). Your POC should cover all business change types, table schema adjustments, and peak-hour stress tests. The SQL Server log format can be unpredictable. To ensure that DTS runs efficiently and stably, make sure your production business logic is consistent with what you tested in the POC.
|