Data Management:ApsaraDB Data Ingestion

Precautions

• To manage instances using DMS, create a separate database account when you create the instance. Avoid sharing accounts that are already in use. The required database account permissions are described as follows:

  • To manage all databases on an instance, grant permissions on all databases of the instance.

  • To manage one or more databases on an instance, grant permissions on the specific databases.

  • To add, delete, modify, and query data, and to modify table schemas, grant the account sufficient DML permissions.

  • To perform operations on objects such as views, stored procedures, triggers, and functions, grant the account sufficient permissions.

• To prevent errors, do not add symbols such as spaces before or after the information that you enter.

• Typically, when you register an Alibaba Cloud database in DMS, DMS prompts you to configure a whitelist. If you are not prompted and cannot manage the database in DMS, you must manually add the DMS IP address ranges to the security settings of the database.

Register an instance

This section uses an ApsaraDB RDS for MySQL database as an example.

1. Log on to the DMS console V5.0.

2. In the Database Instances area on the left side of the console, click the Add Instance icon .

   Note

   You can also choose Data Assets > Instance Management from the menu bar and click Add Instance.

3. On the Add Instance page, you can enter the instance information.

   Category	Parameter	Description
   Data source	-	By default, Alibaba Cloud-MySQL is selected. You can select a different database type.
   Basic information	Database type	Select the type of the database instance. Note After you select a database type, DMS automatically refreshes the parameters on the Basic information page.
   	Instance region	Select the region where the database instance resides.
   	Other primary accounts	This parameter appears if you select Cross-Alibaba Cloud account instances. Select the Alibaba Cloud account to which the database instance belongs. Note If the Alibaba Cloud account is not in the list, click Add User to add the Alibaba Cloud account of the instance purchaser. For more information, see Add a user.
   	Entry method	Select the method to add the database instance. You can add an instance by its Instance ID or Connection String Address.
   	Instance ID or Connection String Address	Enter the Instance ID or Connection String Address. Note A connection string is in the format of `Private network address:Port number`. For example, for an ApsaraDB RDS for MySQL instance: rm-XXXXXXX.mysql.rds.aliyuncs.com:3306.
   	Access mode	Select whether to enable security hosting for the instance and the method to log on to the database. When security hosting is enabled, users do not need to directly handle credentials. This allows for fine-grained permission control. Security Hosting - Automatic (Recommended): DMS automatically enables security hosting for the instance and creates a database account and password to log on to DMS. Note This parameter appears only when you add an ApsaraDB RDS instance. You can view the account that DMS automatically creates in the ApsaraDB RDS console. Do not modify or delete the account. This can affect how DMS functions. Accounts that DMS automatically creates in ApsaraDB RDS instances do not have the permissions to create databases or database accounts. This does not apply to ApsaraDB RDS for PostgreSQL instances. Security Hosting - Manual: DMS automatically enables security hosting for the instance. You must manually enter an existing database account and password. Security Hosting - KMS: DMS automatically enables security hosting for the instance. You must manually select an RDS credential created in Key Management Service (KMS) to log on to the database. Note This parameter appears only when you add an ApsaraDB RDS instance. Disable Security Hosting (Not Recommended): If you disable security hosting, you must frequently use a database account and password to log on. This may affect normal use of the features.
   	Value-added feature package	Select a feature package as needed. Security Collaboration or Stable Change (select one): Security Collaboration includes all features of Stable Change and provides DevOps capabilities. It lets you customize development and approval flows. Stable Change provides solutions for more stable database operations, such as lock-free schema evolution and SQL review. Note If you do not select Security Collaboration or Stable Change, the instance is managed in Flexible Management mode by default. Sensitive Data Protection: This feature lets you manage and mask sensitive data. Enable this feature as needed. For more information, see Enable sensitive data protection.
   	Security rules	This parameter appears if you select Security Collaboration for the feature package. You can select the default security rule or a custom security rule to implement fine-grained control over the database.
   	Classification and grading template	The Classification and grading template parameter appears only when you enable Sensitive Data Protection. You can attach a classification and grading template to an instance. This helps detect if fields in the instance's databases and tables match the template's detection rules. If a field matches a rule, it is tagged with a classification and a sensitivity level to protect highly sensitive fields.
   Advanced information	Environment type	Select the type of database environment.
   	Instance name	Clear the Sync instance name checkbox to customize the display name of the instance in DMS. Note When you add an ApsaraDB instance for the first time, DMS syncs the instance name from the ApsaraDB side. The name is not synced afterward. You can maintain the instance name using the Edit Instance feature.
   	Lock-free Schema Change	Select whether to enable the lock-free schema evolution feature. You can choose one of two methods for executing changes: Execute with DMS lock-free change and Prioritize native lock-free, switch to DMS on failure. Note This parameter appears only when the database is MySQL.
   	Enable SSL	Note This parameter appears only when the database is MySQL or Redis. By default, DMS disables SSL connections. To connect to the database using SSL, manually enable SSL connections in DMS and ensure that SSL is enabled for the database. Secure Sockets Layer (SSL) encrypts network connections at the transport layer. This improves the security and integrity of communication data, but it increases the network connection response time.
   	Instance DBA	Select a DBA role for subsequent processes such as permission requests.
   	Query timeout (s)	Set a security policy. When the specified time is reached, a query statement executing in an SQL window is interrupted to protect the database.
   	Export timeout (s)	Set a security policy. When the specified time is reached, an export statement executing in an SQL window is interrupted to protect the database.

4. After entering the required information, click Test Connection in the lower-left corner.

   Note

   If the connection test fails, review the error message and check the instance information that you entered.

5. After the message Connection successful appears, click Submit.

   The Alibaba Cloud database is now registered in DMS. You can view and manage the database in the instance list on the left side of the DMS console.

Related operations

• If you switch the access mode from Security Hosting - Automatic, where credentials are automatically generated, to another mode and then switch back, DMS resets the password of the account that was originally auto-generated.

• To access resources in the DMS of another Alibaba Cloud account or register your resources in the DMS of another account, you must perform cross-account access or resource registration operations.

• After you register a database in DMS, you may need to perform the following operations:

  • Create databases and tables, and query and modify table data. For more information, see Get started with SQL Console.

  • To modify a large amount of data in a table without locking the table, you can use the lock-free data change feature in DMS.

  • Export table data.

• You can use the AddInstance API operation to register an instance.

FAQ

• Q: When I register an Alibaba Cloud database in DMS, I encounter a problem related to the whitelist, as shown in the following figure. What should I do?

  

  A: Click Set Whitelist in the dialog box. The system automatically attempts to add the DMS server endpoint to the cloud database whitelist. If the automatic addition fails, you must add it manually. For more information, see Add DMS IP address ranges.

• Q: Is the Flexible Management mode no longer available in DMS?

  A: Yes, it is. If you do not select Stable Change or Security Collaboration, the instance defaults to the Flexible Management mode.

• Q: After I enable the Transport Layer Security (TLS/SSL) encryption protocol for Redis, I cannot log on to the Redis database using DMS. A message appears that indicates an SSL connection must be used, as shown in the following figure. How do I resolve this issue?

  

  A: You can resolve this issue by editing the instance:

  1. Log on to the DMS console V5.0.

  2. In the navigation pane on the left side of the DMS homepage, find the target Redis instance in the Database Instances > Not Logged On Instances list. Right-click the instance and click Edit Instance.

  3. In the Advanced Information area, enable SSL.

  4. After you enable it, click Test Connection.

  5. After the test is successful, click Save to complete the connection to DMS.

• Q: Why can't I view table information after I register a MaxCompute instance in DMS? What should I do?

  A: Starting from March 1, 2024, MaxCompute no longer automatically installs the project-level Information Schema for new projects. This means that new projects do not have the Information Schema package by default. You can install the Information Schema permission package as a project owner or a RAM user with the Super_Administrator role to gain access to project metadata. For example, you can log on to the MaxCompute client and execute the install package Information_Schema.systables; command.

  Note

  For more information, see Project-level Information Schema.

• For more information about common issues that may occur when you log on to a database, see Log on to a database.