DataWorks:方案四：部署在ECS中的数据源网络连通

Use this solution if your data source meets the following condition:

• The data source is deployed on an Alibaba Cloud ECS instance.

Same account and region

If the ECS instance where the data source is deployed and the DataWorks workspace are in the same account and region, you can use a VPC (private network) connection. You can deploy the resource group of the DataWorks workspace and the ECS instance in the same VPC to enable network communication.

Different accounts or regions

If the ECS instance where the data source is deployed and the DataWorks workspace are in different accounts or in the same account but different regions, you can use a VPC (private network) connection. You can use a network connectivity tool, such as CEN or a VPC peering connection, to connect the VPC of the DataWorks resource group to the VPC of the ECS instance. This enables network communication.

Step 1: Get the required information

Step 2: Establish network connection

Step 3: Add a route to the resource group

For scenarios involving different accounts or different regions, you must also add a route in the DataWorks resource group that points to the CIDR block of the ECS instance's vSwitch.

1. Go to the DataWorks resource group list. Find the target resource group and click Network Settings in the Actions column.

2. Under the relevant feature module, find the associated VPC and click Custom Route in the Actions column.

3. Click Add Route, set the connection method to Specify CIDR Block, and set Destination CIDR Block to the CIDR block of the vSwitch for the ECS instance.

Step 4: (Optional) Enable remote database access

Some databases require you to enable remote access in a configuration file to allow specified users to connect from external IP addresses and ports. The configuration method varies by database. See the official documentation for your database.

Example: 4. Enable remote access for the MySQL database.

Step 5: Configure the security group

Alibaba Cloud ECS uses security groups to provide firewall capabilities. You must add a rule to the security group of the ECS instance to open the database port to the CIDR block of the DataWorks resource group's vSwitch. This allows the resource group to access the service running on the ECS instance.

1. Go to the Elastic Compute Service console. In the top navigation bar, select the region where your ECS instance is located.

2. In the left-side navigation pane, choose > Instances. Find the ECS instance that hosts your MySQL database and click the instance name to go to the Instance Details page.

3. Click the Security Groups tab and click the name of the security group to go to the Security Group Details page.

4. In the Access Rule section, click Quick Add and configure the following key parameters. Keep the default values for other parameters.

   • Authorization Object: Enter the CIDR block of the vSwitch associated with the DataWorks resource group.

   • Port Range: Select the port used by the database on your ECS instance. For example, MySQL requires port 3306.

1. Basic information

2. Establish network connection

This example uses a VPC Peering Connection to establish network connectivity between the ECS instance and DataWorks.

1. Log on to Account A. Go to the VPC Peering Connection console. In the top navigation bar, select the China (Hangzhou) region, and then click Create Peering Connection. Configure the parameters.

   The following table describes the key parameters for this example. Keep the default values for other parameters.

   Parameter	Configuration and example
   Peering connection name	Enter a custom name. This example uses Account_A to Account_B.
   Requester VPC instance	The VPC in Account A that contains the ECS instance. This example uses Account_A_hangzhou_VPC.
   Accepter account type	This example uses Cross-account.
   UID of the receiver	Enter the UID of the Alibaba Cloud account for Account B.
   Accepter region type	This example uses Cross-region.
   Accepter region	The region where the DataWorks workspace and resource group for Account B are located. Select China (Shanghai).
   Accepter VPC	Manually enter the ID of the VPC associated with the DataWorks resource group in Account B (Account_B_shanghai_VPC).

2. Click OK. You are redirected to the basic information page for the peering connection. The Status of the connection is Peering Accepting.

3. Log on to Account B. Go to the VPC Peering Connection console. In the top navigation bar, select the China (Shanghai) region. You will find the matching peering connection request. In the Actions column, click Accept. After you accept the request, the Status of the peering connection changes to Activated.

4. Click Configure Route under the Accepter VPC Instance. In the Configure Route dialog box, enter a custom Name for the route entry. Set the Destination CIDR Block to the VPC CIDR block of the Requester (the ECS instance), which is 192.168.0.0/16 in this example.

   

5. Log on to Account A. Go to the VPC Peering Connection console. In the top navigation bar, select the China (Hangzhou) region and find the peering connection you created.

6. Click Configure Route under the Requester VPC Instance. In the Configure Route dialog box, enter a custom Name for the route entry. Set the Destination CIDR Block to the VPC CIDR block of the Accepter (the DataWorks resource group), which is 172.16.0.0/12 in this example.

   

3. Add a route to the resource group

1. Log on to Account B. Go to the DataWorks resource group list. Find the target resource group and click Network Settings in the Actions column.

2. Under the relevant feature module, find the associated VPC and click Custom Route in the Actions column.

3. Click Add Route. For Connection Method, select CIDR Block. For Destination CIDR Block, enter the CIDR block of the vSwitch where the ECS instance is located (192.168.6.0/24 in this example).

4. Enable remote database access

Connect to the ECS instance that hosts the MySQL database and enable remote access for the database.

1. Find the location of the my.cnf configuration file. If you use the default installation settings, the file is typically located in the /etc/my.cnf directory.

   find / -name my.cnf

2. Run the vim /etc/my.cnf command to edit the configuration file. Replace the path of the my.cnf file with the actual path that you found in the previous step.

3. At the end of the configuration file, press i to enter insert mode and add the following configuration under [mysqld]:

   bind-address=0.0.0.0

4. Press Esc and enter :wq! to save the file and exit.

5. Run the systemctl restart mysqld command to restart the service.

6. Create a user that DataWorks can use to remotely connect to the MySQL database.

   1. Run the mysql -u root -p command to log on to the database as the administrator.

   2. Create a user and set a password.

      -- "dataworks_user" is the username. You can customize it.
      -- "%" indicates access from any IP address. You can also specify an IP address for finer-grained control.
      -- "StrongPassword123!" is the password. You can customize it.
      CREATE USER 'dataworks_user'@'%' IDENTIFIED BY 'StrongPassword123!';

   3. Grant permissions on the database to the user.

      -- Run one of the following commands.
      
      -- Grants all privileges to the user. Use with caution.
      GRANT ALL PRIVILEGES ON *.* TO 'dataworks_user'@'%' WITH GRANT OPTION;
      
      -- Grants privileges on a specific database (for example, mydatabase) to the user.
      GRANT ALL PRIVILEGES ON mydatabase.* TO 'dataworks_user'@'%' WITH GRANT OPTION;

   4. Run the FLUSH PRIVILEGES; command to flush the privileges, and then exit the database (exit).

   5. Verify the remote connection.

      mysql -u dataworks_user -h <Private_IP_address_of_the_ECS_instance> -p

5. Configure the security group

1. Log on to Account A. Go to the Elastic Compute Service console. In the top navigation bar, select the China (Hangzhou) region.

2. In the left-side navigation pane, choose > Instances. Find the ECS instance that hosts your MySQL database and click the instance name to go to the Instance Details page.

3. Click the Security Groups tab and click the name of the security group to go to the Security Group Details page.

4. In the Access Rule section, click Quick Add and configure the following key parameters. Keep the default values for other parameters.

   • Authorization Object: Enter the CIDR block of the DataWorks resource group's vSwitch (172.16.66.0/24 in this example).

   • Port Range: Select the port used by the database on your ECS instance (3306 in this example).

6. Test the connection

1. Log on to Account B.

2. Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose Data Integration > Data Integration. On the page that appears, select the desired workspace from the drop-down list and click Go to Data Integration.

3. In the left-side navigation pane, click Data source to open the Data Sources page, and then click Add Data Source.

4. Select the MySQL data source type and configure the parameters.

   • For Configuration Mode, select Connection String Mode.

   • For Host IP Address, enter the private IP address of the ECS instance (192.168.6.172 in this example).

   • Set Port Number to 3306.

   • For Database Name, enter the name of an existing database.

   • For Username and Password, enter the dataworks_user username and password you created in the 4. Enable remote access for the MySQL database step.

5. In the Connection Configuration section, click Test Network Connectivity for the associated resource group. Verify that the result is passed.

   

   Note

   If the connectivity test Fails, you can use the Network Connectivity Diagnosis Tool to troubleshoot the issue. If you still cannot connect the Resource Group to the data source, submit a ticket for assistance.