DataWorks:Solution 4: Network connectivity to a data source deployed on ECS

This solution is recommended if your data source meets the following condition:

• The data source is deployed on an Alibaba Cloud ECS instance.

ECS and DataWorks under the same account and in the same region

If the ECS instance hosting the data source and DataWorks are under the same account and in the same region, we recommend that you use a VPC (private network) connection. Deploy the resource group of your DataWorks workspace and the ECS instance in the same VPC to enable network connectivity.

ECS and DataWorks under different accounts, or under the same account in different regions

If the ECS instance hosting the data source and DataWorks are under different accounts or under the same account but in different regions, we recommend that you use a VPC (private network) connection. Use a network connectivity tool (Cloud Enterprise Network or VPC Peering Connection) to connect the VPC of your DataWorks workspace resource group with the VPC where the ECS instance resides.

Step 1: Collect basic information

Step 2: Connect the networks

Step 3: Add a route to the DataWorks resource group

For the same-account cross-region and different-account scenarios, also add a route in the DataWorks resource group pointing to the ECS vSwitch CIDR block.

1. Go to the DataWorks Resource Group List, find the target resource group and click Operation in the Network Settings on ECS.

2. Find the bound VPC under the applicable module and click Operation in the Custom Route on ECS.

3. Click Add Route and set Connection Method to CIDR Block, then set Destination CIDR Block to the ECS vSwitch CIDR block.

Step 4 (optional): Enable remote database access

Some databases require remote access to be enabled in their configuration files so specified users can connect externally by IP address and port. See the database’s official documentation for its configuration.

Example:4. Enable remote MySQL access on ECS.

Step 5: Configure the ECS security group

ECS security groups provide firewall controls. Open the database port to the VPC of the DataWorks resource group so it can access services on ECS.

1. Go to the ECS console and switch to the target ECS region.

2. In the left navigation pane, click Instances & Images > Instance. Find the ECS instance with MySQL and click its name to open the Instance Details page.

3. Switch to the Security Group tab, click the security group name, and open Security Group Details page.

4. In the Access Rules section. Click Add Rule and configure these key parameters; retain defaults for others.

   • Source: Enter the vSwitch CIDR block bound to the DataWorks resource group.

   • Port: Select the database port on ECS. For MySQL, open 3306 ports.

1. Basic information

2. Connect the networks

This example uses VPC peering connection to connect ECS and DataWorks.

1. Log on to Account A and open the VPC Peering Connection console. Switch to China (Hangzhou) at the top, then click Create Peering Connection and configure the parameters.

   Key parameters for this example are listed below; retain defaults for others.

   Parameter	Configuration and example
   Peering connection name	Custom name. This example uses Account_A to Account_B on ECS.
   Requester VPC	Account AThe VPC containing ECS in Account_A_hangzhou_VPC on ECS.
   Recipient account type	Select Cross-account on ECS.
   Recipient account UID	Enter the primary account UID of Account B.
   Recipient region type	Select Cross-region on ECS.
   Recipient region	Account BSelect the region containing the DataWorks workspace and resource group in China (Shanghai) on ECS.
   Recipient VPC	Enter the VPC ID of the DataWorks resource group in Account B (Account_B_shanghai_VPC).

2. Click DetermineFinish configuring peering and open its details page. Its status is Status initially Pending Peer Acceptance on ECS.

3. Log on to Account B and open the VPC Peering Connection console. Switch to China (Shanghai) and find the same peering record. Click Operation in the Receiver on ECS. to accept it. The status Status changes to Active on ECS.

4. Click Recipient VPC Instance > Configure Route Entry. In the Configure Route Entry dialog box, specify a route entry Name and set Destination CIDR Block to the requester ECS VPC CIDR block, which is 192.168.0.0/16.

5. Log on to Account A and open the VPC Peering Connection console. Switch to China (Hangzhou) and find the created peering connection.

6. Click Requester VPC Instance > Configure Route Entry. In the Configure Route Entry dialog box, specify a route entry Name and set Destination CIDR Block to the recipient DataWorks resource group VPC CIDR block, which is 172.16.0.0/12.

3. Add a route to the DataWorks resource group

1. Log on to Account B and go to DataWorks Resource Group List, find the target resource group and click Operation in the Network Settings on ECS.

2. Find the bound VPC under the applicable module and click Operation in the Custom Route on ECS.

3. Click Add Route and set Connection Method to CIDR Block, then set Destination CIDR Block to the ECS vSwitch CIDR block (192.168.6.0/24).

4. Enable remote MySQL access

Connect to the ECS instance running MySQL and enable remote access.

1. Find the my.cnf configuration file (usually at /etc/my.cnf).

   find / -name my.cnf

2. Use the vim /etc/my.cnf command to edit the file (replace the my.cnf path with the path found above).

3. At the end of the file, press i and add this setting under [mysqld]:

   bind-address=0.0.0.0

4. Press Esc, then type :wq! to save and quit.

5. Run systemctl restart mysqld to restart the service.

6. Create a user for DataWorks to connect remotely to MySQL.

   1. Edit the file with mysql -u root -p to log on as an administrator.

   2. Create the user and set a password.

      -- "dataworks_user" is a customizable username.
      -- "%" allows access from any IP address. Specify an IP address for tighter control.
      -- "StrongPassword123!" is a customizable password.
      CREATE USER 'dataworks_user'@'%' IDENTIFIED BY 'StrongPassword123!';

   3. Grant the user database permissions.

      -- Run either command.
      
      -- Grant all privileges to the user (use caution).
      GRANT ALL PRIVILEGES ON *.* TO 'dataworks_user'@'%' WITH GRANT OPTION;
      
      -- Grant privileges on a specified database, such as mydatabase.
      GRANT ALL PRIVILEGES ON mydatabase.* TO 'dataworks_user'@'%' WITH GRANT OPTION;

   4. Run FLUSH PRIVILEGES;Refresh permissions with exit).

   5. and exit MySQL (

      mysql -u dataworks_user -h <ECS_PRIMARY_PRIVATE_IP> -p

5. Configure the ECS security group

1. Log on to Account A and go to ECS console and switch to China (Hangzhou) on ECS.

2. In the left navigation pane, click Instances & Images > Instance. Find the ECS instance with MySQL and click its name to open the Instance Details page.

3. Switch to the Security Group tab, click the security group name, and open Security Group Details page.

4. In the Access Rules section. Click Add Rule and configure these key parameters; retain defaults for others.

   • Source: Enter the DataWorks resource group vSwitch CIDR block (172.16.66.0/24).

   • Port: Select the database port on ECS (3306).

6. Test connectivity

1. Log on to Account B on ECS.

2. Log on to the DataWorks console. In the target region, click Data Integration > Data Integration in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Integration.

3. In the left navigation pane, click Data Sources and open the Data Sources page. Then click Add Connection on ECS.

4. Select MySQL as the data source and configure it.

   • Configuration ModeSelect User-created Data Store with Public IP Addresses on ECS.

   • Host Address ID and enter the ECS private IP (192.168.6.172).

   • Port NumberSet to 3306 on ECS.

   • Database NameSet to an existing database name.

   • Username and PasswordSet to 4. Enable remote MySQL accessUse the dataworks_user user and password created above.

5. In the Connection ConfigurationClick Test Connectivity for the workspace-bound resource group and check whether the result is Connectable on ECS.

   In the data source Connection Configuration section, select a resource group and click Test Connectivity. Confirm the status is Connected on ECS.

   Note

   If the test result shows Failed, use the Network Connectivity Diagnostic Tool to troubleshoot. If connectivity still fails, submit a ticket for assistance.