Data Integration allows you to migrate multiple tables to the cloud at a time in an efficient and cost-effective manner. This topic describes how to migrate multiple tables to the cloud at a time.
Based on your business requirements, you can also configure rules to convert table names, field names, and data types, add fields to the destination tables, specify values for fields in the destination tables, filter data, and add a prefix to the names of the destination tables.
- Go to the Data Integration.
- Log on to the DataWorks console as a developer. In the left-side navigation pane, click Workspaces. On the Workspaces page, find the target workspace and click Data Integration in the Actions column.
- On the Data Integration page, click Sync Tables in the left-side navigation pane. On the Sync Tables page that appears, you can view all the configured sync nodes.
- On the Sync Tables page, you can view the related logs and synchronization rules, but you cannot modify them.
- If you do not commit the sync nodes after submitting synchronization rules, no running time appears for the nodes and the rules do not take effect.
- Click Sync Tables in the upper-right corner.
- On the page that appears, select the source and destination connections.
Select the added source connections and destination MaxCompute connection. You can select multiple source connections of the same type, such as MySQL, Oracle, or SQL Server. For more information about how to add multiple connections at a time, see Add multiple connections at a time.
- Configure the synchronization rules.
Click Add Rule, select rules from the drop-down list as required, and configure the rules. After configuring the rules, click Apply Rules. Then check the DDL statements and synchronization code to confirm the rule effects.Note
- If rules in the drop-down list do not meet your requirements, you can click Switch to Code Editor and configure rules in the code editor.
- After configuring the rules, you must run the rules and commit the sync nodes. Otherwise, the rules are not saved after you refresh or close this page.
Button Parameter Description Add Rule Partition Key Field in Target Table The rule for specifying the format of partition fields in the destination tables based on scheduling parameters. For more information, see Scheduling parameters. Change Table Names The rule for converting table names. You can convert names of the source tables to required names and use them for the destination tables. Change Field Names The rule for converting field names. You can convert field names in the source tables to required names and use them in the destination tables. Convert Field Types The rule for converting data types. You can convert data types in the source tables to required data types and use them in the destination tables. Add Fields to Target Table The rule for adding fields to the destination MaxCompute tables. You can add a field to the destination tables and set the field name as required. Assign Values in Target Table The rule for assigning a value to the added field. Filter Data The rule for filtering data in the source tables of the selected source connections. Target Table Name Prefix The rule for adding a prefix to the names of the destination tables. Switch to Code Editor You can switch to the code editor to configure synchronization rules. Compared with the codeless user interface (UI), you can specify the applicable scope of each rule in the code editor. However, you cannot switch back to the codeless UI after you switch to the code editor. Reset Script You can reset code only after you switch to the code editor. After you click this button, a unified code template appears in the code editor. Apply Rules You can click Apply Rules to view the impacts of the rules on DDL statements and synchronization code. No sync node is created after you click this button. You can only preview DDL statements and synchronization code.
You can select some tables to check whether their DDL statements and synchronization code comply with the rules.
- Select the tables to be synchronized and commit the sync nodes.
You can select multiple tables at a time. Data Integration generates destination MaxCompute tables based on the configured synchronization rules. If the synchronization fails, move the pointer over the synchronization result. Then the failure cause appears.
Button Description DDL You can click this button to view the related DDL table creation statements, but you cannot modify them. Sync Settings You can click this button to view code of a sync node in the code editor. View Table You can click this button to view the details of a destination MaxCompute table. Tasks After committing a sync node, you can click this button to go to thepage and view the node under the corresponding workflow.