Data Integration allows you to migrate multiple tables to the cloud at a time in an efficient and cost-effective manner. This topic describes how to migrate multiple tables to the cloud at a time.
Based on your business requirements, you can also configure rules to convert table names, field names, and data types, add fields to the destination tables, specify values for fields in the destination tables, filter data, and add a prefix to the names of the destination tables.
- Go to the Data Integration. page of
- Log on to the DataWorks console as a developer.
- In the left-side navigation pane, click Workspaces.
- On the Workspaces page, find the workspace that you want to configure and click Data Integration in the Actions column.
- On the Data Integration page, click Sync Tables in the left-side navigation pane. On the Sync Nodes page, you can view all the configured
- On the Sync Nodes page, you can view the related logs and synchronization rules, but you cannot modify them.
- If you do not commit the sync nodes after submitting synchronization rules, no running time is displayed for the nodes and the rules do not take effect.
- Click Sync Tables in the upper-right corner.
- On the page that appears, select the connections to the source and destination data
stores. Select the created connections to the source data stores and destination MaxCompute data store. You can select the connections to multiple source data stores of the same type, such as MySQL, Oracle, or SQL Server. For more information about how to create multiple connections at a time, see Add multiple connections at a time.
- Configure the synchronization rules. Click Add Rule, select rules from the drop-down list as required, and configure the rules. After you configure the rules, click Apply Rules. Then, check the DDL statements and synchronization code to confirm the rule effects.Note
- If rules in the drop-down list do not meet your requirements, you can click Switch to Code Editor and configure rules in the code editor.
- After you configure the rules, you must run the rules and commit the sync nodes. Otherwise, the rules are not saved after you refresh or close this page.
GUI element Rule Description Add Rule Partition Key Field in Target Table The rule for specifying the format of partition fields in the destination tables based on scheduling parameters. For more information, see Scheduling parameters. Change Table Names The rule for converting table names. You can convert names of the source tables to required names and use them for the destination tables. Change Field Names The rule for converting field names. You can convert field names in the source tables to required names and use them in the destination tables. Convert Field Types The rule for converting data types. You can convert data types in the source tables to required data types and use them in the destination tables. Add Fields to Target Table The rule for adding fields to the destination MaxCompute tables. You can add a field to the destination tables and set the field name as required. Assign Values in Target Table The rule for assigning a value to the added field. Filter Data The rule for filtering data in the source tables of the selected source data stores. Target Table Name Prefix The rule for adding a prefix to the names of the destination tables. Switch to Code Editor You can switch to the code editor to configure synchronization rules. Compared with the codeless user interface (UI), you can specify the applicable scope of each rule in the code editor. However, you cannot switch back to the codeless UI after you switch to the code editor. Reset Script You can reset code only after you switch to the code editor. After you click Reset Script, a unified code template is displayed in the code editor. Apply Rules You can click Apply Rules to view the impacts of the rules on DDL statements and synchronization code. No sync node is created after you click Apply Rules. You can only preview DDL statements and synchronization code.
You can select some tables to check whether their DDL statements and synchronization code comply with the rules.
- Select the tables to be synchronized and commit the sync nodes. You can select multiple tables at a time. Data Integration generates destination MaxCompute tables based on the configured synchronization rules. If the synchronization fails, move the pointer over the synchronization result. Then, the failure cause is displayed.
GUI element Description DDL Click DDL to view the related DDL table creation statements, but you cannot modify them. Sync Settings Click Sync Settings to view code of a sync node in the code editor. View Table Click View Table to view the details of a destination MaxCompute table. Tasks After you commit a sync node, click Tasks to go to thepage and view the node under the corresponding workflow.