MaxCompute:Migrate data from MaxCompute to Tablestore

Procedure

1. Activate services

   1. Activate MaxCompute

   2. Activate DataWorks

2. Create a table in DataWorks

   1. Log in to the DataWorks console and select a region in the upper-left corner.

   2. In the navigation pane on the left, choose Workspace.

   3. On the Workspaces page, in the Actions column of the target workspace, choose Shortcuts > DataStudio.

   4. On the DataStudio page, create a MaxCompute SQL node. Run the following statement to create a table named transs.

      CREATE TABLE transs
      (name    STRING,
      id    STRING,
      gender    STRING);

3. Import data into the transs table

   1. Download the sample CSV file demo_data.csv to your computer.

   2. In the navigation pane on the left, choose Data Integration > Data Upload and Download.

   3. Click Go to Data Upload and Download.

   4. In the left navigation bar, click the upload icon , and click Data Upload.

      Upload the data file to the MaxCompute table named transs.

4. Create a table in the Tablestore console

   1. Log on to the Tablestore console and create an instance.

   2. Create a data table named trans. For more information, see Create a data table.

      Set the primary key to name and id.

5. Add a MaxCompute data source in DataWorks

   1. Log in to the DataWorks console and select a region in the upper-left corner.

   2. In the navigation pane on the left, choose Workspace.

   3. On the Workspaces page, click the name of the target workspace.

   4. On the Workspace Details page, click Data Sources in the navigation pane on the left.

   5. On the Data Source tab, click Add Connection, and then select MaxCompute.

   6. On the Add MaxCompute Data Source page, fill in the Basic Information.

      For more information, see Configure a MaxCompute data source.

6. Add a Tablestore data source. For more information, see Configure a Tablestore data source.

7. Configure the MaxCompute (ODPS) Reader and Tablestore Writer

   1. Log in to the DataWorks console and select a region in the upper-left corner.

   2. In the navigation pane on the left, choose Data Development and O&M > Data Development.

   3. Select Workspace, and click Go to Data Studio.

   4. On the left side of DataStudio, click , and select Create Node > Data Integration > Batch Synchronization.

      • For Data source, select Tablestore.

      • Select MaxCompute as the data destination.

   5. On the node configuration page, configure the following parameters.

      • Data Source-Source: Select the MaxCompute data source you added.

      • Data Source-Destination: Select the Tablestore data source you added.

      • Data Source-Table: Select the source table in MaxCompute.

      • Destination-Table: Select the destination table in Tablestore.

      • Runtime Resource: Select the exclusive resource group you created.

      • Keep the default values for other parameters.

      Alternatively, click the icon above the configuration area to switch to script mode.

      {
          "type": "job",
          "steps": [
              {
                  "stepType": "odps",
                  "parameter": {
                      "partition": [],
                      "datasource": "odps_first",
                      "column": [
                          "name",
                          "id",
                          "gender"
                      ],
                      "table": "transs"
                  },
                  "name": "Reader",
                  "category": "reader"
              },
              {
                  "stepType": "ots",
                  "parameter": {
                      "datasource": "transs",
                      "column": [
                          {
                              "name": "gender",
                              "type": "STRING"
                          }
                      ],
                      "writeMode": "UpdateRow",
                      "table": "trans",
                      "primaryKey": [
                          {
                              "name": "name",
                              "type": "STRING"
                          },
                          {
                              "name": "id",
                              "type": "STRING"
                          }
                      ]
                  },
                  "name": "Writer",
                  "category": "writer"
              }
          ],
          "version": "2.0",
          "order": {
              "hops": [
                  {
                      "from": "Reader",
                      "to": "Writer"
                  }
              ]
          },
          "setting": {
              "errorLimit": {
                  "record": "0"
              },
              "speed": {
                  "throttle": false,
                  "concurrent": 1,
                  "dmu": 1
              }
          }
      }

8. Verify the migrated data in the Tablestore console.

   1. Log on to the Tablestore console. In the upper-left corner, select a region.

   2. In the left-side navigation pane, choose All Instances.

   3. On the All Instances page, click an instance name to open the Instance Management page.

   4. On the Instance Management page, click the Instance Details tab.

   5. On the Instance Details tab, in the Tables section, click the name of a data table.

   6. On the data table management page, click the Query Data tab to view the table's data.