Tablestore:Export full data to MaxCompute

Codeless UI (default)

Configure the following items in the codeless UI:

• Data Source: Select the source and destination data sources.

• Runtime Resource: Select a resource group. The system automatically tests the data source connectivity after you make a selection.

• Data Source:

  • Table: Select the source data table from the drop-down list.

  • Primary Key Range (Start): Specify the start of the primary key range from which to read data. The value must be in the JSON array format. inf_min represents an infinitely small value.

    For example, if the primary key consists of an int column named id and a string column named name, the sample configuration is as follows:

    Specific primary key range	Full data
    [ { "type": "int", "value": "000" }, { "type": "string", "value": "aaa" } ]	[ { "type": "inf_min" }, { "type": "inf_min" } ]

  • Primary Key Range (End): Specify the end of the primary key range from which to read data. The value must be in the JSON array format. inf_max represents an infinitely large value.

    For example, if the primary key consists of an int column named id and a string column named name, the sample configuration is as follows:

    Specific primary key range	Full data
    [ { "type": "int", "value": "999" }, { "type": "string", "value": "zzz" } ]	[ { "type": "inf_max" }, { "type": "inf_max" } ]

  • Splitting Configuration: Specify custom split configuration information in the JSON array format. In most cases, we recommend that you do not configure this parameter. If you do not configure this parameter, set it to [].

    If hot spots occur in Tablestore data storage and the automatic splitting policy of Tablestore Reader is ineffective, use a custom splitting rule. The rule specifies the split points within the start and end primary key range. You only need to configure the shard key and do not need to specify all primary keys.

• Destination: Configure the following items. keep the default values for other parameters or modify them as needed.

  • Production Project Name: The name of the MaxCompute project that is associated with the destination data source. This parameter is automatically populated.

  • Tunnel Resource Group: The default value is Common transmission resources. This resource group uses the free quota of MaxCompute. Select a dedicated Tunnel resource group as needed.

  • Table: Select the destination table. Click Generate Target Table Schema to automatically create the destination table.

  • Partition: Specify the partition where the synchronized data is saved. This parameter can be used for daily incremental synchronization.

  • Write Mode: Select whether to clear existing data before writing or to append data to the table.

• Destination Field Mapping: Configure the field mapping from the source table to the destination table. The system automatically maps fields based on the source table. Modify the mapping as needed.

After you complete the configuration, click Save at the top of the page.

Code editor

Click Code Editor at the top of the page to edit the script in the editor.

The following sample script is for a source data table where the primary key consists of an int column named id and a string column named name. The attribute column is an int field named age. In your script, replace the values for the datasource and table parameters with your actual values.

{
    "type": "job",
    "version": "2.0",
    "steps": [
        {
            "stepType": "ots",
            "parameter": {
                "datasource": "source_data",
                "column": [
                    {
                        "name": "id",
                        "type": "INTEGER"
                    },
                    {
                        "name": "name",
                        "type": "STRING"
                    },
                    {
                        "name": "age",
                        "type": "INTEGER"
                    }
                ],
                "range": {
                    "begin": [
                        {
                            "type": "inf_min"
                        },
                        {
                            "type": "inf_min"
                        }
                    ],
                    "end": [
                        {
                            "type": "inf_max"
                        },
                        {
                            "type": "inf_max"
                        }
                    ],
                    "split": []
                },
                "table": "source_table",
                "newVersion": "true"
            },
            "name": "Reader",
            "category": "reader"
        },
        {
            "stepType": "odps",
            "parameter": {
                "partition": "pt=${bizdate}",
                "truncate": true,
                "datasource": "target_data",
                "tunnelQuota": "default",
                "column": [
                    "id",
                    "name",
                    "age"
                ],
                "emptyAsNull": false,
                "guid": null,
                "table": "source_table",
                "consistencyCommit": false
            },
            "name": "Writer",
            "category": "writer"
        }
    ],
    "setting": {
        "errorLimit": {
            "record": "0"
        },
        "speed": {
            "concurrent": 2,
            "throttle": false
        }
    },
    "order": {
        "hops": [
            {
                "from": "Reader",
                "to": "Writer"
            }
        ]
    }
}

After you finish editing the script, click Save at the top of the page.