Tablestore:Export full data to OSS

Codeless UI (default)

Configure the following parameters:

• Data Source: Select the source and destination data sources.

• Runtime Resource: Select a resource group. After you make a selection, the system automatically tests the data source connectivity.

• Data Source:

  • Table: Select the source table from the drop-down list.

  • Primary Key Range (Start): The starting primary key of the range to read. The value is a JSON array. inf_min represents negative infinity.

    When the primary key includes an int column named id and a string column named name, the following configurations are examples:

    Specified primary key range	Full data
    [ { "type": "int", "value": "000" }, { "type": "string", "value": "aaa" } ]	[ { "type": "inf_min" }, { "type": "inf_min" } ]

  • Primary Key Range (End): The end of the primary key range for the data read, specified as a JSON array. inf_max represents positive infinity.

    When the primary key includes an int column named id and a string column named name, the following configurations are examples:

    Specified primary key range	Full data
    [ { "type": "int", "value": "999" }, { "type": "string", "value": "zzz" } ]	[ { "type": "inf_max" }, { "type": "inf_max" } ]

  • Splitting Configuration: The custom shard configuration in JSON array format. Typically, leave this parameter unconfigured by setting it to [].

    If hotspots occur in Tablestore data storage and the automatic sharding policy of Tablestore Reader is ineffective, we recommend that you use custom sharding rules. Custom sharding rules allow you to specify shard keys within the primary key range. You need to configure only the shard keys, not all primary keys.

• Destination: Select the Text Type and configure the corresponding parameters.

  • Text Type: Valid values are csv, text, orc, and parquet.

  • Object Name (Path Included): The full path to the file in the OSS bucket. For example, tablestore/resource_table.csv.

  • Column Delimiter: The default value is ,. If the separator is a non-printable character, enter its Unicode encoding, such as \u001b or \u007c.

  • Object Path: The path of the file in the OSS bucket. This parameter is required only for the parquet file type.

  • File Name: The name of the file in the OSS bucket. This parameter is required only for files in parquet format.

• Destination Field Mapping: Maps fields from the source table to the destination file. Each line represents a field in JSON format.

  • Source Field: The primary key fields and attribute columns of the source table.

    When the primary key includes an int column named id and a string column named name, and the attribute columns include an int field named age, the following configuration is an example:

    {"name":"id","type":"int"}
    {"name":"name","type":"string"}
    {"name":"age","type":"int"}

  • Target Field: The primary key fields and attribute columns of the source table.

    When the primary key includes an int column named id and a string column named name, and the attribute columns include an int field named age, the following configuration is an example:

    {"name":"id","type":"int"}
    {"name":"name","type":"string"}
    {"name":"age","type":"int"}

After the configuration, click Save at the top of the page.

Code editor

Click Code Editor at the top of the page. The code editor opens. Edit the script.

The following example shows a configuration where the destination file type is CSV. The source table has a primary key that includes an int column named id and a string column named name. The attribute column is an int field named age. When you configure the script, replace the datasource, table name table, and destination file name object in the example script with your actual values.

{
    "type": "job",
    "version": "2.0",
    "steps": [
        {
            "stepType": "ots",
            "parameter": {
                "datasource": "source_data",
                "column": [
                    {
                        "name": "id",
                        "type": "int"
                    },
                    {
                        "name": "name",
                        "type": "string"
                    },
                    {
                        "name": "age",
                        "type": "int"
                    }
                ],
                "range": {
                    "begin": [
                        {
                            "type": "inf_min"
                        },
                        {
                            "type": "inf_min"
                        }
                    ],
                    "end": [
                        {
                            "type": "inf_max"
                        },
                        {
                            "type": "inf_max"
                        }
                    ],
                    "split": []
                },
                "table": "source_table",
                "newVersion": "true"
            },
            "name": "Reader",
            "category": "reader"
        },
        {
            "stepType": "oss",
            "parameter": {
                "dateFormat": "yyyy-MM-dd HH:mm:ss",
                "datasource": "target_data",
                "writeSingleObject": false,
                "column": [
                    {
                        "name": "id",
                        "type": "int"
                    },
                    {
                        "name": "name",
                        "type": "string"
                    },
                    {
                        "name": "age",
                        "type": "int"
                    }
                ],
                "writeMode": "truncate",
                "encoding": "UTF-8",
                "fieldDelimiter": ",",
                "fileFormat": "csv",
                "object": "tablestore/source_table.csv"
            },
            "name": "Writer",
            "category": "writer"
        }
    ],
    "setting": {
        "errorLimit": {
            "record": "0"
        },
        "speed": {
            "concurrent": 2,
            "throttle": false
        }
    },
    "order": {
        "hops": [
            {
                "from": "Reader",
                "to": "Writer"
            }
        ]
    }
}

After you edit the script, click Save at the top of the page.