DataWorks:SelectDB

Step 1: Confirm the SelectDB version

Data Integration has version requirements for SelectDB. Verify that your SelectDB version meets the requirements listed in Supported SelectDB versions. Check your version in the SelectDB console.

Step 2: Create an account and grant permissions

Create a dedicated SelectDB account for DataWorks to use when accessing the data source. If you use the default admin user of the SelectDB data warehouse instead, set a password for that user.

Step 3: Configure network connectivity

Choose a connection type based on your network environment.

A private network connection provides higher bandwidth and a more stable connection than an Internet connection.

Single-table offline synchronization

Configure a task using either the codeless UI or the code editor:

• Codeless UI: Configure an offline synchronization task using the codeless UI

• Code editor: Configure an offline synchronization task using the code editor

For the complete parameter reference and script examples, see Appendix: Script demo and parameter reference.

Entire-database offline synchronization

Synchronize an entire database from MySQL or PostgreSQL to SelectDB. For the configuration procedure, see Configure an Entire-Database Offline Synchronization task.

Script examples

All examples use "stepType": "selectdb" to identify the SelectDB Writer plug-in.

Example 1: Write in JSON format (default)

JSON is the default import format. Use this when the source data is already structured as JSON objects.

{
  "stepType": "selectdb",
  "parameter": {
    "datasource": "selectdb_datasource",
    "table": "selectdb_table_name",
    "column": ["id", "table_id", "table_no", "table_name", "table_status"],
    "loadProps": {
      "format": "json",
      "strip_outer_array": true
    }
  },
  "name": "Writer",
  "category": "writer"
}

Example 2: Write in CSV format

Use CSV format when you need custom column and row delimiters.

{
  "stepType": "selectdb",
  "parameter": {
    "datasource": "selectdb_datasource",
    "table": "selectdb_table_name",
    "column": ["id", "table_id", "table_no", "table_name", "table_status"],
    "loadProps": {
      "format": "csv",
      "column_separator": "\\x01",
      "line_delimiter": "\\x02"
    }
  },
  "name": "Writer",
  "category": "writer"
}

If you use CSV format without specifying column_separator and line_delimiter, all imported data is converted to strings, with \t as the column delimiter and \n as the row delimiter.

Writer parameters