ApsaraDB for OceanBase:Direct load

Overview

OceanBase Database allows you to insert data into a database by using direct load. The direct load feature can get around conventional SQL-layer APIs and directly write data into files in a database, thereby improving the data import efficiency. For more information, see Overview.

Prerequisites

Before you enable direct load, make sure that the following conditions are met:

• The current cluster is in the running state.

• The current tenant is in the running state.

• The version of OceanBase Database Proxy (ODP) is V4.3.0 or later.

  Note

  If the ODP version does not meet the requirement, contact OceanBase Technical Support to assist in upgrading.

• The private address of the primary address is in the running state.

Enable direct load

1. Log in to the ApsaraDB for OceanBase console.

2. In the left-side navigation pane, click Instances.

3. In the instance list, click the name of the target cluster instance to go to the Cluster Instance Workspace page.

4. In the left-side navigation pane, click Database Proxy.

5. In the proxy address list, click Enable to enable direct load for the private address of the primary address.

6. In the confirmation window that appears, click OK.

Import data

1. Download obloader & obdumper of the latest version.

   For more information, see Introduction.

2. View and set environment variables.

   1. View the Java and PATH environment variables.

      [root@iZbxxxx rpc]# env | grep JAVA
      JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.382.b05-2.0.3.al8.x86_64
      [root@iZbxxxx rpc]# env | grep PATH
      PATH=/root/ob-loader-dumper-4.3.0-RELEASE/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin

   2. Set the Java and PATH environment variables.

      [root@iZbxxxx rpc]# export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.382.b05-2.0.3.al8.x86_64
      [root@iZbxxxx rpc]# export PATH=/root/ob-loader-dumper-4.3.0-RELEASE/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:$PATH

3. Import data in direct load mode.

   Note

   The format of content in the imported file must meet the related specifications. Identify the data format in the file based on Is your data ready?

   [root@iZbxxxx rpc]# obloader -h<host> -u<user_name> -p<pass_word> -D <db_name> --table '<table_name>' -f output1 -t <tenant_name> --public-cloud --csv --direct --rpc-port 3307 -P3306

   Parameter	Description
   -h	The host address of the OceanBase database.
   -u	The username, tenant name, and cluster name for connecting to the OceanBase database, in the <user>@<tenant>#<cluster> format, such as -u user@tenant#cluster.
   -p	The password for connecting to the OceanBase database.
   -D	The name of the OceanBase database into which database object definitions and table data are to be imported.
   --table	The table into which data is to be imported, such as --table 'example'.
   -f	The prefix of the name of the file to be imported. For example, in the preceding command, the data to be imported is stored in a file prefixed with output1.
   -t	The name of the tenant of OceanBase Database that you connect. This parameter is used in combination with the --public-cloud parameter.
   --public-cloud	Imports database object definitions or table data from an ApsaraDB for OceanBase cluster.
   --csv	Imports data from a CSV file. You can specify another file format based on the actual situation, such as --sql, --par, or --orc.
   --direct	Imports data in direct load mode. This parameter is used in combination with the --rpc-port and --parallel parameters.
   --parallel	The degree of parallelism (DOP) for loading data in direct load mode. The default value is 1. This parameter is optional and used in combination with --rpc-port and --direct.
   --rpc-port 3307	The RPC port for connecting to the OceanBase database. Though the port for connecting to the database is 3306, which is specified by -P 3306, data is imported by using the RPC service on port 3307 to improve the import performance.
   -P 3306	The host port for connecting to the OceanBase database.

Disable direct load

1. Log in to the ApsaraDB for OceanBase console.

2. In the left-side navigation pane, click Instances.

3. In the instance list, click the name of the target cluster instance to go to the Cluster Instance Workspace page.

4. In the left-side navigation pane, click Database Proxy.

5. In the proxy address list, click Disable to disable direct load for the private address of the primary address.

6. In the confirmation window that appears, click OK.