This topic describes how to use Lindorm Distributed Processing System (LDPS) to read and write data by executing SQL statements.

Prerequisites

Step 1: Make preparations

  1. Download the Spark release package.
  2. Decompress the downloaded Spark release package.
  3. Set the environment variable SPARK_HOME to the path to which the package is decompressed. 
    export SPARK_HOME=/path/to/spark/;
  4. Configure parameters in the following configuration file: $SPARK_HOME/conf/beeline.conf.
    • endpoint: Set this parameter to the JDBC endpoint of LDPS. You can view the JDBC endpoint of LDPS by performing the following steps in the Lindorm console: Click the ID of the Lindorm instance. In the left-side navigation bar, click Database Connections, and then click the Compute Engine tab. 352057-en
    • user: Set this parameter to the username that is used to access the wide table.
    • password: Set this parameter to the password that is used to access the wide table.
    • shareResource: Specify whether the Spark resources are shared by multiple interactive sessions. The default value of this parameter is true.
  5. Activate Hive Metastore. For more information, see Activate Hive Metastore.

Step 2: Write data to a table

  1. Run the /bin/beeline command.
  2. In the interactive session, execute the following SQL statements: 
    CREATE TABLE test (id INT, name STRING);
INSERT INTO test VALUES (0, 'Jay'), (1, 'Edison');

Step 3: Read data from a table

  1. Run the /bin/beeline command.
  2. In the interactive session, execute the following SQL statement: 
    SELECT id, name FROM test;