edit-icon download-icon

Environment preparations

Last Updated: Mar 15, 2018

Access Table Store tables with Hive and HadoopMR

You can use Hive and HadoopMR to access data in Table Store directly by using the dependency package released by Table Store and E-MapReduce.

Install JDK-7+

  1. Download and install the relevant installation package of JDK-7+.

  2. Check the installation status as follows.

    1. $ java -version
    2. java version "1.8.0_77"
    3. Java(TM) SE Runtime Environment (build 1.8.0_77-b03)
    4. Java HotSpot(TM) 64-Bit Server VM (build 25.77-b03, mixed mode)

Install Hadoop

  1. Download Hadoop (version 2.6.0 or later).

  2. Unpack the installation package and install Hadoop for your cluster.

  3. Run Hadoop as follows.

    1. $ bin/start-all.sh
    2. # Check the hadoop service
    3. $ jps
    4. 24017 NameNode
    5. 24835 Jps
    6. 24131 DataNode
    7. 24438 ResourceManager
    8. 5114 HMaster
    9. 24287 SecondaryNameNode
    10. 24527 NodeManager
  4. Add the path of Hadoop to /etc/profile and run source /etc/profile to make the configuration take effect.

    1. export HADOOP_HOME=/data/hadoop/hadoop-2.6.0
    2. export PATH=$PATH:$HADOOP_HOME/bin

Install Hive

  1. Download Hive, specifically the bin.tar.gz type.

  2. Unpack the installation package as follows.

    1. $ mkdir /home/admin/hive-2.1.0
    2. $ tar -zxvf apache-hive-2.1.0-bin.tar.gz -C /home/admin/
    3. $ mv /home/admin/apache-hive-2.1.0-bin /home/admin/hive-2.1.0/
  3. Initialize the chema as follows.

    1. # Enter the specified directory
    2. $ cd /home/admin/hive-2.1.0/
    3. $ ./bin/schematool -initSchema -dbType derby
  4. Run Hive as follows.

    1. $ ./bin/hive
    2. # check hive
    3. hive> show databases;
    4. OK
    5. default
    6. Time taken: 0.207 seconds, Fetched: 1 row(s)

Download Java SDK for Table Store

  1. Download the Java SDK dependency package (version 4.1.0 or later).

    Note: The SDK dependency package is updated with Java SDK. Download the dependency package according to the latest Java SDK.

  2. Copy the SDK to the Hive directory as follows.

    1. $ mv tablestore-4.1.0-jar-with-dependencies.jar /home/admin/hive-2.1.0/

Download EMR dependency package

  1. Download the Alibaba Cloud EMR dependency package.

    Note: For more information on EMR, click here.

  2. Rename the emr-sdk_2.10-1.3.0-20161025.065936-1.jar file.

    1. mv emr-sdk_2.10-1.3.0-20161025.065936-1.jar /home/admin/hive-2.1.0/emr-sdk_2.10-1.3.0-SNAPSHOT.jar
Thank you! We've received your feedback.