Hive - Migration Hub - Alibaba Cloud Documentation Center

Service account authorization

1. Metadata based on HiveMetaStore

For Kerberos authentication:

A valid Keytab file and a corresponding principal are required.
The principal requires access permissions to the MetaStore service.
The krb5.conf file must be configured correctly.

2. Metadata based on MySQL

Required system table permissions:

-- Create user if not exists
CREATE USER IF NOT EXISTS '<username>'@'<host>' IDENTIFIED BY 'your_password';

-- Grant SELECT permission (principle of least privilege)
GRANT SELECT ON hive_metastore.dbs TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.tbls TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.sds TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.columns_v2 TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.partition_keys TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.serdes TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.serde_params TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.partitions TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.partition_key_vals TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.partition_params TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.table_params TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.funcs TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.func_ru TO '<username>'@'<host>';

-- Flush privileges
FLUSH PRIVILEGES;

3. Metadata based on AWS Glue

Required IAM policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "GlueMetadataReadAccess",
            "Effect": "Allow",
            "Action": [
                "glue:GetDatabases",
                "glue:GetTables",
                "glue:GetTable",
                "glue:GetPartitions",
                "glue:GetUserDefinedFunctions"
            ],
            "Resource": [
                "arn:aws:glue:${Region}:${AccountId}:catalog",
                "arn:aws:glue:${Region}:${AccountId}:database/*",
                "arn:aws:glue:${Region}:${AccountId}:table/*/*",
                "arn:aws:glue:${Region}:${AccountId}:userDefinedFunction/*/*"
            ]
        }
    ]
}

Data source management

1. Metadata based on HiveMetaStore

Option	Description
`--metastore-uris`	The URI of the Hive MetaStore. This parameter is required.
`-v, --version`	The version of Hive.

lhm datasource set hive -n hive1 --meta-type METASTORE \
    --metastore-uris thrift://host:9083 \
    -v 3.1.2

Enable Kerberos authentication:

Option	Description
`--metastore-sasl-enabled`	Specifies whether to enable SASL authentication.
`--kerberos-keytab-file-path`	The path to the Keytab file. This parameter is required if SASL is enabled.
`--kerberos-conf-file-path`	The path to the krb5.conf file. This parameter is required if SASL is enabled.
`--principal`	The Kerberos principal. This parameter is required if SASL is enabled.
`--hive-principal`	The Hive server-side principal. This parameter is optional.

lhm datasource set hive -n hive1 --meta-type METASTORE \
    --metastore-uris thrift://host:9083 \
    --metastore-sasl-enabled true \
    --kerberos-keytab-file-path /path/to/keytab \
    --kerberos-conf-file-path /path/to/krb5.conf \
    --principal user@REALM \
    --hive-principal hive/_HOST@REALM
    -v 3.1.2

2. Metadata based on MySQL

Option	Description
`--jdbc-url`	The JDBC connection URL. This parameter is required.
`--jdbc-username`	The username. This parameter is required.
`--jdbc-password`	The password. This parameter is required.

lhm datasource set hive -n hive1 --meta-type MYSQL \
    --jdbc-url jdbc:mysql://host:3306/hive \
    --jdbc-username user \
    --jdbc-password '$DB_PASSWORD'

3. Metadata based on AWS Glue

Option	Description
`--aws-access-key`	The AWS access key. This parameter is required.
`--aws-secret-key`	The AWS secret key. This parameter is required.
`--aws-region`	The AWS region. This parameter is required.

lhm datasource set hive -n hive1 --meta-type AWS_GLUE \
    --aws-access-key '$AWS_ACCESS_KEY' \
    --aws-secret-key '$AWS_SECRET_KEY' \
    --aws-region us-east-1