Migration Hub:Hive

1. Metadata based on HiveMetaStore

For Kerberos authentication:

• A valid Keytab file and a corresponding principal are required.

• The principal requires access permissions to the MetaStore service.

• The krb5.conf file must be configured correctly.

2. Metadata based on MySQL

Required system table permissions:

-- Create user if not exists
CREATE USER IF NOT EXISTS '<username>'@'<host>' IDENTIFIED BY 'your_password';

-- Grant SELECT permission (principle of least privilege)
GRANT SELECT ON hive_metastore.dbs TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.tbls TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.sds TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.columns_v2 TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.partition_keys TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.serdes TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.serde_params TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.partitions TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.partition_key_vals TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.partition_params TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.table_params TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.funcs TO '<username>'@'<host>';
GRANT SELECT ON hive_metastore.func_ru TO '<username>'@'<host>';

-- Flush privileges
FLUSH PRIVILEGES;

3. Metadata based on AWS Glue

Required IAM policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "GlueMetadataReadAccess",
            "Effect": "Allow",
            "Action": [
                "glue:GetDatabases",
                "glue:GetTables",
                "glue:GetTable",
                "glue:GetPartitions",
                "glue:GetUserDefinedFunctions"
            ],
            "Resource": [
                "arn:aws:glue:${Region}:${AccountId}:catalog",
                "arn:aws:glue:${Region}:${AccountId}:database/*",
                "arn:aws:glue:${Region}:${AccountId}:table/*/*",
                "arn:aws:glue:${Region}:${AccountId}:userDefinedFunction/*/*"
            ]
        }
    ]
}

1. Metadata based on HiveMetaStore

lhm datasource set hive -n hive1 --meta-type METASTORE \
    --metastore-uris thrift://host:9083 \
    -v 3.1.2

Enable Kerberos authentication:

lhm datasource set hive -n hive1 --meta-type METASTORE \
    --metastore-uris thrift://host:9083 \
    --metastore-sasl-enabled true \
    --kerberos-keytab-file-path /path/to/keytab \
    --kerberos-conf-file-path /path/to/krb5.conf \
    --principal user@REALM \
    --hive-principal hive/_HOST@REALM
    -v 3.1.2

2. Metadata based on MySQL

lhm datasource set hive -n hive1 --meta-type MYSQL \
    --jdbc-url jdbc:mysql://host:3306/hive \
    --jdbc-username user \
    --jdbc-password '$DB_PASSWORD'

3. Metadata based on AWS Glue

lhm datasource set hive -n hive1 --meta-type AWS_GLUE \
    --aws-access-key '$AWS_ACCESS_KEY' \
    --aws-secret-key '$AWS_SECRET_KEY' \
    --aws-region us-east-1