Document Center
edit-icon download-icon

Hive authorization

Last Updated: Apr 12, 2018

Two authorization features are built in Hive:

  • Storage Based Authorization

  • SQL Standards Based Authorization

For more information, see Hive official documentation.

Note

The two authorization features can be configured at the same time without conflict.

Storage Based Authorization (for HiveMetaStore)

Scenario:

If a user in the cluster has a direct access to data in Hive through HDFS/Hive Client, a permission control must be performed on Hive data in HDFS.

Through HDFS permission control, operation permissions related to Hive SQL can be controlled.

For more information, see Hive documentation.

1. Add configuration

In the cluster Configuration Management page, click Hive > Configuration > hive-site.xml > Add Custom Configuration.

  1. <property>
  2. <name>hive.metastore.pre.event.listeners</name>
  3.     <value>org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener</value>
  4. </property>
  5. <property>
  6. <name>hive.security.metastore.authorization.manager</name>
  7.     <value>org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider</value>
  8. </property>
  9. <property>
  10. <name>hive.security.metastore.authenticator.manager</name>
  11.     <value>org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator</value>
  12. </property>
2. Restart HiveMetaStore

Restart HiveMetaStore in the cluster Configuration Management page.

3. HDFS permission control

HDFS related permissions of warehouse in Hive has been set for Kerberos security cluster in the EMR.

For non-Kerberos security cluster, users must set basic HDFS permission through the following steps:

  • EnableHDFS permissions

  • Configure permissions of warehouse in Hive

    1. hadoop fs -chmod 1771 /user/hive/warehouse
    3. It can be set as follows, in which 1 denotes stick bit (i.e. cannot delete files/folders created by others)
    4. hadoop fs -chmod 1777 /user/hive/warehouse

With the basic permission set (mentioned earlier), related users/user groups can normally create/read/write tables through authorizing the folder warehouse.

  1.       sudo su has
  3.       #Grant rwx permission of folder warehouse to user test
  4.       hadoop fs -setfacl -m user:test:rwx /user/hive/warehouse
  6.       #Grant rwx permission of folder warehouse to user hivegrp
  7.       hadoo fs -setfacl -m group:hivegrp:rwx /user/hive/warehouse

With the HDFS authorization, related users/user groups can normally create/read/write tables, and data in Hive tables created by different users in HDFS can only be accessed by the users themselves.

4. Verification

  • User test creates a table testtbl.

    1. hive> create table testtbl(a string);
    2. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: org.apache.hadoop.security.AccessControlException Permission denied: user=test, access=WRITE, inode="/user/hive/warehouse/testtbl":hadoop:hadoop:drwxrwx--t
    3. at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:320)
    4. at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

    An error occurs due to no permissions. Permissions should be granted to the user test.

    1. #Switch from root account to has account
    2. su has
    4. #Add ACL and grant rwx permissions of the directory warehouse to the account test.
    5.  hadoop fs -setfacl -m user:test:rwx /user/hive/warehouse

    The account test recreates the database successfully.

    1. hive> create table testtbl(a string);
    2. OK
    3. Time taken: 1.371 seconds
    5. #View the directory of testtbl in HDFS. From the permissions it can be seen that only the groups test and hadoop can read data from the table created by the user test, while other users have no permissions
    6. hadoop fs -ls /user/hive/warehouse
    7. drwxr-x---   - test hadoop          0 2017-11-25 14:51 /user/hive/warehouse/testtbl
    9. #Insert a record
    10. hive>insert into table testtbl select "hz"
  • User foo accesses to table testtbl.
  1. #drop table
  2. hive> drop table testtbl;
  3. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Permission denied: user=foo, access=READ, inode="/user/hive/warehouse/testtbl":test:hadoop:drwxr-x---
  4.     at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:320)
  5.     at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:219)
  6.     at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
  8. #alter table
  9. hive> alter table testtbl add columns(b string);
  10. FAILED: SemanticException Unable to fetch table testtbl. java.security.AccessControlException: Permission denied: user=foo, access=READ, inode="/user/hive/warehouse/testtbl":test:hadoop:drwxr-x---
  11.     at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:320)
  12.     at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:219)
  13.     at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
  14.     at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1720)
  16. #select
  17. hive> select * from testtbl;
  18. FAILED: SemanticException Unable to fetch table testtbl. java.security.AccessControlException: Permission denied: user=foo, access=READ, inode="/user/hive/warehouse/testtbl":test:hadoop:drwxr-x---
  19.     at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:320)
  20.     at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:219)

It can be seen that the user foo cannot perform any operations on the table created by the user test. HDFS authorization is needed to grant permissions to foo.

  1. su has
  2. #Only read permission is granted, and write permission can also be granted as needed (for example, alter)
  3. #Note: -R: Set files in the folder testtbl to readable
  4. hadoop fs -setfacl -R -m user:foo:r-x /user/hive/warehouse/testtbl
  6. #The table can be selected successfully
  7. hive> select * from testtbl;
  8. OK
  9. hz
  10. Time taken: 2.134 seconds, Fetched: 1 row(s)

Note:

In general, a Hive user group can be created and authorized, then add new users the group.

SQL Standards Based Authorization

Scenario

If a cluster user can’t access through a HDFS/Hive client, and the only way is to run Hive related commands through HiveServer (beeline, jdbc, and so on). SQL Standards Based Authorization can be used.

If you are able to use methods such as Hive shell, as long as no related configuration has been made to the hive-site.xml in the user’s client, Hive can still be normally accessed even if the following settings are implemented.

For more information, see Hive documentation.

1. Add configuration

The configuration is provided to HiveServer.

In the cluster Configuration Management page, click Hive > Configuration > hive-site.xml > Add Custom Configuration.

  1. <property>
  2. <name>hive.security.authorization.enabled</name>
  3.  <value>true</value>
  4. </property>
  5.  <property>
  6. <name>hive.users.in.admin.role</name>
  7.  <value>hive</value>
  8. </property>
  9.  <property>
  10. <name>hive.security.authorization.createtable.owner.grants</name>
  11.  <value>ALL</value>
  12. </property>
2. Restart HiveServer2

Restart HHiveServer2 in the cluster Configuration Management page.

3. Permission operation commands

For detailed command operations, click here.

4. Verification

  • User foo access to user test’s table testtbl through beeline.

    1. 2: jdbc:hive2://emr-header-1.cluster-xxx:10> select * from testtbl;
    2. Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: Principal [name=foo, type=USER] does not have following privileges for operation QUERY [[SELECT] on Object [type=TABLE_OR_VIEW, name=default.testtbl]] (state=42000,code=40000)

  • Grant permissions.

    1. Switch to account test to grant select permission to user foo
    2. hive> grant select on table testtbl to user foo;
    3.  OK
    4.  Time taken: 1.205 seconds

  • User foo can normally select.

    1. 0: jdbc:hive2://emr-header-1.cluster-xxxxx:10> select * from testtbl;
    2. INFO  : OK
    3. +------------+--+
    4. | testtbl.a  |
    5. +------------+--+
    6. | hz         |
    7. +------------+--+
    8. 1 row selected (0.787 seconds)
  • Revoke permission.
  1.   Switch to account test, and revoke the select permission from user foo
  2.   hive> revoke select from user foo;
  3.     OK
  4.     Time taken: 1.094 seconds
  • User foo cannot select data from table testtbl.
  1. 0: jdbc:hive2://emr-header-1.cluster-xxxxx:10> select * from testtbl;
  2. Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: Principal [name=foo, type=USER] does not have following privileges for operation QUERY [[SELECT] on Object [type=TABLE_OR_VIEW, name=default.testtbl]] (state=42000,code=40000)
Thank you! We've received your feedback.
Free Trial Free Trial