To run EMR tasks in DataWorks, you must configure authentication and authorization in both EMR and DataWorks to ensure that your tasks run as expected.
Background information
In DataWorks, you can obtain EMR engine permissions by mapping workspace members to EMR cluster accounts. This mapping handles user authentication and authorization on the cluster, so different Alibaba Cloud accounts, task owners, or RAM users have different data permissions when running EMR tasks, which enables permission isolation. For more information about the required permissions, see EMR cluster-side configurations and DataWorks-side configurations.
Limits
DataWorks supports only two methods for mapping workspace members to EMR cluster accounts: using LDAP accounts or system accounts. When you configure this account mapping for a cluster that is registered as a compute engine instance in DataWorks, the following limits apply:
-
Account mapping in DataWorks is configured at the cluster level, and you can select only one authentication method per cluster.
-
The cluster account credentials (username and password) that you map in DataWorks must match the cluster's actual credentials.
EMR tasks in DataWorks fail if authentication is not enabled on the cluster, or if the account credentials mapped in DataWorks do not match the cluster's actual credentials.
|
Scenario |
Description |
|
Mapping to a system account |
If the cluster account mapped in DataWorks does not match the actual account on the cluster, EMR tasks fail. |
|
Mapping to an LDAP account |
EMR tasks fail in DataWorks in the following scenarios:
|
Supported authentication methods vary by EMR engine. Check your product documentation to confirm whether your target engine supports LDAP authentication.
EMR cluster
-
Enable an authentication service
To use a non-system account for identity authentication on the cluster, first enable the corresponding authentication service. Then, add the accounts used for EMR task development in DataWorks to that service.
-
Enable the authentication service on the cluster.
To use LDAP for identity authentication, first enable it on your cluster. For more information, see Enable LDAP.
-
Plan the accounts that will be used to run tasks and add them to both the LDAP service and the DataWorks workspace.
We recommend that you add all users who need to create, test, commit, and publish EMR tasks in DataStudio to both the LDAP service and the DataWorks workspace. For more information, see Add users to a DataWorks workspace.
-
-
Control data permissions
You can use permission management components on the EMR cluster to isolate data permissions for DataWorks users. For example, you can use Ranger to manage the permissions of cluster users that correspond to Alibaba Cloud accounts.
DataWorks
-
EMR engine permissions
Before you run EMR tasks in DataWorks, you must register an EMR cluster as a compute engine. The account that performs this registration must have the
AliyunEMRFullAccesspolicy attached. For more information about granting theAliyunEMRFullAccesspolicy to a user, see Overview of users, roles, and permissions. -
Permissions to use DataWorks features
To run EMR tasks in DataWorks, you need permissions for features such as DataStudio, Data Map, Data Quality, and Operation Center to develop tasks and perform O&M and quality control. For more information, see Feature permissions.
-
Configure account mapping
After you configure your EMR engine to run in security mode, go to the page in DataWorks to map workspace members to cluster accounts. This grants members the permissions of their mapped cluster accounts, enabling relevant components to enforce permission isolation.
NoteFor more information about EMR engine configuration and account mapping, see Configure DataWorks.