E-MapReduce:Manage projects

Limitations

Only an Alibaba Cloud account can create projects, add project members, and add cluster resources. Therefore, the Create Project, Users, and Cluster Settings features are visible only to the Alibaba Cloud account owner, not to RAM users.

Create a project

1. Go to the Projects page in Data Platform.

   1. Log in to the E-MapReduce console with your Alibaba Cloud account.

   2. In the top navigation bar, select the region and resource group.
      

   3. Click the Data Development tab.

      An Alibaba Cloud account can view all projects under the account. A RAM user can view only projects for which they have development permissions. To grant these permissions, the Alibaba Cloud account owner must follow the steps in User management.

2. On the Projects page, click Create Project in the upper-right corner.

3. In the Create Project dialog box, enter a Project Name and Project Description, and select an existing resource group from the Select Resource Group list.

   Note

   If you do not specify a resource group, the project is added to the default resource group. For more information about resource groups, see Use resource groups.

4. Click Create.

   You can view and manage the new project on the Projects page.

Basic project information

1. Go to the Projects page in Data Platform.

   1. Log in to the E-MapReduce console with your Alibaba Cloud account.

   2. In the top navigation bar, select the region and resource group.
      

   3. Click the Data Development tab.

2. Go to the project management page of the target project.

   1. On the Projects page, click the ID of the target project.

   2. Click the Project Management tab.

3. View the basic information of the project.

   On the Basic Information page, you can view the project's name, creation time, creator, and description.

General configuration

Security mode controls permissions for jobs that run in Data Platform.

After you enable security mode, you must add the EMR users who submit jobs to the user list on the running cluster. For more information, see Manage users. When you submit a job in a project where security mode is enabled, jobs submitted by an Alibaba Cloud account run as the default hadoop user, and jobs submitted by a RAM user run as an EMR user with the same name as the RAM user.

1. Go to the Projects page in Data Platform.

   1. Log in to the E-MapReduce console with your Alibaba Cloud account.

   2. In the top navigation bar, select the region and resource group.
      

   3. Click the Data Development tab.

2. Go to the project management page of the target project.

   1. On the Projects page, click the ID of the target project.

   2. Click the Project Management tab.

3. Configure the security mode.

   1. In the left-side navigation pane, click General Configuration.

   2. Based on your job requirements, enable or disable security mode.

      Important

      After you enable security mode, Shell and Hive jobs cannot run.

User management

Grant or revoke a RAM user's development permissions on a project.

1. Go to the Projects page in Data Platform.

   1. Log in to the E-MapReduce console with your Alibaba Cloud account.

   2. In the top navigation bar, select the region and resource group.
      

   3. Click the Data Development tab.

2. Go to the project management page of the target project.

   1. On the Projects page, click the ID of the target project.

   2. Click the Project Management tab.

3. In the left-side navigation pane, click Users.

4. On the Users page, add or remove users.

   • Add a user.

     1. In the upper-right corner, click Add User.

     2. In the Add User dialog box, select the RAM user that you want to add, and then click Add.

        After the user is added, you can view their information on the Users page.

        Note

        The added RAM user becomes a project member and can view and develop jobs and workflows in the project.

   • Remove a user.

     On the Users page, find the user that you want to remove and click Delete in the user's row.

Cluster settings

Associate clusters with a project so that jobs in the project can run on them.

1. Go to the Projects page in Data Platform.

   1. Log in to the E-MapReduce console with your Alibaba Cloud account.

   2. In the top navigation bar, select the region and resource group.
      

   3. Click the Data Development tab.

2. Go to the project management page of the target project.

   1. On the Projects page, click the ID of the target project.

   2. Click the Project Management tab.

3. In the left-side navigation pane, click Cluster Settings.

4. On the Cluster Settings page, you can perform the following operations:

   • Add a cluster.

     1. In the upper-right corner, click Add Cluster.

     2. In the Add Cluster dialog box, select a resource group and cluster information.

        From the Cluster drop-down list, choose a subscription or pay-as-you-go cluster. Clusters created from a cluster template are not supported.

     3. Click OK.

        The new cluster appears on the Cluster Settings page.

   • Modify cluster configurations.

     1. Find the target cluster and click Modify Configuration in the cluster's row.

     2. In the Modify Configuration dialog box, configure the queues, user information, and client whitelist for job submissions to this cluster.

        Parameter	Description
        Default job submission user	The default user for submitting jobs from the project to the selected cluster. Default value: hadoop. Only one default user can be set.
        Default job submission queue	The default queue for submitting jobs from the project to the selected cluster. Default value: default.
        Job submission user whitelist	The users allowed to submit jobs. Separate multiple users with a comma (,).
        Job submission queue whitelist	The queues on the selected cluster where jobs from this project can run. Separate multiple queues with a comma (,).
        client whitelist	The clients from which jobs can be submitted. You can use a master node or a gateway node. Self-built gateways on ECS instances cannot be configured here.

     3. Click OK.

   • Disassociate a cluster.

     On the Cluster Settings page, find the target cluster and click Delete in the cluster's row.

Variable definitions

Define project-level custom variables that jobs in the project can reference as global variables.

1. Go to the Projects page in Data Platform.

   1. Log in to the E-MapReduce console with your Alibaba Cloud account.

   2. In the top navigation bar, select the region and resource group.
      

   3. Click the Data Development tab.

2. Go to the project management page of the target project.

   1. On the Projects page, click the ID of the target project.

   2. Click the Project Management tab.

3. In the left-side navigation pane, click Variable Definition.

4. On the Variable Definition page, you can add or remove custom variables.

   • Add a custom variable.

     1. In the upper-right corner, click Add.

     2. In the Add Custom Variable dialog box, enter a Variable Name and a Variable Value. To encrypt the value, select the Set as Password option.

        In a job, reference a variable using the ${VariableName} format. For example, if you add a variable with the name ENV_ABC and the value 12345, and you do not select Set as Password, the content of a Shell job can be as follows.

        echo ${ENV_ABC}

        The output is:

        12345

        This setting is equivalent to running the following script.

        export ENV_ABC=12345

     3. Click OK.

        You can view the new variable on the Variable Definition page.

   • Remove a custom variable.

     Find the target variable and click Delete in the variable's row.