E-MapReduce:Manage workspaces

A workspace is the basic unit in EMR Serverless Spark for managing jobs, members, roles, and permissions. Administrators can add members and assign roles such as workspace administrator, data analyst, data developer, or guest for collaborative work.

Prerequisites

• You have registered an Alibaba Cloud account and completed real-name verification.

• You must have an account with the permissions required to create a workspace:

  • If you use your Alibaba Cloud account, see Assign roles to an Alibaba Cloud account for authorization details.

  • If you use a RAM user or a RAM role, ensure that the AliyunEMRServerlessSparkFullAccess, AliyunOSSFullAccess, and AliyunDLFFullAccess access policies are attached to the RAM user or RAM role. Then, on the Access Control page, add the RAM user or RAM role and grant it the administrator role. For more information, see Grant permissions to a RAM user and Manage users and roles.

• Data Lake Formation (DLF) is activated. For more information, see Quick Start. For a list of regions that support DLF, see Regions and endpoints.

• Object Storage Service (OSS) is activated and a bucket is created. For more information, see Activate OSS and Create a bucket.

Create a workspace

1. Go to the EMR Serverless Spark workspace page.

   1. Log on to the E-MapReduce console.

   2. In the left-side navigation pane, choose EMR Serverless > Spark.

2. On the Spark page, click Create Workspace.

3. In the Create Workspace dialog box, configure the following parameters.

   Parameter	Description
   Region	Select the region where your data resides. The region cannot be changed after creation.
   Billing Method	The supported billing methods are Subscription and Pay-as-you-go.
   Workspace Name	The name must be 1 to 64 characters long and can contain only Chinese characters, letters, digits, hyphens (-), and underscores (_). Note If you enter a name that is already in use, the system prompts you to provide a different one.
   Maximum Quota	Maximum compute units (CUs) available for concurrent job execution in the workspace.
   GPU quota	The maximum number of GPU cards that can be concurrently used for job execution in the workspace. Limit GPU instance types: enabled by default. When this option is enabled, only the GPU instance types within the selected range can be used. Note This feature is supported only in the China (Beijing), China (Shanghai), China (Hangzhou), China (Shenzhen), China (Ulanqab), and China (Hong Kong) regions. This parameter is available only for pay-as-you-go workspaces. The supported instance types and unit prices are subject to the console. If you require other instance types, submit a ticket. If you want to use more GPU resources while the workspace is in use, see Edit the pay-as-you-go GPU quota and Manage subscription GPU quotas.
   Workspace Directory	Stores data files such as job logs, runtime events, and resources. For near-real-time incremental log viewing during O&M, select an OSS-HDFS-enabled bucket.
   DLF for Metadata Storage	Stores and manages metadata. Select the data catalog to associate with the workspace. To create a data catalog: Click Create Catalog. In the dialog box that appears, enter a Catalog Name and click OK. From the drop-down list, select the data catalog that you created. Note After creating a workspace, you can add an existing DLF data catalog. Data catalog.
   Execution Role	Specifies the role that Serverless Spark assumes to run jobs. The default role is AliyunEMRSparkJobRunDefaultRole. Serverless Spark uses this role to access your resources in other services, such as OSS and DLF. If you want to control the permissions of the execution role, you can use a custom execution role. For more information, see Execution role.
   Advanced Settings	Tags: Tags are identifiers for your cloud resources. You can use tags to classify, search for, and aggregate resources that share the same characteristics, which improves resource management efficiency. You can bind up to 20 tags to each workspace. Each tag consists of a custom tag key and tag value. Tags also enable cost allocation and fine-grained management of pay-as-you-go resources. You can bind tags when you create a workspace, or add or modify tags at any time on the workspace list page. Binding tags to resources helps you classify them and optimize operations.

4. Click Create Workspace.

Edit the pay-as-you-go GPU quota

If you have created a workspace and want to adjust the pay-as-you-go GPU card limit and available instance types based on your business requirements, you can edit the pay-as-you-go GPU quota.

1. On the Spark page, choose Resource Management > Edit Pay-As-You-Go GPU Quota in the Actions column for the workspace.

2. On the Edit Pay-As-You-Go GPU Quota page, adjust the GPU card limit and supported instance types.

3. Confirm the configuration and click OK.

After you adjust the pay-as-you-go GPU quota, GPU queues are initialized. To adjust GPU queues based on your business requirements, see Manage resource queues.

Manage subscription GPU quotas

If you have created a workspace and want to purchase and manage subscription GPU resources based on your business requirements, perform the following steps.

1. On the Spark page, choose Resource Management > Buy Subscription GPU Quota in the Actions column for the workspace.

2. On the Buy Subscription GPU Quota page, configure the GPU instance type, number of GPU cards, and subscription duration.

3. Confirm the configuration and click OK. Then, complete the payment as prompted.

After you purchase subscription GPU quotas, GPU queues are initialized. To adjust GPU queues based on your business requirements, see Manage resource queues.

If subscription GPU quotas are purchased for a workspace, you can choose Resource Management > Manage Subscription GPU Quota to go to the subscription GPU quota management page, where you can renew or scale up the quotas as needed.

Delete a workspace

Deletion steps vary by billing method:

• Subscription: Unsubscribe from the quota order first. The workspace is deleted after unsubscription. Unsubscription and refunds.

• Pay-as-you-go:

  1. On the Spark page, click Delete in the Actions column for the workspace.

  2. In the dialog box, enter the workspace name for confirmation and click OK.

Adjust the subscription quota

For subscription workspaces, you can upgrade or downgrade the CU quota.

1. On the Spark page, choose > Upgrade Subscription Quota or Downgrade Subscription Quota in the Actions column for the workspace.

2. On the EMR Serverless Spark Reserved Resources | Change Specification page, adjust the CU Quota parameter. The price difference is calculated automatically.

3. Confirm the configuration and click Buy Now.

Modify the pay-as-you-go quota

1. On the Spark page, choose > Modify Pay-as-you-go Quota in the Actions column for the workspace.

2. In the dialog box that appears, adjust the Maximum Pay-as-you-go Quota (CUs) and click OK.

Related documents

• Add RAM users and assign roles for collaborative development using Access Control. Manage users and roles.

• Isolate and manage resources by adding queues. Manage resource queues.