Lindorm:Create and manage

Prerequisites

Before you begin, ensure that you have:

• An activated LindormTable instance

• An activated Lindorm compute engine

Choose a resource group type

ETL resource groups support two types. Choose based on your workload characteristics:

CU-hours (CUH): one compute unit (CU) running for one hour.

Create an ETL resource group

1. Log on to the Lindorm console. In the upper-left corner, select the region of your instance. On the Instances page, click the instance ID or click View Instance Details in the Actions column.

2. On the Instance Details page, in the Configurations section, click Resource Groups in the Actions column for Compute Engine.

3. On the Resource Group Details page, click Create Resource Group.

4. Set Resource Group Type to ETL.

5. In the Resource Group Name field, enter a name. The name can contain only lowercase letters and digits, and can be up to 63 characters long. For example: etlcg0.

6. Set Whether resident and configure the parameters for your chosen type:

   Not resident:

   Parameter	Description
   Daily Resource Quota	Maximum resources consumable per day, in CU-hours (CUH). Valid range: 0–100,000. A value of 0 means no limit. When the daily limit is reached, no additional jobs can be submitted that day.
   Maximum number of CPU cores	Maximum CPU cores available in the group. Minimum value: 100. When running jobs exceed this limit, no additional jobs can be submitted.
   Maximum memory size	Maximum memory available in the group, in GB. Minimum value: 400 GB. When running jobs exceed this limit, no additional jobs can be submitted.
   Authorize Users	Users who can submit jobs to this resource group. Enter * to allow all users.

   Resident:

   Parameter	Description
   Node Specifications	Specifications of the worker nodes in the Spark cluster.
   Working Nodes in Job	Number of worker nodes in the Spark cluster.

7. Click OK.

Submit a job to a specific ETL resource group

To route a job to a specific resource group, add the compute-group parameter to the job configuration:

compute-group=${ETLComputeGroupName}

Replace ${ETLComputeGroupName} with the name of your resource group.

Edit a resource group

After creating an ETL resource group, you can update its daily resource quota, CPU limit, memory limit, and authorized users.

1. On the Resource Group Details page, click Edit in the Actions column.

2. On the Update Resource Group page, modify the settings as needed.

3. Click OK.

Access an ETL resource group from Alibaba Cloud DataWorks

DataWorks is a one-stop platform for intelligent big data development and administration. Use it to submit Spark or Spark SQL tasks to the Lindorm compute engine and orchestrate workflows for offline task scheduling. For details, see:

• Lindorm Spark node

• Lindorm Spark SQL node

Delete a resource group

To delete the resource group:

1. On the Resource Group Details page, click in the Actions column.

2. In the confirmation dialog, click Delete.

3. Click OK.