If a large number of nodes must be run in parallel, exclusive computing resources are required to ensure that the nodes are run as scheduled. In this case, we recommend that you use exclusive resource groups for scheduling. This topic describes the details about exclusive resource groups for scheduling.

Limits

  • Exclusive resource groups for scheduling cannot be used for Machine Learning nodes.
  • Network connectivity between data sources and exclusive resource groups for scheduling

    If you want to change the resource group on which nodes other than MaxCompute nodes are run, you must make sure that the new resource group you want to use can access the compute engine instances of the nodes.

  • Scale-out of exclusive resource groups for scheduling
    • Each time you purchase exclusive resources for scheduling, the resources belong to only one exclusive resource group for scheduling. After an exclusive resource group for scheduling is created, you can scale out the resource group by adding resources to the resource group.
    • A scale-out only increases the number of resources in your resource group. A scale-out does not upgrade the CPU and memory configurations of the resources or increase the number of VPCs with which your resource group is associated.
    • When you perform a scale-out for your resource group, you can add only resources of the same specifications as existing resources in the resource group.
    • You can perform a scale-out for your resource group without the need to stop the nodes that are running on the resource group. After you perform a scale-out for your resource group, the new configuration takes effect about 20 minutes later after you pay for your order.
  • Scale-in of exclusive resource groups for scheduling
    • Each time you purchase exclusive resources for scheduling, the resources belong to only one exclusive resource group for scheduling. After an exclusive resource group for scheduling is created, you can scale in the resource group by removing resources from the resource group.
    • A scale-in decreases the number of resources in your resource group. You must stop nodes that are running on your resource group before you perform a scale-in for the resource group. After you perform a scale-in for your resource group, the new configuration takes effect about 20 minutes later.
  • Deletion of exclusive resource groups for scheduling

    You cannot delete an exclusive resource group for scheduling in DataWorks. An exclusive resource group for scheduling can be suspended or released after the exclusive resource group for scheduling expires. For more information about suspension and release of an exclusive resource group for scheduling, see Other instructions.

Scenarios

  • If you want to deploy your nodes in the production environment, we recommend that you use exclusive resource groups. This is because the resources in exclusive resource groups can be scheduled at any time to ensure the outputs of nodes.
  • If you want to run a large number of nodes and want the nodes to generate data at the earliest opportunity, we recommend that you use exclusive resource groups.
  • If your nodes need to connect to the Internet or virtual private clouds (VPCs), use exclusive resource groups.
  • If an IP address whitelist is configured for the compute engine instance of your node, use exclusive resource groups.
  • If your VPC is connected to a data center and your node need to connect to the data center, associate your exclusive resource group with the VPC.

Performance metrics

Specifications Maximum number of parallel instances
4c8g 16
8c16g 32
12c24g 48
16c32g 64
24c48g 96

Billing method

Exclusive resource groups for scheduling are charged based on the subscription billing method. You can purchase exclusive resources with appropriate specifications for your exclusive resource group for scheduling based on your business requirements. For more information, see Performance metrics and pricing of exclusive resource groups for scheduling.

Network connectivity

A DataWorks resource group is a group of Alibaba Cloud ECS instances. To run nodes, such as Data Integration nodes or data analytics nodes, make sure that resource groups and data sources are connected. In addition, make sure that special security settings such as whitelists do not affect the connections between resource groups and data sources.

  • Network connectivity
    • Data source deployed on the Internet: If your data source is deployed on the Internet, your exclusive resource group can directly access the data source.
    • Data source deployed in a VPC:
      • If your data source resides in the same region as your exclusive resource group, we recommend that you associate the exclusive resource group with the VPC where the data source resides and with a vSwitch that resides in the VPC. Then, the system adds a route for the exclusive resource group. This way, the exclusive resource group can access the data source.
      • If your data source resides in a different region from your exclusive resource group, you can use an Express Connect circuit or VPN gateway to connect the VPC where the exclusive resource group resides to the VPC where the data source resides.
    • Data source deployed in a data center: If your data source is deployed in a data center, you can use an Express Connect circuit or VPN gateway to connect the VPC where the exclusive resource group resides to the data center where the data source resides.
    • Data source deployed on the classic network:
  • Whitelist configuration: If an IP address whitelist is configured for your data source, you must add the elastic IP address (EIP) of your exclusive resource group for Data Integration or the CIDR block of the vSwitch to which the resource group is bound to the whitelist. For more information, see Add the EIP or CIDR block of an exclusive resource group for Data Integration to an IP address whitelist of a data source.

Other instructions

  • Scale-out
    You can scale out your exclusive resource group for scheduling based on your business requirements. You need to only pay for the added resources that are used from the time you scale out the resource group to the time the original order expires.
    Note
    • Each time you purchase exclusive resources for scheduling, the resources belong to only one exclusive resource group for scheduling. After an exclusive resource group for scheduling is created, you can scale out the resource group by adding resources to the resource group.
    • A scale-out only increases the number of resources in your resource group. A scale-out does not upgrade the CPU and memory configurations of the resources or increase the number of VPCs with which your resource group is associated.
    • When you perform a scale-out for your resource group, you can add only resources of the same specifications as existing resources in the resource group.
    • You can perform a scale-out for your resource group without the need to stop the nodes that are running on the resource group. After you perform a scale-out for your resource group, the new configuration takes effect about 20 minutes later after you pay for your order.
  • Scale-in
    You can scale in your exclusive resource group for scheduling based on your business requirements.
    Note
    • Each time you purchase exclusive resources for scheduling, the resources belong to only one exclusive resource group for scheduling. After an exclusive resource group for scheduling is created, you can scale in the resource group by removing resources from the resource group.
    • A scale-in decreases the number of resources in your resource group. You must stop nodes that are running on your resource group before you perform a scale-in for the resource group. After you perform a scale-in for your resource group, the new configuration takes effect about 20 minutes later.
  • Specification change instructions

    If the specifications of the current exclusive resource group for scheduling do not meet your business requirements, you can change the specifications of the exclusive resource group for scheduling. After you change the specifications of the current resource group, the specifications of all ECS instances in the resource group are changed at the same time.

  • Renewal

    You can renew your exclusive resource group for scheduling before the resource group expires or within 30 days after the resource group expires. If you do not renew the resource group within 15 days after the resource group expires, the system immediately suspends the resource group.

  • Expiration
    • Suspension

      By default, the system sends expiration notifications to the mobile phone number and email address that are bound to your Alibaba Cloud account 14 days, 12 days, and 8 days before your exclusive resource group for scheduling expires. If you do not renew the resource group within 15 days after the resource group expires, the system immediately suspends the resource group.

    • Release

      After exclusive resources for scheduling expire, they are retained for 15 days. If you do not renew the resource group within 15 days after the resource group is suspended, the system releases the resource group.

      The system sends release notifications to the mobile phone number and email address that are bound to your Alibaba Cloud account one day before the system releases your exclusive resource group for scheduling.