All Products
Search
Document Center

Use cluster

Last Updated: May 10, 2018

1. Differences

Item AutoCluster Cluster
Create cluster Automatically created when tasks start. Manually created in advance. You must specify the ImageId, InstanceType, and number of VMs in the cluster.
Use cluster Specify the ImageId, InstanceType, and number of VMs when submitting jobs. Specify a cluster ID when submitting jobs.
Release cluster Automatically released after all tasks are completed. Manually deleted. If a cluster is no longer used, delete it. Otherwise, you are charged for this cluster.

2. How to use a cluster

2.1. Use Python SDK

  1. # AutoCluster is not required if you specify a cluster ID.
  2. job_desc['DAG']['Tasks']['my-task']['ClusterId'] = "cls-xxxxxxx"

2.2. Use Java SDK

The following is a code snippet:

  1. // AutoCluster is not required if you specify a cluster ID.
  2. desc.setClusterId("cls-xxxxxxx");

2.3. Use command line tool

  1. # Specify a cluster ID
  2. bcs sub "python main.py" -c cls-0101010299123

3. How to use AutoCluster

3.1. Use Python SDK

  1. ...
  2. autoCluster = {
  3. 'ImageId': 'img-ubuntu',
  4. 'InstanceType': 'ecs.sn1.medium'
  5. }
  6. ...
  7. job_desc['DAG']['Tasks']['my-task']['AutoCluster'] = autoCluster
  8. ...

3.2. Use Java SDK

The following is a code snippet:

  1. AutoCluster autoCluster = new AutoCluster();
  2. autoCluster.setImageId("img-ubuntu");
  3. autoCluster.setInstanceType("ecs.sn1.medium");
  4. TaskDescription desc = new TaskDescription();
  5. desc.setAutoCluster(autoCluster);

3.3. Use command line tool

  1. # Use AutoCluster
  2. bcs sub "python main.py" -c img=img-ubuntu:type=ecs.sn1.medium

FAQ

Should AutoCluster or Cluster be used?

If you specify a cluster ID when submitting a job, the job is dispatched to the specified cluster to run.

If you do not specify a cluster, you can use the AutoCluster configuration. In this case, specify an image and an instance type. The corresponding cluster is automatically created when tasks start to run and is automatically released after tasks are completed.

When a cluster should be used?

Use a cluster if you have a large number of jobs to run.

For example, if you have 100 jobs, create a cluster with 10 VMs and submit all these jobs to the cluster. The system automatically runs the tasks one by one, and you only need to wait until all the tasks are completed. After all the tasks are completed, manually release (delete) the cluster. This method saves time and cost.

When AutoCluster should be used?

To use AutoCluster, specify the number of instances and instance capacity when submitting jobs. The system automatically creates a cluster when running tasks of the job and releases the cluster after completing all tasks.

You can use AutoCluster if you can accept a long waiting time or have only a few jobs.