ds-controller is a command tool that you can use to create, release, and scale an E-MapReduce (EMR) cluster. You can also use ds-controller to manage resources related to an EMR cluster, such as the virtual private cloud (VPC), vSwitch, and security group to which the EMR cluster belongs. ds-controller runs in command-line interface (CLI) mode or service mode.

Prerequisites

An EMR Data Science cluster is created. For more information, see Create a cluster.

Background information

Notice To release the Elastic Compute Service (ECS) instances of an EMR cluster, you must strictly run the specified commands. Otherwise, the stability of the cluster is affected. Exercise caution.
You can run ds-controller in one of the following modes:
  • CLI mode: After you install ds-controller, you can run commands in the Data Science cluster to manage resources.

    For more information about use examples, see Run ds-controller in CLI mode.

  • Service mode: After you install ds-controller, you can deploy ds-controller on an ECS instance. Then, you can use ds-controller to manage resources by calling RESTful APIs.

    For more information about use examples, see Run ds-controller in service mode.

Install ds-controller

  1. Obtain the ds-controller installation package.
  2. Decompress the ds-controller installation package.
    gunzip ds-controller.gz
  3. Grant the execute permission on the ds-controller file.
    chmod +x ds-controller
  4. Copy the ds-controller file to the /bin directory.
    cp ds-controller /bin/

Run ds-controller in CLI mode

  1. Log on to the EMR Data Science cluster in SSH mode.
    For more information, see Log on to a cluster.
  2. Install ds-controller. For more information, see Install ds-controller.
  3. Create a configuration file.
    You can create a directory named .ds and create a file named config.yaml in the directory. Then, you can configure the following information in the config.yaml file:
    access_id: <yourAccessKeyId>
    access_key: <yourAccessKeySecret>
    region_id: <yourEndpoint>
    rtccache: "./_cache/"
    Note After you configure the preceding information in the config.yaml file, you do not need to specify the region parameter in each of the ds-controller commands. If the region, AccessKey ID, and AccessKey secret parameters are configured in the configuration file and the ds-controller commands, the configurations in the ds-controller commands take effect.
    • <yourAccessKeyId>: the AccessKey ID of your Alibaba Cloud account.
    • <yourAccessKeySecret>: the AccessKey secret of your Alibaba Cloud account.
    • <yourEndpoint>: the ID of the region where your Data Science cluster resides. cn-huhehaote is used in this topic.
  4. Run ds-controller in CLI mode. You can run ds-controller in this mode to perform the following operations:
    • Manage an EMR cluster
      • Create an EMR Data Science cluster.
        ds-controller emr createclusterv2 --clustername <dataScience-test>

        <dataScience-test> indicates the name of the Data Science cluster that you create.

      • Query the basic information about a cluster.
        ds-controller emr describeclusterbasicinfo --clusterid C-CB14657FAEE2**** | jq
        In the examples provided in this topic, C-CB14657FAEE2**** indicates the ID of a Data Science cluster.
        Note You can include jq in the preceding command to view formatted output data.
      • Query the information about a cluster.
        ds-controller emr describeclusterv2 --clusterid C-CB14657FAEE2****
      • Query information about the ECS instance groups in a cluster, including the master, core, and task groups.
        ds-controller emr listclusterhostgroup --clusterid C-CB14657FAEE2****
      • Query the clusters that are in the IDLE state.
        ds-controller emr listclusters --status IDLE

        In the examples provided in this topic, IDLE indicates that a cluster is idle.

      • Query the EMR version.
        ds-controller emr listemrmainversion --pagenumber 1 --pagesize 10
        • In the examples provided in this topic, pagenumber indicates the number of the page to return when you query data by page.
        • In the examples provided in this topic, pagesize indicates the number of entries to return per page when you query data by page.
      • Modify the name of a cluster.
        ds-controller emr modifyclustername --clusterid C-CB14657FAEE2**** --name <dataScience-test-new>

        <dataScience-test-new> indicates the new name of the Data Science cluster.

      • Release a cluster.
        ds-controller emr releasecluster --clusterid C-CB14657FAEE2****
      • Create an ECS instance group, such as a task group.
        ds-controller emr createclusterhostgroup --clusterid C-CB14657FAEE2**** --hostgroupname taskgroup --hostgrouptype TASK --regionid cn-huhehaote
        Notice We recommend that you create only one task group.
      • Scale out a cluster. For example, you can add task nodes to the cluster.
        ds-controller emr resizeclusterv2 --clusterid C-CB14657FAEE2**** --hostgrouptype TASK
      • Release nodes in a cluster by ECS instance group ID. For example, you can release nodes in the cluster by the ID of a task group.
        ds-controller emr releaseclusterhostgroup --clusterid C-CB14657FAEE2**** --hostgroupid G-08DBEB063DD4****

        In the examples provided in this topic, G-08DBEB063DD4**** indicates the ID of an ECS instance group.

      • Release nodes in a cluster by ECS instance group type. For example, you can release nodes in the cluster by the task group type.
        ds-controller emr releaseclusterhostgroupbytype --clusterid C-CB14657FAEE2**** --hostgrouptype TASK
      • View the scale-out status of a cluster and check whether the required task nodes are added.
        You can run one of the following commands to view the scale-out status of the cluster:
        • ds-controller emr listoperation --regionid cn-huhehaote --operationstatus IN_PROGRESS --operationitemtype CLUSTER --operationitemid C-7B6A735717FC988C  --operationtype RESIZE_CLUSTER
        • ds-controller emr listoperation --regionid cn-huhehaote --operationstatus IN_PROGRESS --operationitemtype CLUSTER --operationitemid C-7B6A735717FC988C  --operationtype RESIZE_CLUSTER|jq .OperationInfoList.OperationInfo

        If the returned array is [], the scale-out is complete.

    • Manage an ECS instance
      • Apply for a public IP address.
        ds-controller ecs allocatepublicipaddress --instanceid i-hp3h472yhby69kwh****

        In the examples provided in this topic, i-hp3h472yhby69kwh**** indicates the ID of an ECS instance.

      • Add a security group rule. For example, add Port 22.
        ds-controller ecs authorizesecuritygroup --securitygroupid sg-hp3hk9cues1zuyzn**** --portrange "22/22" --ipprotocol TCP

        In the examples provided in this topic, sg-hp3hk9cues1zuyzn**** indicates the ID of a security group.

      • Create an ECS instance.
        ds-controller ecs createinstance
      • Create a security group.
        ds-controller ecs createsecuritygroup --vpcid vpc-hp3wsdsed3enmxdvi****

        In the examples provided in this topic, vpc-hp3wsdsed3enmxdvi**** indicates the ID of the VPC with which you associate a security group when you create the security group.

      • Delete an ECS instance.
        ds-controller ecs deleteinstance --instanceid i-hp3h472yhby69kwh****
      • Delete a security group.
        ds-controller ecs deletesecuritygroup --securitygroupid sg-hp3hk9cues1zuyzn****
      • Query available regions.
        ds-controller ecs describeregions
      • Query the quotas of ECS resources that you can create in the current region. The resources that you can query include the numbers of security groups, elastic network interfaces (ENIs), pay-as-you-go instance vCPUs, preemptible instance vCPUs, and dedicated hosts, and the total capacity quota of pay-as-you-go disks. You can also query the information such as the network type or whether real-name verification is passed by an account.
        ds-controller ecs describeaccountattributes
      • Query your resources in the current region.
        ds-controller ecs describeavailableresource --destinationresource InstanceType
      • View image resources.
        ds-controller ecs describeimages --pagenumber 1 --pagesize 100
      • Query the information about ECS instances.
        • Query the information about a single ECS instance.
          ds-controller ecs describeinstances --instanceids [\"i-hp3h472yhby69kwh****\"]
        • Query the information about multiple ECS instances. Separate IDs of multiple ECS instances with commas (,).
          ds-controller ecs describeinstances --instanceids [\"i-hp3h472yhby69kwh****\",...,\"i-hp2h472yhby69kwh****\"]
      • Query a list of zones that are provided by Alibaba Cloud. This command also returns some information about the resources that are supported in each zone.
        ds-controller ecs describezones
      • Modify the bandwidth configuration of an ECS instance.
        ds-controller ecs modifyinstancenetworkspec --instanceid i-hp3h472yhby69kwh****
      • Restart an ECS instance.
        ds-controller ecs rebootinstance --instanceid i-hp3h472yhby69kwh****
      • Start an ECS instance.
        ds-controller ecs startinstance --instanceid i-hp3h472yhby69kwh****
      • Stop an ECS instance.
        ds-controller ecs stopinstance --instanceid i-hp3h472yhby69kwh****
      • Query the metadata information about an ECS instance, such as the hostname, network type, MAC address, region, zone, and image ID. The metadata information is used to identify the current environment.
        ds-controller ecs metadata
        ds-controller ecs metadata --path network-type
    • Manage a VPC
      • Create a VPC.
        ds-controller vpc createvpc
      • Create a vSwitch.
        ds-controller vpc createvsw --vpcid vpc-hp3wsdsed3enmxdvi**** --zoneid cn-huhehaote-a
      • Delete a VPC.
        ds-controller vpc deletevpc --vpcid vpc-hp3wsdsed3enmxdvi****
      • Delete a vSwitch.
        ds-controller vpc deletevsw --vswitchid vsw-hp3antge3fe8p3e2s****

        In the examples provided in this topic, vsw-hp3antge3fe8p3e2s**** indicates the ID of a vSwitch.

      • Query the information about VPCs.
        ds-controller vpc describevpcs
      • Query the information about a vSwitch.
        ds-controller vpc describevsws --vpcid vpc-hp3wsdsed3enmxdvi****
    • Manage an SMS message
      Use SendSms to send an SMS message.
      ds-controller sms sendsms --templateparam clusterid:C-CB14657FAEE2****,regionid:cn-huhehaote

Run ds-controller in service mode

The common and Docker-based modes are supported:
  • Common mode
    ./ds-controller
  • Docker-based mode
    docker pull registry.cn-huhehaote.aliyuncs.com/pai-recommend2/ds-controller:0.0.1
    # No configuration file is available.
    docker run -it -d -p 8080:8080 registry-vpc.cn-huhehaote.aliyuncs.com/pai-recommend2/ds-controller:0.0.1
    # The local configuration file is used.
    docker run -it -d -p 8080:8080 -v /root/.ds/:/go/app/conf registry-vpc.cn-huhehaote.aliyuncs.com/pai-recommend2/ds-controller:0.0.1
Example of running ds-controller in service mode
  • Manage an EMR cluster
    • Create an EMR Data Science cluster.
      curl -v -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/emr/createclusterv2" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "EmrVer": "EMR-3.31.0",
      "ZoneId": "cn-huhehaote-a",
      "ClusterName": "<dataScience-test>",
      "ClusterType": "DATA_SCIENCE",
      "RegionId": "cn-huhehaote",
      
      "Master_NodeCount": "1",
      "Master_InstanceType": "ecs.g5.4xlarge",
      "Master_DiskType": "CLOUD_EFFICIENCY",
      "Master_DiskCapacity": "80",
      "Master_DiskCount": "1",
      "Master_SysDiskType": "CLOUD_ESSD",
      "Master_SysDiskCapacity": "120",
      
      "Core_NodeCount": "2",
      "Core_InstanceType": "ecs.g5.4xlarge",
      "Core_DiskType": "CLOUD_EFFICIENCY",
      "Core_DiskCapacity": "120",
      "Core_DiskCount": "4",
      "Core_SysDiskType": "CLOUD_ESSD",
      "Core_SysDiskCapacity": "120",
      
      "Task_NodeCount": "1",
      "Task_InstanceType": "ecs.g5.4xlarge",
      "Task_DiskType": "CLOUD_EFFICIENCY",
      "Task_DiskCapacity": "120",
      "Task_DiskCount": "4",
      "Task_SysDiskType": "CLOUD_ESSD",
      "Task_SysDiskCapacity": "120",
      
      "VSwitchId": "vsw-hp3antge3fe8p3e2s****",
      "SecurityGroupId": "sg-hp3hk9cues1zuyzn****",
      "SecurityGroupName": "emr-test",
      "ChargeType": "PostPaid",
      "VpcId":"vpc-hp3wsdsed3enmxdvi****",
      "NetType": "vpc",
      "MasterPwd": "EMRtest1234!",
      "IsOpenPublicIp": true,
      "SshEnable": true,
      "IoOptimized": true,
      "HighAvailabilityEnable": false
      }'
    • Query the basic information about a cluster.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/emr/describeclusterbasicinfo" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "ClusterId": "C-CB14657FAEE2****"
      }'
    • Query the information about a cluster.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/emr/listclusterhostgroup" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "ClusterId": "C-CB14657FAEE2****"
      }'
    • Query the clusters that are in the IDLE state.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/emr/listclusters" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "ClusterType": "DATA_SCIENCE",
      "Status": "IDLE"
      }'
    • Query the EMR version.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/emr/listemrmainversion" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "PageNumber": 1,
      "PageSize": 10
      }'
    • Modify the name of a cluster.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/emr/modifyclustername" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "ClusterId": "C-CB14657FAEE2****",
      "Name": "<dataScience-test-new>"
      }'
    • Release a cluster.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/emr/releasecluster" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "ClusterId": "C-CB14657FAEE2****"
      }'
    • Scale out a cluster. For example, you can add task nodes to the cluster.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/emr/resizeclusterv2" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "ClusterId": "C-CB14657FAEE2****",
      "HostGroupType": "TASK",
      "NodeCount": "1",
      "InstanceType": "ecs.g5.4xlarge",
      "DiskType": "CLOUD_EFFICIENCY",
      "DiskCapacity": "80",
      "DiskCount": "1",
      "SysDiskType": "CLOUD_ESSD",
      "SysDiskCapacity": "120",
      "VSwitchId": "vsw-hp3antge3fe8p3e2s****",
      "ChargeType": "PostPaid"
      }'
    • Release nodes in a cluster by ECS instance group ID. For example, you can release nodes in the cluster by the ID of a task group.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/emr/releaseclusterhostgroup" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "ClusterId": "C-CB14657FAEE2****",
      "HostGroupId": "G-08DBEB063DD4****"
      }'
    • Release nodes in a cluster by ECS instance group type. For example, you can release nodes in the cluster by the task group type.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/emr/releaseclusterhostgroupbytype" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "ClusterId": "C-CB14657FAEE2****",
      "HostGroupType": "TASK"
      }'
  • Manage an ECS instance
    • Apply for a public IP address.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/allocatepublicipaddress" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "InstanceId": "i-hp3h472yhby69kwh****"
      }'
    • Add a security group rule. For example, add Port 22.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/authorizesecuritygroup" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "SecurityGroupId": "sg-hp3hk9cues1zuyzn****",
      "IpProtocol": "TCP",
      "PortRange": "22/22",
      "SourceCidrIp": "0.0.0.0/0"
      }'
    • Create an ECS instance.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/createinstance" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "InstanceType": "ecs.g6.large",
      "ImageId": "ubuntu_18_04_64_20G_alibase_20190624.vhd",
      "SecurityGroupId": "sg-hp3hk9cues1zuyzn****",
      "InstanceName": "<emr_ds_insname>",
      "InternetChargeType": "PayByTraffic",
      "Password": "EMRtest1234!",
      "ZoneId": "cn-huhehaote-a",
      "SystemDiskSize": 80,
      "SystemDiskCategory": "cloud_efficiency",
      "VSwitchId": "vsw-hp3antge3fe8p3e2s****",
      "InstanceChargeType": "PostPaid",
      "InternetMaxBandwidthIn": 1,
      "InternetMaxBandwidthOut": 1
      }'

      In the examples provided in this topic, <emr_ds_insname> indicates the name of an ECS instance.

    • Create a security group.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/createsecuritygroup" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "SecurityGroupName": "<emr_ds_groupname>",
      "SecurityGroupType": "normal",
      "VpcId": "vpc-hp3wsdsed3enmxdvi****"
      }'

      In the examples provided in this topic, <emr_ds_groupname> indicates the name of a security group.

    • Delete an ECS instance.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/deleteinstance" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "InstanceId": "i-hp3h472yhby69kwh****",
      "Force": true
      }'
    • Query your resources in the current region.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/describeavailableresource" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "DestinationResource": "InstanceType"
      }'
    • View image resources.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/describeimages" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "PageNumber": 1,
      "PageSize": 100
      }'
    • Query the information about ECS instances.
      • Query the information about a single ECS instance.
        curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/describeecsinstances" -d '{
        "AKId": "<yourAccessKeyId>",
        "AKSecret": "<yourAccessKeySecret>",
        "RegionId": "cn-huhehaote",
        "InstanceId": "[\"i-hp3h472yhby69kwh****\"]"
        }'
      • Query the information about multiple ECS instances. Separate IDs of multiple ECS instances with commas (,).
        curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/describeecsinstances" -d '{
        "AKId": "<yourAccessKeyId>",
        "AKSecret": "<yourAccessKeySecret>",
        "RegionId": "cn-huhehaote",
        "InstanceId": "[\"i-hp3h472yhby69kwh****\"],...,[\"i-hp2h472yhby69kwh****\"]"
        }'
    • Query available regions.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/describeregions" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote"
      }'
    • Query the quotas of ECS resources that you can create in the current region. The resources that you can query include the numbers of security groups, elastic network interfaces (ENIs), pay-as-you-go instance vCPUs, preemptible instance vCPUs, and dedicated hosts, and the total capacity quota of pay-as-you-go disks. You can also query the information such as the network type or whether real-name verification is passed by an account.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/describeaccountattributes" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote"
      }'
    • Query a list of zones that are provided by Alibaba Cloud. This command also returns some information about the resources that are supported in each zone.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/describeregions" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote"
      }'
    • Modify the bandwidth configuration of an ECS instance.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/modifyinstancenetworkspec" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "InternetMaxBandwidthOut": 6,
      "InternetMaxBandwidthIn": 6,
      "NetworkChargeType": "PayByTraffic",
      "InstanceId": "i-hp3h472yhby69kwh****"
      }'
    • Restart an ECS instance.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/rebootinstance" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "InstanceId": "i-hp3h472yhby69kwh****"
      }'
    • Start an ECS instance.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/startinstance" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "InstanceId": "i-hp3h472yhby69kwh****"
      }'
    • Stop an ECS instance.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/stopinstance" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "InstanceId": "i-hp3h472yhby69kwh****"
      }'
    • Query the metadata information about an ECS instance, such as the hostname, network type, MAC address, region, zone, and image ID. The metadata information is used to identify the current environment.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/ecs/metadata" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "Path": "/mac"
      }'
  • Manage a VPC
    • Create a VPC.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/vpc/createvpc" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "CidrBlock": "192.168.0.0/16",
      "VpcName": "vpc-hp3wsdsed3enmxdvi****",
      "EnableIpv6": false
      }'
    • Create a vSwitch.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/vpc/createvsw" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "ZoneId": "cn-huhehaote-a",
      "CidrBlock": "192.168.0.0/18",
      "VSwitchName": "<emr_ds_vswname>",
      "VpcId": "vpc-hp3wsdsed3enmxdvi****"
      }'

      In the example, <emr_ds_vswname> indicates the name of a vSwitch.

    • Delete a VPC.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/vpc/deletevpc" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "VpcId": "vpc-hp3wsdsed3enmxdvi****"
      }'
    • Delete a vSwitch.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/vpc/deletevsw" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote",
      "VSwitchId": "vsw-hp3antge3fe8p3e2s****"
      }'
    • Query the information about VPCs.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/vpc/describevpcs" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote"
      }'
    • Query the information about a vSwitch.
      curl -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/vpc/describevsws" -d '{
      "AKId": "<yourAccessKeyId>",
      "AKSecret": "<yourAccessKeySecret>",
      "RegionId": "cn-huhehaote"
      }'
  • Manage an SMS message
    Use SendSms to send an SMS message.
    curl -v -H"Content-Type:application/json" "http://127.0.0.1:8080/v1/sms/sendsms" -d '{
    "AKId": "<yourAccessKeyId>",
    "AKSecret": "<yourAccessKeySecret>",
    "PhoneNumbers": "186212xxxxx,186213xxxxx",
    "SignName":"Datascience",
    "TemplateCode":"SMS_22062xxxx",
    "TemplateParam":"clusterid:C-CB14657FAEE2****,regionid:cn-huhehaote"
    }'

Obtain the required commands

  1. Log on to the EMR Data Science cluster in SSH mode.

    For more information, see Log on to a cluster.

  2. Run the following commands to obtain the required commands.
    • View all the supported resource management commands.
      ./ds-controller -h 
    • View all the commands that are used to manage an EMR cluster.
      ./ds-controller emr -h 
    • View all the commands that are used to manage an ECS instance.
      ./ds-controller ecs -h
    • View all the commands that are used to manage a VPC.
      ./ds-controller vpc -h