This topic describes how to call EMR API operations to modify the configurations of a cluster.

Prerequisites

Modify the configurations of a cluster

Call the ModifyClusterServiceConfig operation to modify existing configurations or add parameters.
  • Modify existing configurations.
    Parameter Type Required Example Description
    RegionId String Yes cn-hangzhou The ID of the region.
    ClusterId String Yes C-C0D50B0FA24A**** The ID of the cluster.
    ServiceName String Yes HDFS The name of the service. For more information, see Service details.
    ConfigParams String Yes {"hdfs-site":{"dfs.namenode.resource.check.interval":"6000"}} The specific modifications. Set this parameter to a JSON string.
    Comment String No modify hdfs config The comment about the modifications.
    Sample code:
    #! /usr/bin/env python
    #coding=utf-8
    
    from aliyunsdkcore.client import AcsClient
    from aliyunsdkcore.acs_exception.exceptions import ClientException
    from aliyunsdkcore.acs_exception.exceptions import ServerException
    from aliyunsdkemr.request.v20160408.ModifyClusterServiceConfigRequest import ModifyClusterServiceConfigRequest
    
    client = AcsClient('<accessKeyId>', '<accessSecret>', 'cn-hangzhou')
    
    request = ModifyClusterServiceConfigRequest()
    request.set_accept_format('json')
    
    request.set_ClusterId("C-C0D50B0FA24A****")
    request.set_ServiceName("HDFS")
    request.set_ConfigParams("{\"hdfs-site\":{\"dfs.namenode.resource.check.interval\":\"6000\"}}")
    request.set_Comment("modify hdfs config")
    
    response = client.do_action_with_exception(request)
    # python2:  print(response) 
    print(str(response, encoding='utf-8'))
  • Add parameters.
    Add parameters to a specific instance of a cluster.
    Parameter Type Required Example Description
    RegionId String Yes cn-hangzhou The ID of the region.
    ClusterId String Yes C-2FD9050E78C8**** The ID of the cluster.
    ServiceName String Yes HDFS The name of the service. For more information, see Service details.
    ConfigParams String Yes {} {} indicates that new parameters are added.
    Comment String No add new config The comment about the added parameters.
    CustomConfigParams String No {"hdfs-site":{"newkey":{"Value":"customValue"}}} Configurations added to custom configuration items.
    HostInstanceId String No i-bp146tnrkq4tcso2**** The ID of the instance. In most cases, this parameter is set to the ID of an ECS instance.
    Sample code:
    #! /usr/bin/env python
    #coding=utf-8
    
    from aliyunsdkcore.client import AcsClient
    from aliyunsdkcore.acs_exception.exceptions import ClientException
    from aliyunsdkcore.acs_exception.exceptions import ServerException
    from aliyunsdkemr.request.v20160408.ModifyClusterServiceConfigRequest import ModifyClusterServiceConfigRequest
    
    client = AcsClient('<accessKeyId>', '<accessSecret>', 'cn-hangzhou')
    
    request = ModifyClusterServiceConfigRequest()
    request.set_accept_format('json')
    
    request.set_ClusterId("C-2FD9050E78C8****")
    request.set_ServiceName("HDFS")
    request.set_ConfigParams("{}")
    request.set_Comment("add new config")
    request.set_CustomConfigParams("{\"hdfs-site\":{\"newkey\":{\"Value\":\"customValue\"}}}")
    request.set_HostInstanceId("i-bp146tnrkq4tcso2****")
    
    response = client.do_action_with_exception(request)
    # python2:  print(response) 
    print(str(response, encoding='utf-8'))

Make the configurations of a cluster take effect

  • Service parameters
    After you modify service parameters, you must restart the related service for the modifications to take effect. You can restart the service in the EMR console or call the RunClusterServiceAction operation to restart the service.
    Parameter Type Required Example Description
    RegionId String Yes cn-hangzhou The ID of the region.
    ClusterId String Yes C-C0D50B0FA24A**** The ID of the cluster.
    ServiceName String Yes HDFS The name of the service. For more information, see Service details.
    ServiceActionName String Yes RESTART The name of the action. Valid values:
    • START: Start the service.
    • RESTART: Restart the service.
    • STOP: Stop the service.
    • CONFIGURE: Send configurations to a specific instance.
    Sample code:
    #! /usr/bin/env python
    #coding=utf-8
    
    from aliyunsdkcore.client import AcsClient
    from aliyunsdkcore.acs_exception.exceptions import ClientException
    from aliyunsdkcore.acs_exception.exceptions import ServerException
    from aliyunsdkemr.request.v20160408.RunClusterServiceActionRequest import RunClusterServiceActionRequest
    
    client = AcsClient('<accessKeyId>', '<accessSecret>', 'cn-hangzhou')
    
    request = RunClusterServiceActionRequest()
    request.set_accept_format('json')
    
    request.set_ClusterId("C-C0D50B0FA24A****")
    request.set_ServiceName("HDFS")
    request.set_ServiceActionName("RESTART")
    
    response = client.do_action_with_exception(request)
    # python2:  print(response) 
    print(str(response, encoding='utf-8'))
  • Client parameters
    After you modify client parameters, you must call the RunClusterServiceAction operation to send the configurations to a specific instance of the cluster and then deploy the configurations.
    Parameter Type Required Example Description
    RegionId String Yes cn-hangzhou The ID of the region.
    ClusterId String Yes C-ADB1FDB7AA82**** The ID of the cluster.
    ServiceName String Yes HDFS The name of the service. For more information, see Service details.
    ServiceActionName String Yes CONFIGURE The name of the action. Valid values:
    Sample code:
    #! /usr/bin/env python
    #coding=utf-8
    
    from aliyunsdkcore.client import AcsClient
    from aliyunsdkcore.acs_exception.exceptions import ClientException
    from aliyunsdkcore.acs_exception.exceptions import ServerException
    from aliyunsdkemr.request.v20160408.RunClusterServiceActionRequest import RunClusterServiceActionRequest
    
    client = AcsClient('<accessKeyId>', '<accessSecret>', 'cn-hangzhou')
    
    request = RunClusterServiceActionRequest()
    request.set_accept_format('json')
    
    request.set_ClusterId("C-ADB1FDB7AA82A****")
    request.set_ServiceName("HDFS")
    request.set_ServiceActionName("CONFIGURE")
    
    response = client.do_action_with_exception(request)
    # python2:  print(response) 
    print(str(response, encoding='utf-8'))

Modify API parameters

You can delete the ConfigType parameter.

Related API operations

Service details

  • Services
    SPARK
    KAFKA
    HIVE
    TEZ
    STORM
    ZEPPELIN
    HBASE
    YARN
    HDFS
    PRESTO
    IMPALA
    DRUID
    KNOX
    KAFKA-MANAGER
    GANGLIA
    HUE
    PHOENIX
    FLINK
    RANGER
    APACHEDS
    HAS
    SQOOP
    PIG
    ZOOKEEPER
    OOZIE
    TENSORFLOW
    SUPERSET
    TENSORFLOW-ON-YARN
    LIVY
    JUPYTER
    ANALYTICS-ZOO
    FLUME
    SMARTDATA
    BIGBOOT
    HIVE-CDH
    KUDU
    OPENLDAP
    TENSORFLOW-ON-SPARK
    GRAFANA
    PROMETHEUS
    ALERTMANAGER
    CLICKHOUSE
  • Service components
    
    ANALYTICS-ZOO
            AnalyticsZooPython
            AnalyticsZooScala
    GANGLIA
            GMETAD
            GMOND
    HBASE
            HMaster
            HRegionServer
            ThriftServer
    HDFS
            KMS
            HttpFS
            NameNode
            DataNode
            SecondaryNameNode
    HIVE
            HiveServer
            HiveMetaStore
    IMPALA
            ImpalaRuntime
            ImpalaCatalogd
            ImpalaStatestored
            Impalad
    KUDU
            KuduMaster
            KuduTserver
    PRESTO
            PrestoMaster
            PrestoWorker
    RANGER
            RangerAdmin
            RangerUserSync
            RangerPlugin
    SPARK
            SparkHistory
    YARN
            ResourceManager
            JobHistory
            TimeLineServer
            WebAppProxyServer
            NodeManager