This topic describes the deployment details for the components on different instance nodes in an EMR cluster.

When you create an EMR cluster, components are deployed on instance nodes based on the Instance types. For example, the NameNode component of Hadoop HDFS is deployed on the master instance node. For large-scale clusters, you can use bootstrap actions to complete custom deployment and allocation. For example, a stand-alone ZooKeeper cluster uses an Alibaba Cloud RDS instance as the Hive metastore. The following tables describe the service components that run on different instance nodes.

Hadoop cluster

Deployment details for service components that run on a Hadoop cluster
  • Required services
    Service name Master instance node Core instance node
    HDFS
    • KMS
    • SecondaryNameNode
    • HttpFS
    • HDFS Client
    • NameNode
    • DataNode
    • HDFS Client
    YARN
    • ResourceManager
    • App Timeline Server
    • JobHistory
    • WebAppProxyServer
    • Yarn Client
    • Yarn Client
    • NodeManager
    Hive
    • Hive MetaStore
    • HiveServer2
    • Hive Client
    		 Hive Client
    Spark jobs
    • Spark Client
    • SparkHistory
    • ThriftServer
    		 Spark Client
    Knox Knox N/A
    Zeppelin Zeppelin N/A
    Tez
    • Tomcat
    • Tez Client
    		 Tez Client
    Ganglia
    • Gmond
    • Httpd
    • Gmetad
    • Ganglia Client
    • Gmond
    • Ganglia Client
    Pig Pig Client Pig Client
    Sqoop Sqoop Client Sqoop Client
    Bigboot
    • Bigboot Client
    • Bigboot Monitor
    • Bigboot Client
    • Bigboot Monitor
    OpenLDAP OpenLDAP N/A
    Hue Hue N/A
  • Optional services
    Service name Master instance node Core instance node
    Flume
    • Flume Agent
    • Flume Client
    • Flume Agent
    • Flume Client
    Livy Livy N/A
    Superset Superset N/A
    Flink
    • FlinkHistoryServer
    • Flink Client
    		 Flink Client
    Ranger
    • RangerPlugin
    • RangerAdmin
    • RangerUserSync
    		 RangerPlugin
    Storm
    • Storm Client
    • UI
    • Nimbus
    • Logviewer
    • Storm Client
    • Supervisor
    • Logviewer
    Phoenix Phoneix Client Phoneix Client
    Analytics Zoo
    • Analytics-Zoo-Scala
    • Analytics-Zoo-Python
    • Analytics-Zoo-Scala
    • Analytics-Zoo-Python
    SmartData
    • JindoFS Namespace Service
    • JindoFS Storage Service
    • SmartData Client
    • JindoFS Storage Service
    • SmartData Client
    Kudu
    • Kudu Master
    • Kudu Client
    • Kudu Tserver
    • Kudu Master
    • Kudu Client
    HBase
    • HMaster
    • HBase Client
    • ThriftServer
    • HBase Client
    • HRegionServer
    ZooKeeper
    • ZooKeeper follower
    • ZooKeeper Client
    • ZooKeeper leader
    • ZooKeeper follower
    • ZooKeeper Client
    Oozie Oozie N/A
    Presto
    • Presto Client
    • PrestoMaster
    • Presto Client
    • PrestoWorker
    Impala
    • Impala Runtime and Shell
    • Impala Catalog Server
    • Impala StateStore Server
    • Impala Runtime and Shell
    • Impala Daemon Server
    • Impala StateStore Server

Druid cluster

Deployment details for service components on a Druid cluster
  • Required services
    Service name Master instance node Core instance node
    Druid
    • Druid Client
    • Coordinator
    • Overlord
    • Broker
    • MiddleManager
    • Historical
    • Druid Client
    HDFS
    • KMS
    • SecondaryNameNode
    • HttpFS
    • HDFS Client
    • NameNode
    • DataNode
    • HDFS Client
    Ganglia
    • Gmond
    • Httpd
    • Gmetad
    • Ganglia Client
    • Gmond
    • Ganglia Client
    ZooKeeper
    • ZooKeeper follower
    • ZooKeeper Client
    • ZooKeeper leader
    • ZooKeeper follower
    • ZooKeeper Client
    OpenLDAP OpenLDAP N/A
  • Optional services
    Service name Master instance node Core instance node
    YARN
    • ResourceManager
    • App Timeline Server
    • JobHistory
    • WebAppProxyServer
    • Yarn Client
    • Yarn Client
    • NodeManager
    Superset Superset N/A

Kafka cluster

Deployment details for service components on a Kafka cluster
  • Required services
    Service name Master instance node Core instance node
    Kafka-Manager Kafka Manager N/A
    Kafka
    • Kafka Client
    • KafkaMetadataMonitor
    • Kafka Rest Proxy
    • Kafka Broker controller
    • Kafka Schema Registry
    • Kafka Broker broker
    • Kafka Client
    Ganglia
    • Gmond
    • Httpd
    • Gmetad
    • Ganglia Client
    • Gmond
    • Ganglia Client
    ZooKeeper
    • ZooKeeper follower
    • ZooKeeper Client
    • ZooKeeper leader
    • ZooKeeper follower
    • ZooKeeper Client
  • Optional services
    Service name Master instance node Core instance node
    Ranger
    • RangerPlugin
    • RangerUserSync
    • RangerAdmin
    		 RangerPlugin
    Knox Knox N/A
    OpenLDAP OpenLDAP N/A

Zookeeper cluster

Deployment details for service components on a ZooKeeper cluster
Service name Master instance node Core instance node
Ganglia N/A
  • Gmond
  • Httpd
  • Gmetad
  • Ganglia Client
ZooKeeper N/A
  • ZooKeeper leader
  • ZooKeeper Client
  • ZooKeeper follower

Data Science cluster

Deployment details for service components on a Data Science cluster
  • Required services
    Service name Master instance node Core instance node
    HDFS
    • KMS
    • SecondaryNameNode
    • HttpFS
    • HDFS Client
    • NameNode
    • DataNode
    • HDFS Client
    Jupyter Jupyter Jupyter
    Ganglia
    • Gmond
    • Httpd
    • Gmetad
    • Ganglia Client
    • Gmond
    • Ganglia Client
    ZooKeeper
    • ZooKeeper follower
    • ZooKeeper Client
    • ZooKeeper leader
    • ZooKeeper follower
    • ZooKeeper Client
    Analytics Zoo
    • Analytics-Zoo-Python
    • Analytics-Zoo-Scala
    • Analytics-Zoo-Python
    • Analytics-Zoo-Scala
    ApacheDS ApacheDS N/A
    Tensorflow on YARN
    • TensorFlow-On-YARN
    • TensorFlow-On-YARN-Gateway
    • TensorFlow-On-YARN-History-Server
    • TensorFlow-On-YARN-Client
    • TensorFlow-On-YARN-Gateway
    Zeppelin Zeppelin N/A
    Spark jobs
    • Spark Client
    • ThriftServer
    • SparkHistory
    		 Spark Client
    YARN
    • ResourceManager
    • App Timeline Server
    • JobHistory
    • WebAppProxyServer
    • Yarn Client
    • Yarn Client
    • NodeManager
  • Optional services
    Service name Master instance node Core instance node
    Hue Hue RangerPlugin
    Hive
    • HiveServer2
    • Hive MetaStore
    • Hive Client
    		 Hive Client
    Tensorflow Tensorflow Tensorflow