本文介绍服务组件在E-MapReduce不同集群中各节点的部署情况。

背景信息

创建E-MapReduce集群时,不同类型的实例节点上会部署不同的服务角色。例如,Hadoop HDFS中的NameNode部署在Master节点中。

查看集群服务组件具体部署信息

  1. 登录阿里云E-MapReduce控制台
  2. 在顶部菜单栏处,根据实际情况选择地域(Region)和资源组
  3. 单击上方的集群管理页签。
  4. 集群管理页面,单击相应集群所在行的详情
  5. 在左侧导航栏,单击集群服务,选择查看的集群服务。
  6. 单击部署拓扑
    您可以查看服务组件的具体部署信息。

Hadoop集群

EMR-3.29.0版本,Hadoop集群服务组件具体部署信息:
  • 必选服务
    服务名 主实例节点 核心实例节点
    HDFS
    • KMS
    • SecondaryNameNode
    • HttpFS
    • HDFS Client
    • NameNode
    • DataNode
    • HDFS Client
    YARN
    • ResourceManager
    • App Timeline Server
    • JobHistory
    • WebAppProxyServer
    • Yarn Client
    • Yarn Client
    • NodeManager
    Hive
    • Hive MetaStore
    • HiveServer2
    • Hive Client
    Hive Client
    Spark
    • Spark Client
    • SparkHistory
    • ThriftServer
    Spark Client
    Knox Knox
    Tez
    • Tomcat
    • Tez Client
    Tez Client
    Ganglia
    • Gmond
    • Httpd
    • Gmetad
    • Ganglia Client
    • Gmond
    • Ganglia Client
    Sqoop Sqoop Client Sqoop Client
    Bigboot
    • Bigboot Client
    • Bigboot Monitor
    • Bigboot Client
    • Bigboot Monitor
    OpenLDAP OpenLDAP
    Hue Hue
    SmartData
    • Jindo Namespace Service
    • Jindo Storage Service
    • Jindo Client
    • Jindo Storage Service
    • Jindo Client
  • 可选服务
    服务名 主实例节点 核心实例节点
    LIVY Livy
    Superset Superset
    Flink
    • FlinkHistoryServer
    • Flink Client
    Flink Client
    RANGER
    • RangerPlugin
    • RangerAdmin
    • RangerUserSync
    • Solr
    RangerPlugin
    Storm
    • Storm Client
    • UI
    • Nimbus
    • Logviewer
    • Storm Client
    • Supervisor
    • Logviewer
    Phoenix Phoenix Client Phoenix Client
    Kudu
    • Kudu Master
    • Kudu Client
    • Kudu Tserver
    • Kudu Master
    • Kudu Client
    HBase
    • HMaster
    • HBase Client
    • ThriftServer
    • HBase Client
    • HRegionServer
    ZooKeeper
    • ZooKeeper follower
    • ZooKeeper Client
    • ZooKeeper follower
    • ZooKeeper leader
    • ZooKeeper Client
    Oozie Oozie
    Presto
    • Presto Client
    • PrestoMaster
    • Presto Client
    • PrestoWorker
    Impala
    • Impala Runtime and Shell
    • Impala Catalog Server
    • Impala StateStore Server
    • Impala Runtime and Shell
    • Impala Daemon Server
    Pig Pig Client Pig Client
    Zeppelin Zeppelin
    FLUME
    • Flume Agent
    • Flume Client
    • Flume Agent
    • Flume Client

Druid集群

EMR-3.29.0版本,Druid集群服务组件具体部署信息:
  • 必选服务
    服务名 主实例节点 核心实例节点
    Druid
    • Druid Client
    • Coordinator
    • Overlord
    • Broker
    • Router
    • MiddleManager
    • Historical
    • Druid Client
    HDFS
    • KMS
    • SecondaryNameNode
    • HttpFS
    • HDFS Client
    • NameNode
    • DataNode
    • HDFS Client
    Ganglia
    • Gmond
    • Httpd
    • Gmetad
    • Ganglia Client
    • Gmond
    • Ganglia Client
    ZooKeeper
    • ZooKeeper follower
    • ZooKeeper Client
    • ZooKeeper leader
    • ZooKeeper follower
    • ZooKeeper Client
    OpenLDAP OpenLDAP
    Bigboot
    • Bigboot Client
    • Bigboot Monitor
    • Bigboot Client
    • Bigboot Monitor
    SmartData
    • Jindo Namespace Service
    • Jindo Storage Service
    • Jindo Client
    • Jindo Storage Service
    • Jindo Client
  • 可选服务
    服务名 主实例节点 核心实例节点
    YARN
    • ResourceManager
    • App Timeline Server
    • JobHistory
    • WebAppProxyServer
    • Yarn Client
    • Yarn Client
    • NodeManager
    Superset Superset

Kafka集群

EMR-3.29.0版本,Kafka集群服务组件具体部署信息:
  • 必选服务
    服务名 主实例节点 核心实例节点
    Kafka-Manager Kafka Manager
    Kafka
    • Kafka Client
    • KafkaMetadataMonitor
    • Kafka Rest Proxy
    • Kafka Broker broker
    • Kafka Schema Registry
    • Kafka Broker broker
    • Kafka Client
    Ganglia
    • Gmond
    • Httpd
    • Gmetad
    • Ganglia Client
    • Gmond
    • Ganglia Client
    ZooKeeper
    • ZooKeeper follower
    • ZooKeeper Client
    • ZooKeeper leader
    • ZooKeeper follower
    • ZooKeeper Client
    OpenLDAP OpenLDAP
  • 可选服务
    服务名 主实例节点 核心实例节点
    RANGER
    • RangerPlugin
    • RangerUserSync
    • RangerAdmin
    • Solr
    RangerPlugin
    Knox Knox

ZooKeeper集群

EMR-3.29.0版本,ZooKeeper集群服务组件具体部署信息如下。
服务名 主实例节点 核心实例节点
Ganglia
  • Gmond
  • Httpd
  • Gmetad
  • Ganglia Client
ZooKeeper
  • ZooKeeper leader
  • ZooKeeper Client
  • ZooKeeper follower

Dataflow集群

EMR-3.28.2版本,Dataflow集群服务组件具体部署信息:
  • 必选服务
    服务名 主实例节点 核心实例节点
    HDFS
    • KMS
    • SecondaryNameNode
    • HttpFS
    • HDFS Client
    • NameNode
    • DataNode
    • HDFS Client
    YARN
    • ResourceManager
    • App Timeline Server
    • JobHistory
    • WebAppProxyServer
    • Yarn Client
    • Yarn Client
    • NodeManager
    Ganglia
    • Gmond
    • Httpd
    • Gmetad
    • Ganglia Client
    • Gmond
    • Ganglia Client
    ZooKeeper
    • ZooKeeper follower
    • ZooKeeper Client
    • ZooKeeper leader
    • ZooKeeper follower
    • ZooKeeper Client
    Flink-Vvp Flink-Vvp
    SmartData
    • JindoFS Namespace Service
    • JindoFS Storage Service
    • SmartData Client
    • JindoFS Storage Service
    • SmartData Client
    Bigboot
    • Bigboot Client
    • Bigboot Monitor
    • Bigboot Client
    • Bigboot Monitor
    OpenLDAP OpenLDAP
  • 可选服务

    PAI-Alink:Alink

Data Science集群

EMR-3.29.1版本,Dataflow集群服务组件具体部署信息:
  • 必选服务
    服务名 主实例节点 核心实例节点
    HDFS
    • HDFS Client
    • KMS
    • HttpFS
    • NameNode
    • SecondaryNameNode
    • HDFS Client
    • DataNode
    YARN
    • WebAppProxyServer
    • JobHistory
    • App Timeline Server
    • ResourceManager
    • Yarn Client
    • NodeManager
    • Yarn Client
    ZooKeeper
    • ZooKeeper Client
    • ZooKeeper follower
    • ZooKeeper Client
    • ZooKeeper leader
    • ZooKeeper follower
    Knox Knox
    Tensorflow on YARN
    • TensorFlow-On-YARN-Gateway
    • TensorFlow-On-YARN-History-Server
    • TensorFlow-On-YARN
    • TensorFlow-On-YARN-Client
    • TensorFlow-On-YARN-Gateway
    SmartData
    • Jindo Namespace Service
    • Jindo Storage Service
    • Jindo Client
    • Jindo Storage Service
    • Jindo Client
    Bigoot
    • Bigboot Monitor
    • Bigboot Client
    • Bigboot Monitor
    • Bigboot Client
    PAI-EASYREC Easyrec Easyrec
    PAI-EAS PAIEAS PAIEAS
    PAI-Faiss Faiss Faiss
    PAI-Redis Redis Redis
    PAI-Alink Alink
    Flink-Vvp Flink-Vvp
    OpenLDAP OpenLDAP
    Jindo SDK Jindo SDK Jindo SDK
  • 可选服务
    服务名 主实例节点 核心实例节点
    Zeppelin Zeppelin
    PAI-REC Rec
    AUTOML AUTOML AUTOML
    TensorFlow TensorFlow TensorFlow