The real-time check feature of E-MapReduce (EMR) Doctor can be used to check the status of a cluster in real time at an interval of 5 minutes. You can view the status of a cluster, related issues, and causes for issues, and troubleshoot issues. This helps ensure cluster execution stability.
You must activate EMR Doctor before you can use health check in the EMR console. For information about how to activate EMR Doctor, see Activate EMR Doctor (Hadoop clusters).
Start real-time check
- Go to the Basic Information tab.
- Log on to the EMR on ECS console.
- In the top navigation bar, select the region where your cluster resides and select a resource group based on your business requirements.
- On the EMR on ECS page, find the desired cluster and click the name of the cluster in the Cluster ID/Name column.
- On the page that appears, click the Health Check tab.
- On the Health Check tab, click Start Real-time Check. After the real-time check is complete, click View Latest Health Check Report to view the real-time check report of the cluster.
The real-time check report is not automatically saved by default. If you want to view recent real-time check reports, you must manually save them. You can save up to 30 recently generated real-time check reports.
Status analysis for computing resources
This section displays the analysis details and scores of the jobs in the cluster in recent 5 minutes, and provides optimization suggestions. You can optimize the jobs based on the suggestions. This section also displays the jobs in which abnormal behaviors are detected. You can troubleshoot issues based on the displayed information.
Basic computing information
- Memory consumed by different types of engines (GB*Sec)
- vCPUs consumed by different types of engines (VCore*Sec)
- Pie chart for cluster computing power consumed by different types of engines
- Pie chart for memory consumed by jobs that are submitted by different users
EMR Doctor collects information about the jobs that are complete in recent 5 minutes and the jobs that are still running, processes and analyzes the jobs in real time, and displays the key jobs that affect the cluster execution based on real-time analysis results. You can optimize jobs based on suggestions or handle exceptions that occurred on the jobs to improve cluster stability.
The real-time check feature can analyze and check for jobs of different types of compute engines. Supported compute engines include MapReduce, Tez, and Spark.
|Job Name||The name of the job.|
|Engine Type||The type of the compute engine. Compute engines include MapReduce, Tez, and Spark.|
|SQL Statement||This parameter needs to be configured only for SQL-type jobs.|
|APP IDS||For Hive on MapReduce jobs, an SQL statement may contain multiple application IDs.|
|Username||The user who submitted the job.|
|Score||The score of the job.|
|Health Status||Specifies whether to mark the job for governance.|
|Suggestion||The optimization suggestion for the job.|
|Memory (GB*Sec)||The total cluster memory consumed by the job.|
|Memory Usage||The average memory usage of the job.|
|CPU (vCore*Sec)||The total cluster vCPUs consumed by the job.|
|CPU Utilization||The average CPU utilization of the job.|