The real-time check feature of E-MapReduce (EMR) Doctor can be used to check the status of a cluster in real time at an interval of 5 minutes. You can view the status of a cluster, related issues, and causes for issues, and troubleshoot issues. This helps ensure cluster execution stability.

Precautions

You must activate EMR Doctor before you can use health check in the EMR console. For information about how to activate EMR Doctor, see Activate EMR Doctor (Hadoop clusters).

Start real-time check

  1. Go to the Real-time Check tab.
    1. Log on to the EMR on ECS console.
    2. In the top navigation bar, select the region where your cluster resides and select a resource group based on your business requirements.
    3. On the EMR on ECS page, find the desired cluster and click the name of the cluster in the Cluster ID/Name column.
    4. On the page that appears, click the Health Check tab.
      The Real-time Check tab appears.
  2. Click Start Check.
    Start Check
  3. After the real-time check is complete, click View Latest Health Check Report.
  4. Click Save Report in the lower-right corner of the report.
    The real-time check report is not automatically saved by default. If you want to view recent real-time check reports, you must manually save them. You can save up to 30 recently generated real-time check reports.

Status analysis for computing resources

Detailed analysis

This section displays the analysis details and scores of the jobs in the cluster in recent 5 minutes, and provides optimization suggestions. You can optimize the jobs based on the suggestions. This section also displays the jobs in which abnormal behaviors are detected. You can troubleshoot issues based on the displayed information.

Basic computing information

The tables and charts in this section display the following information about the jobs that are run in recent 5 minutes:
  • Memory consumed by different types of engines (GB*Sec)
  • vCPUs consumed by different types of engines (VCore*Sec)
  • Pie chart for cluster computing power consumed by different types of engines
  • Pie chart for memory consumed by jobs that are submitted by different users

Job information

EMR Doctor collects information about the jobs that are complete in recent 5 minutes and the jobs that are still running, processes and analyzes the jobs in real time, and displays the key jobs that affect the cluster execution based on real-time analysis results. You can optimize jobs based on suggestions or handle exceptions that occurred on the jobs to improve cluster stability.

The real-time check feature can analyze and check for jobs of different types of compute engines. Supported compute engines include MapReduce, Tez, and Spark.

The real-time check feature allows you to view the list of jobs that consume the most memory (GB*Sec) in descending order. The feature also allows you to view the list of jobs sorted by scores in ascending order. The following table describes the information in each data record.
ParameterDescription
Job NameThe name of the job.
Engine TypeThe type of the compute engine. Compute engines include MapReduce, Tez, and Spark.
SQL StatementThis parameter needs to be configured only for SQL-type jobs.
APP IDSFor Hive on MapReduce jobs, an SQL statement may contain multiple application IDs.
UsernameThe user who submitted the job.
ScoreThe score of the job.
Health StatusSpecifies whether to mark the job for governance.
SuggestionThe optimization suggestion for the job.
Memory (GB*Sec)The total cluster memory consumed by the job.
Memory UsageThe average memory usage of the job.
CPU (vCore*Sec)The total cluster vCPUs consumed by the job.
CPU UtilizationThe average CPU utilization of the job.
Note EMR Doctor can summarize the existing issues of jobs of different types of engines and provide optimization suggestions. You can manually optimize the jobs based on the suggestions. EMR Doctor is not responsible for optimization results.