View Spark job information
The Lindorm Distributed Processing System (LDPS) provides a Spark web UI to view detailed information about your Spark jobs, such as their status, submission times, and resource usage.
Access the Spark web UI
-
To obtain the Spark web UI address, see Manage jobs in the Lindorm console.
-
For more information about the open source Spark web UI, see Web UI.
To access the Spark web UI, paste its address into your browser's address bar. The interface opens to the Jobs page by default. The header displays basic application information, such as User, Total Uptime, and Scheduling Mode (for example, FIFO). Below the header is an expandable Event Timeline section.
Table 1. Spark web UI tabs
|
Tab |
Description |
|
Jobs |
Displays detailed information about all jobs. |
|
Stages |
Displays status information for all stages of a Spark job. |
|
Storage |
Displays storage information for persisted resilient distributed datasets (RDDs) and DataFrames (DFs). |
|
Environment |
Displays the environment configuration for the Spark job, compiled at startup from the runtime environment, configuration files, and user-specified parameters. |
|
Executors |
Displays the status of the executors for the Spark job. |
|
SQL |
Displays details for SQL queries executed by the Lindorm Distributed Processing System (LDPS). |
|
Kyuubi Query Engine |
Displays information about Java Database Connectivity (JDBC) sessions. |