E-MapReduce:Manage jobs using kubectl

Check job status

Run kubectl describe against the SparkApplication resource to inspect the current state of a job.

Before you run the command, get the job name and namespace:

• Job name: In the EMR console, go to the cluster details page and find the job in the Jobs section.

• Namespace: In the EMR console, go to the Cluster Details tab of the cluster details page.

kubectl describe SparkApplication <job-name> --namespace <namespace>

The output shows the full resource spec and status. Look for the Application State field under Status:

Name:         spark-pi-simple
Namespace:    c-48e779e0d9ad****
Labels:       <none>
Annotations:  <none>
API Version:  sparkoperator.k8s.io/v1beta2
Kind:         SparkApplication
Metadata:
  Creation Timestamp:  2021-07-22T06:25:33Z
  Generation:          1
  Resource Version:  7503740
  UID:               930874ad-bb17-47f1-a556-55118c1d****
Spec:
  Arguments:
    1000
  Driver:
    Core Limit:  1000m
    Cores:       1
    Memory:      4g
  Executor:
    Core Limit:           1000m
    Cores:                1
    Instances:            1
    Memory:               8g
    Memory Overhead:      1g
  Image:                  registry-vpc.cn-hangzhou.aliyuncs.com/emr/spark:emr-2.4.5-1.0.0
  Main Application File:  local:///opt/spark/examples/target/scala-2.11/jars/spark-examples_2.11-2.4.5.jar
  Main Class:             org.apache.spark.examples.SparkPi
  Spark Version:          2.4.5
  Type:                   Scala
Status:
  Application State:
    State:  RUNNING
  Driver Info:
    Pod Name:                spark-pi-simple-driver
    Web UI Address:          172.16.230.240:4040
    Web UI Ingress Address:  spark-pi-simple.c-48e779e0d9ad4bfd.c7f6b768c34764c27ab740bdb1fc2a3ff.cn-hangzhou.alicontainer.com
    Web UI Ingress Name:     spark-pi-simple-ui-ingress
    Web UI Port:             4040
    Web UI Service Name:     spark-pi-simple-ui-svc
  Execution Attempts:        1
  Executor State:
    spark-pi-1626935142670-exec-1:  RUNNING
  Last Submission Attempt Time:     2021-07-22T06:25:33Z
  Spark Application Id:             spark-15b44f956ecc40b1ae59a27ca18d****
  Submission Attempts:              1
  Submission ID:                    d71f30e2-9bf8-4da1-8412-b585fd45****
  Termination Time:                 <nil>
Events:
  Type    Reason                     Age   From            Message
  ----    ------                     ----  ----            -------
  Normal  SparkApplicationAdded      17s   spark-operator  SparkApplication spark-pi-simple was added, enqueuing it for submission
  Normal  SparkApplicationSubmitted  14s   spark-operator  SparkApplication spark-pi-simple was submitted successfully
  Normal  SparkDriverRunning         13s   spark-operator  Driver spark-pi-simple-driver is running
  Normal  SparkExecutorPending       7s    spark-operator  Executor spark-pi-1626935142670-exec-1 is pending
  Normal  SparkExecutorRunning       6s    spark-operator  Executor spark-pi-1626935142670-exec-1 is running

View job logs

The driver pod name follows the pattern <job-name>-driver. For example, if the job name is spark-pi-simple, the driver pod name is spark-pi-simple-driver.

kubectl logs <job-name>-driver -n <namespace>

Example output:

......
Pi is roughly 3.141488791414888
21/07/22 14:37:57 INFO SparkContext: Successfully stopped SparkContext
21/07/22 14:37:57 INFO ShutdownHookManager: Shutdown hook called
21/07/22 14:37:57 INFO ShutdownHookManager: Deleting directory /var/data/spark-b6a43b55-a354-44d7-ae5e-45b8b1493edb/spark-56aae0d1-37b9-4a7d-9c99-4e4ca12deb4b
21/07/22 14:37:57 INFO ShutdownHookManager: Deleting directory /tmp/spark-e2500491-6ed7-48d7-b94e-a9ebeb899320

Delete a job

Deleting a SparkApplication stops the running job and removes all associated resources.

kubectl delete SparkApplication <job-name> -n <namespace>

A confirmation message appears when the deletion is complete:

sparkapplication.sparkoperator.k8s.io "spark-pi-simple" deleted