In this tutorial, you will learn how to configure a Spark SQL job.

Note By default, the mode of Spark SQL used for submitting a job is YARN.

Procedure

  1. Log on to the Alibaba Cloud E-MapReduce console.
  2. At the top of the navigation bar, click Data Platform.
  3. In the Actions column, click Design Workflow next to the specified project.
  4. On the left of the Job Editing page, right-click the folder you want to operate and select New Job.
  5. In the New Job dialog box, enter the job name and description.
  6. Click OK.
    Note You can also create subfolders, rename folders, and delete folders by right-clicking on them.
  7. Select the Spark SQL job type to create a Spark SQL job. This type of job is submitted in the background using the following method.
    spark-sql [options] [cli option]
  8. Enter the parameters in the Content field after the Spark SQL commands.
    • -e option
      -e options can be written to the running SQL by inputting them into the Content field of the job. For example:
      -e "show databases;"
    • -f option
      -f options can be used to specify a Spark SQL script file. Uploading well-prepared Spark SQL script files to OSS can provide greater flexibility. We recommend that you use this operation mode. For example:
      -f ossref://your-bucket/your-spark-sql-script.sql
  9. Click Save to complete Spark SQL job configuration.