In this tutorial, you will learn how to configure a Spark SQL job.
Note By default, the Spark SQL mode to submit a job is Yarn mode.
- Log on to the Alibaba Cloud E-MapReduce console.
- At the top of the navigation bar, click Data Platform.
- In the Actions column, click Design Workflow of the specified project.
- On the left side of the Job Editing page, right-click on the folder you want to operate and select New Job.
- In the New Job dialog box, enter the job name and description.
- Click OK.
Note You can also create subfolder, rename folder, and delete folder by right-clicking on the folder.
- Select the Spark SQL job type to create a Spark SQL job. This type of job is submitted in the background by using the following method:
spark-sql [options] [cli option]
- Enter the parameters in the Content field with parameters subsequent to Spark SQL commands.
- -e option
Directly write running SQL for -e options by inputting it into the Content field of the job, for example:
-e "show databases;"
- -f option
-f options can be used to specify a Spark SQL script file. Loading well prepared Spark SQL script files on OSS can give more flexibility. We recommend that you use this operation mode, for example:
- -e option
- Click Save to complete Spark SQL job configuration.