Spark SQL job configuration
Note By default, the Spark SQL mode to submit a job is Yarn mode.
- Log on to the Alibaba Cloud E-MapReduce Console to enter the Cluster List page with primary account.
- Click the Data Platform tab on the top to enter the Project List page.
- Click Design Workflow of the specified project in the Operation column.
- On the left side of the Job Editing page, right-click on the folder you want to operate and select New Job.
- In the New Job dialog box, enter the job name, job description.
Once the job type is selected, it cannot be modified.
- Click OK.
Note You can also create subfolder, rename folder, and delete folder by right-clicking on the folder.
- Select the Spark SQL job type to create a Spark SQL job. This type of job is submitted in the background by using the following process:
spark-sql [options] [cli option]
- Enter the parameters in the Content box with parameters subsequent to Spark SQL commands.
- -e option
Directly write running SQL for -e options by inputting it into the Parameters box of the job, for example:
-e "show databases;"
- -f option
-f options can be used to specify a Spark SQL script file. Loading well prepared Spark SQL script files on OSS can give more flexibility. We recommend that you use this operation mode, for example:
- -e option
- Click Save to complete Spark SQL job configuration.