All Products
Search
Document Center

createJob

Last Updated: May 11, 2018

Method description

Creates and submits a job.

Parameter description

Parameter Type Required Description
jobDescription JobDescription Yes Description of each task in the job, and DAG dependencies between tasks.

Note: The package names of the following JobDescription, TaskDag, TaskDescription, and ResourceDescription are all com.aliyuncs.batchcompute.pojo.v20151111.

Attributes of JobDescription

The getter and setter methods are available to all attributes.

Attribute Type Required Description
name String Yes Name of a job.
priority int Yes The priority is specified by an integer ranging from 0 to 1000. Priority increases with the value.
description String No Brief description of a job.
dag DAG No It is required when type is DAG. It describes each task in the job and DAG dependencies between tasks.
app AppJobDescription No It is required when type is App. It describes App configuration parameters, inputs, and outputs.
jobFailOnInstanceFail Boolean Yes Specifies whether a job fails when an instance fails. Generally, the value is true.
autoRelease Boolean No Specifies whether the system automatically releases (deletes) a job after the job runs successfully. Default value: false.
type String Yes Type of a job. Currently, a task can be described only in the form of a directed acyclic graph (DAG). Values: DAG and App.
notification Notification No Notification configuration. You can configure the topic or job related events of the MNS.

Attributes of DAG

The getter and setter methods are available to all attributes.

Attribute Type Required Description
tasks Map Yes Mapping between all tasks names and task descriptions.
dependencies Map> No Dependencies between the task. It is also a map that describes the one-to-many relationship, where “key” indicates the source task name, and “value” indicates the target task name. For more information, see the example.

Attributes of TaskDescription

The getter and setter methods are available to all attributes.

Attribute Type Required or not Description
taskName String Yes Name of a task.
clusterId String No ID of a cluster. You can select only one of clusterId and AutoCluster.
autoCluster AutoCluster No You can select only one of autoCluster and clusterId.
parameters Parameters Yes Configurations related to program running.
inputMapping Map No Mapping from the OSS paths and local mount positions. All mount positions must end with a slash (/). They can be on the same or different buckets, but must belong to the same user. The mount position is a drive letter in the Windows operating system, and a folder in the Linux operating system. “key” indicates an OSS path, and “value” indicates a mount path, for example, “D:” or “/home/admin/dist1”.
outMapping Map No Mapping from the local paths to OSS paths. “key” indicates a local path, and “value” indicates an OSS path.
logMapping Map No Mapping from the log paths to OSS paths. “key” indicates a local path, and “value” indicates an OSS path.
instanceCount int Yes Number of instances in a task. It is a positive integer.
timeout int Yes Maximum execution time (timeout) of an instance in the task. The value ranges from 1 to 86400. The unit is second.
mounts Mounts No Network mounting configurations of an instance, which is described by Mounts. Currently, NAS mounting and OSS mounting are supported.
maxRetryCount int No Maximum number of retries. Default value: 0
writeSupport Boolean No Write support for the read-only mounted directory in inputMapping. Note that files that have been mounted are read-only and cannot be overwritten.

Attributes of AutoCluster

The getter and setter methods are available to all attributes.

Attribute Type Required Description
ECSImageId String Yes It can be an ECS image ID or a registered image ID.
instanceType String Yes Type of the instance. Supported instance types vary according to the region.
resourceType String Yes Type of the resource. Currently, this parameter can only be set to OnDemand, which is the default value.
SpotStrategy String Yes Spot strategy for an instance. It takes effect only when ResourceType is Spot. Values: SpotWithPriceLimit (set the maximum price for the spot instance) and SpotAsPriceGo (the system provides a price automatically, and the maximum value is the Pay-As-You-Go price).
SpotPriceLimit Float No Maximum price of an instance per hour. The value can contain at most three decimals. It takes effect when SpotStrategy is SpotWithPriceLimit.
userData Map No User data
configs Configs No Cluster configurations, such as the disk configuration of the instance. It is described by [Configs].
ReserveOnFail Bool No When a task fails, related VMs are not released, and the fees for these resources are still charged until the user deletes the job. The default value is False. It is used only in a survey.

Attributes of Parameters

The getter and setter methods are available to all attributes.

Attribute Type Required Description
command Command Yes Command line parameter related to a user program.
inputMappingConfig InputMappingConfig No Configuration of an NFS mounting service.
stdoutRedirectPath String Yes OSS directory to which the standard output is redirected.
stderrRedirectPath String Yes OSS directory to which the standard error is redirected.

Attributes of Command (the getter and setter methods are available to all attributes)

Attribute Type Required Description
commandLine String Yes Specifies a command for executing a user program.
PackagePath String Yes OSS path in which a user program is located.
EnvVars Map No Environment variables to be configured when the user program starts running.

Attributes of InputMappingConfig

The getter and setter methods are available to all attributes.

Attribute Type Required Description
lock Boolean No Specifies whether the NFS mounting service supports the network file lock. If it is set to true, the network file lock is enabled to provide backend support for the file lock. If it is set to false, the network file lock is disabled and the file access performance can be improved. The file lock is supported only at the NFS client and will not be synchronized to the server. In some operating systems, the displayed file lock methods may fail.
locale String No All objects on OSS are named using UTF-8. This parameter determines the local character set used after mounting. Optional values include GBK, GB2312-80, BIG5, ANSI, EUC-JP, EUC-TW, EUC-KR, SHIFT-JIS, and KSC5601.

Attributes of Notification

The getter and setter methods are available to all attributes.

Attribute Type Required Description
topic Topic No Message topic

Attributes of Topic

The getter and setter methods are available to all attributes.

Attribute Type Required Description
endpoint String Yes End point of the MNS region. The format is http://${your_user_id}.mns.${region}-internal.aliyuncs.com/. Use the intranet endpoint whenever possible.
name String Yes Name of the topic
events List<String> Yes List of events. You must enter the job related events.

Attributes of AppJobDescription

Attribute Type Required Description
appName String Yes App name. The valid character set is [a-zA-Z0-9_-:]. The length cannot exceed 1024 characters. Two formats are available for the App name, and are used to access a public App and your private App, respectively. For example, to access the public App “App1”, set appName to Public:App1. “Public” can be excluded, that is, the App name changes to “:App1”. If App1 is your private App, directly set appName to App1.
inputs Map Yes Input parameters of the job. “Map” must be a value between 0 and 50. “Key” must be a parameter name defined in InputParameters of the App, and “Value” must be consistent with the type of the parameter in the App.
outputs Map No Output parameters of the job. “Map” must be a value between 0 and 50. “Key” must be a parameter name that is defined in OutputParameters of the App and LocalPath of which is not null, and “Value” must be an OSS path. Default value:{}.
logging AppJobDescription.Loggin No Log configuration.
config Map No Running configuration of the job.

Attributes of AppJobDescription.Loggin

Parameter Type Required Description
stdoutPath String No OSS directory to which the standard output is redirected. The length cannot exceed 1000 characters. Default value: null
stderrPath String No OSS directory to which the standard error is redirected. The length cannot exceed 1000 characters. Default value: null

Response description

If creation succeeds, a CreateJobResponse instance is returned. You can use response.getJobId() to obtain the name of the created job.

Type Description
CreateJobResponse Obtains the ID of the created job.

The CreateJobResponse package name is com.aliyuncs.batchcompute.model.v20151111.Other responses listed in the following are under this package.

  • If creation fails, a ClientException is thrown.

Sample code for submitting a DAG job

The structure of JobDescription is complex. Therefore, we recommend using IntelliJ IDEA Community for development.

  1. package com.aliyuncs.batchcompute.sample.v20151111;
  2. import com.aliyuncs.batchcompute.main.v20151111.*;
  3. import com.aliyuncs.batchcompute.model.v20151111.*;
  4. import com.aliyuncs.batchcompute.pojo.v20151111.*;
  5. import com.aliyuncs.exceptions.ClientException;
  6. public class CreateJob {
  7. public static void main(String[] args) {
  8. BatchCompute client = new BatchComputeClient("cn-shenzhen", "your_access_id", "your_access_secret");
  9. try {
  10. JobDescription jobDescription = getJobDesc();
  11. CreateJobResponse response = client.createJob(jobDescription);
  12. String jobId = response.getJobId();
  13. //Creation succeeded
  14. System.out.println("Got job id:" + jobId);
  15. } catch (ClientException e) {
  16. e.printStackTrace();
  17. //Creation failed
  18. }
  19. }
  20. private static JobDescription getJobDesc() {
  21. JobDescription desc = new JobDescription();
  22. desc.setName("testJob");
  23. desc.setPriority(1);
  24. desc.setDescription("Demo");
  25. desc.setType("DAG");
  26. desc.setJobFailOnInstanceFail(true);
  27. desc.setAutoRelease(false);
  28. DAG dag = new DAG();
  29. dag.addTask(getTaskDesc());
  30. desc.setDag(dag);
  31. Notification noti = new Notification();
  32. Topic topic = new Topic();
  33. topic.addEvent(Topic.ON_JOB_FAILED);
  34. topic.addEvent(Topic.ON_JOB_FINISHED);
  35. noti.setTopic(topic);
  36. topic.setName("tp_n1");
  37. topic.setEndpoint("xxxxx");
  38. desc.setNotification(noti);
  39. return desc;
  40. }
  41. private static TaskDescription getTaskDesc() {
  42. TaskDescription task = new TaskDescription();
  43. task.setClusterId("cls-xxxxxx");
  44. task.setInstanceCount(1);
  45. task.setMaxRetryCount(2);
  46. task.setTaskName("task_1");
  47. task.setTimeout(10000);
  48. Parameters parameters = new Parameters();
  49. Command cmd = new Command();
  50. cmd.setCommandLine("python main.py");
  51. cmd.setPackagePath("oss://my-bucket/test/worker.tar.gz");
  52. parameters.setCommand(cmd);
  53. parameters.setStderrRedirectPath("oss://my-bucket/test/logs/");
  54. parameters.setStdoutRedirectPath("oss://my-bucket/test/logs/");
  55. InputMappingConfig input = new InputMappingConfig();
  56. input.setLocale("GBK");
  57. input.setLock(true);
  58. parameters.setInputMappingConfig(input);
  59. task.setParameters(parameters);
  60. task.addInputMapping("oss://my-bucket/disk1/", "/home/admin/disk1/");
  61. task.addOutputMapping("/home/admin/disk2/", "oss://my-bucket/disk2/");
  62. return task;
  63. }
  64. }

Sample code for submitting a App job

  1. package com.aliyuncs.batchcompute;
  2. import com.aliyuncs.batchcompute.main.v20151111.BatchCompute;
  3. import com.aliyuncs.batchcompute.main.v20151111.BatchComputeClient;
  4. import com.aliyuncs.batchcompute.model.v20151111.CreateJobResponse;
  5. import com.aliyuncs.batchcompute.pojo.v20151111.AppJobDescription;
  6. import com.aliyuncs.batchcompute.pojo.v20151111.JobDescription;
  7. import com.aliyuncs.exceptions.ClientException;
  8. public class CreateAppJob {
  9. public static void main(String[] args){
  10. BatchCompute client = new BatchComputeClient("cn-shenzhen", "your_access_id", "your_access_secret");
  11. try {
  12. JobDescription jobDescription = getJobDesc();
  13. CreateJobResponse response = client.createJob(jobDescription);
  14. String jobId = response.getJobId();
  15. //Creation succeeded
  16. System.out.println("Got job id:" + jobId);
  17. } catch (ClientException e) {
  18. e.printStackTrace();
  19. //Creation failed
  20. }
  21. }
  22. private static JobDescription getJobDesc() {
  23. JobDescription jobDesc = new JobDescription();
  24. AppJobDescription app = new AppJobDescription();
  25. app.setAppName("myApp");
  26. app.addInputs("abc", "oss://bucket/inp");
  27. app.addOutputs("out", "oss://bucket/out");
  28. AppJobDescription.Logging logging = new AppJobDescription.Logging();
  29. logging.setStderrPath("oss://bucket/key/logs");
  30. logging.setStdoutPath("oss://bucket/key/logs");
  31. app.setLogging(logging);
  32. jobDesc.setName("java-sdk-test");
  33. jobDesc.setDescription("java sdk test job");
  34. jobDesc.setType("App");
  35. jobDesc.setApp(app);
  36. return jobDesc;
  37. }
  38. }