Method description
Creates and submits a job.
Parameter description
Parameter | Type | Required | Description |
jobDescription | JobDescription | Yes | Description of each task in the job, and DAG dependencies between tasks. |
Note: The package names of the following JobDescription, TaskDag, TaskDescription, and ResourceDescription are all com.aliyuncs.batchcompute.pojo.v20151111.
Attributes of JobDescription
The getter and setter methods are available to all attributes.
Attribute | Type | Required | Description |
name | String | Yes | Name of a job. |
priority | int | Yes | The priority is specified by an integer ranging from 0 to 1000. Priority increases with the value. |
description | String | No | Brief description of a job. |
dag | DAG | No | It is required when type is DAG. It describes each task in the job and DAG dependencies between tasks. |
app | AppJobDescription | No | It is required when type is App. It describes App configuration parameters, inputs, and outputs. |
jobFailOnInstanceFail | Boolean | Yes | Specifies whether a job fails when an instance fails. Generally, the value is true. |
autoRelease | Boolean | No | Specifies whether the system automatically releases (deletes) a job after the job runs successfully. Default value: false. |
type | String | Yes | Type of a job. Currently, a task can be described only in the form of a directed acyclic graph (DAG). Values: DAG and App. |
notification | Notification | No | Notification configuration. You can configure the topic or job related events of the MNS. |
Attributes of DAG
The getter and setter methods are available to all attributes.
Attribute | Type | Required | Description |
tasks | Map | Yes | Mapping between all tasks names and task descriptions. |
dependencies | Map | No | Dependencies between the task. It is also a map that describes the one-to-many relationship, where “key” indicates the source task name, and “value” indicates the target task name. For more information, see the example. |
Attributes of TaskDescription
The getter and setter methods are available to all attributes.
Attribute | Type | Required or not | Description |
taskName | String | Yes | Name of a task. |
clusterId | String | No | ID of a cluster. You can select only one of clusterId and AutoCluster. |
autoCluster | AutoCluster | No | You can select only one of autoCluster and clusterId. |
parameters | Parameters | Yes | Configurations related to program running. |
inputMapping | Map | No | Mapping from the OSS paths and local mount positions. All mount positions must end with a slash (/). They can be on the same or different buckets, but must belong to the same user. The mount position is a drive letter in the Windows operating system, and a folder in the Linux operating system. “key” indicates an OSS path, and “value” indicates a mount path, for example, “D:” or “/home/admin/dist1”. |
outMapping | Map | No | Mapping from the local paths to OSS paths. “key” indicates a local path, and “value” indicates an OSS path. |
logMapping | Map | No | Mapping from the log paths to OSS paths. “key” indicates a local path, and “value” indicates an OSS path. |
instanceCount | int | Yes | Number of instances in a task. It is a positive integer. |
timeout | int | Yes | Maximum execution time (timeout) of an instance in the task. The value ranges from 1 to 86400. The unit is second. |
mounts | Mounts | No | Network mounting configurations of an instance, which is described by Mounts. Currently, NAS mounting and OSS mounting are supported. |
maxRetryCount | int | No | Maximum number of retries. Default value: 0 |
writeSupport | Boolean | No | Write support for the read-only mounted directory in inputMapping. Note that files that have been mounted are read-only and cannot be overwritten. |
Attributes of AutoCluster
The getter and setter methods are available to all attributes.
Attribute | Type | Required | Description |
ECSImageId | String | Yes | It can be an ECS image ID or a registered image ID. |
instanceType | String | Yes | Type of the instance. Supported instance types vary according to the region. |
resourceType | String | Yes | Type of the resource. Currently, this parameter can only be set to OnDemand, which is the default value. |
SpotStrategy | String | Yes | Spot strategy for an instance. It takes effect only when ResourceType is Spot. Values: SpotWithPriceLimit (set the maximum price for the spot instance) and SpotAsPriceGo (the system provides a price automatically, and the maximum value is the Pay-As-You-Go price). |
SpotPriceLimit | Float | No | Maximum price of an instance per hour. The value can contain at most three decimals. It takes effect when SpotStrategy is SpotWithPriceLimit. |
userData | Map | No | User data |
configs | Configs | No | Cluster configurations, such as the disk configuration of the instance. It is described by [Configs]. |
ReserveOnFail | Bool | No | When a task fails, related VMs are not released, and the fees for these resources are still charged until the user deletes the job. The default value is False. It is used only in a survey. |
Attributes of Parameters
The getter and setter methods are available to all attributes.
Attribute | Type | Required | Description |
command | Command | Yes | Command line parameter related to a user program. |
inputMappingConfig | InputMappingConfig | No | Configuration of an NFS mounting service. |
stdoutRedirectPath | String | Yes | OSS directory to which the standard output is redirected. |
stderrRedirectPath | String | Yes | OSS directory to which the standard error is redirected. |
Attributes of Command (the getter and setter methods are available to all attributes)
Attribute | Type | Required | Description |
commandLine | String | Yes | Specifies a command for executing a user program. |
PackagePath | String | Yes | OSS path in which a user program is located. |
EnvVars | Map | No | Environment variables to be configured when the user program starts running. |
Attributes of InputMappingConfig
The getter and setter methods are available to all attributes.
Attribute | Type | Required | Description |
lock | Boolean | No | Specifies whether the NFS mounting service supports the network file lock. If it is set to true, the network file lock is enabled to provide backend support for the file lock. If it is set to false, the network file lock is disabled and the file access performance can be improved. The file lock is supported only at the NFS client and will not be synchronized to the server. In some operating systems, the displayed file lock methods may fail. |
locale | String | No | All objects on OSS are named using UTF-8. This parameter determines the local character set used after mounting. Optional values include GBK, GB2312-80, BIG5, ANSI, EUC-JP, EUC-TW, EUC-KR, SHIFT-JIS, and KSC5601. |
Attributes of Notification
The getter and setter methods are available to all attributes.
Attribute | Type | Required | Description |
topic | Topic | No | Message topic |
Attributes of Topic
The getter and setter methods are available to all attributes.
Attribute | Type | Required | Description | |
endpoint | String | Yes | End point of the MNS region. The format is http://${your_user_id}.mns.${region}-internal.aliyuncs.com/. Use the intranet endpoint whenever possible. | |
name | String | Yes | Name of the topic | |
events |
| Yes | List of events. You must enter the job related events. |
Attributes of AppJobDescription
Attribute | Type | Required | Description |
appName | String | Yes | App name. The valid character set is [a-zA-Z0-9_-:]. The length cannot exceed 1024 characters. Two formats are available for the App name, and are used to access a public App and your private App, respectively. For example, to access the public App “App1”, set appName to Public:App1. “Public” can be excluded, that is, the App name changes to “:App1”. If App1 is your private App, directly set appName to App1. |
inputs | Map | Yes | Input parameters of the job. “Map” must be a value between 0 and 50. “Key” must be a parameter name defined in InputParameters of the App, and “Value” must be consistent with the type of the parameter in the App. |
outputs | Map | No | Output parameters of the job. “Map” must be a value between 0 and 50. “Key” must be a parameter name that is defined in OutputParameters of the App and LocalPath of which is not null, and “Value” must be an OSS path. Default value:{}. |
logging | AppJobDescription.Loggin | No | Log configuration. |
config | Map | No | Running configuration of the job. |
Attributes of AppJobDescription.Loggin
Parameter | Type | Required | Description |
stdoutPath | String | No | OSS directory to which the standard output is redirected. The length cannot exceed 1000 characters. Default value: null |
stderrPath | String | No | OSS directory to which the standard error is redirected. The length cannot exceed 1000 characters. Default value: null |
Response description
If creation succeeds, a CreateJobResponse instance is returned. You can use response.getJobId() to obtain the name of the created job.
Type | Description |
CreateJobResponse | Obtains the ID of the created job. |
The CreateJobResponse package name is com.aliyuncs.batchcompute.model.v20151111.Other responses listed in the following are under this package.
If creation fails, a ClientException is thrown.
Sample code for submitting a DAG job
The structure of JobDescription is complex. Therefore, we recommend using IntelliJ IDEA Community for development.
package com.aliyuncs.batchcompute.sample.v20151111;
import com.aliyuncs.batchcompute.main.v20151111.*;
import com.aliyuncs.batchcompute.model.v20151111.*;
import com.aliyuncs.batchcompute.pojo.v20151111.*;
import com.aliyuncs.exceptions.ClientException;
public class CreateJob {
public static void main(String[] args) {
BatchCompute client = new BatchComputeClient("cn-shenzhen", "your_access_id", "your_access_secret");
try {
JobDescription jobDescription = getJobDesc();
CreateJobResponse response = client.createJob(jobDescription);
String jobId = response.getJobId();
//Creation succeeded
System.out.println("Got job id:" + jobId);
} catch (ClientException e) {
e.printStackTrace();
//Creation failed
}
}
private static JobDescription getJobDesc() {
JobDescription desc = new JobDescription();
desc.setName("testJob");
desc.setPriority(1);
desc.setDescription("Demo");
desc.setType("DAG");
desc.setJobFailOnInstanceFail(true);
desc.setAutoRelease(false);
DAG dag = new DAG();
dag.addTask(getTaskDesc());
desc.setDag(dag);
Notification noti = new Notification();
Topic topic = new Topic();
topic.addEvent(Topic.ON_JOB_FAILED);
topic.addEvent(Topic.ON_JOB_FINISHED);
noti.setTopic(topic);
topic.setName("tp_n1");
topic.setEndpoint("xxxxx");
desc.setNotification(noti);
return desc;
}
private static TaskDescription getTaskDesc() {
TaskDescription task = new TaskDescription();
task.setClusterId("cls-xxxxxx");
task.setInstanceCount(1);
task.setMaxRetryCount(2);
task.setTaskName("task_1");
task.setTimeout(10000);
Parameters parameters = new Parameters();
Command cmd = new Command();
cmd.setCommandLine("python main.py");
cmd.setPackagePath("oss://my-bucket/test/worker.tar.gz");
parameters.setCommand(cmd);
parameters.setStderrRedirectPath("oss://my-bucket/test/logs/");
parameters.setStdoutRedirectPath("oss://my-bucket/test/logs/");
InputMappingConfig input = new InputMappingConfig();
input.setLocale("GBK");
input.setLock(true);
parameters.setInputMappingConfig(input);
task.setParameters(parameters);
task.addInputMapping("oss://my-bucket/disk1/", "/home/admin/disk1/");
task.addOutputMapping("/home/admin/disk2/", "oss://my-bucket/disk2/");
return task;
}
}Sample code for submitting an App job
package com.aliyuncs.batchcompute;
import com.aliyuncs.batchcompute.main.v20151111.BatchCompute;
import com.aliyuncs.batchcompute.main.v20151111.BatchComputeClient;
import com.aliyuncs.batchcompute.model.v20151111.CreateJobResponse;
import com.aliyuncs.batchcompute.pojo.v20151111.AppJobDescription;
import com.aliyuncs.batchcompute.pojo.v20151111.JobDescription;
import com.aliyuncs.exceptions.ClientException;
public class CreateAppJob {
public static void main(String[] args){
BatchCompute client = new BatchComputeClient("cn-shenzhen", "your_access_id", "your_access_secret");
try {
JobDescription jobDescription = getJobDesc();
CreateJobResponse response = client.createJob(jobDescription);
String jobId = response.getJobId();
//Creation succeeded
System.out.println("Got job id:" + jobId);
} catch (ClientException e) {
e.printStackTrace();
//Creation failed
}
}
private static JobDescription getJobDesc() {
JobDescription jobDesc = new JobDescription();
AppJobDescription app = new AppJobDescription();
app.setAppName("myApp");
app.addInputs("abc", "oss://bucket/inp");
app.addOutputs("out", "oss://bucket/out");
AppJobDescription.Logging logging = new AppJobDescription.Logging();
logging.setStderrPath("oss://bucket/key/logs");
logging.setStdoutPath("oss://bucket/key/logs");
app.setLogging(logging);
jobDesc.setName("java-sdk-test");
jobDesc.setDescription("java sdk test job");
jobDesc.setType("App");
jobDesc.setApp(app);
return jobDesc;
}
}