Note: Replace your-bucket in the examples with a real bucket.
Description
In this example, a job is started. The job includes a task. The task will start two instances that run concurrently in two VMs.
Programs run in the two VMs are both the python sum.py
command specified in the sum task. In the programs, the environment variable BATCH_COMPUTE_DAG_INSTANCE_ID is used to obtain InstanceId, so as to differentiate between the input data.
InstanceId increases from 0.
In each VM, after the task program processes ${InstanceId}-input.txt and writes the result into the file /home/outputs/${InstanceId}-output.txt, the system automatically uploads the file to the corresponding OSS path: oss://your-bucket/sum/outputs/.
When all programs in the two VMs finish running, both the task and the job end.
To download the example click here.
Upload data files to OSS
The data files are in the data directory: 0-input.txt and 1-input.txt.
Content of 0-input.txt:
1 20 45
Content of 1-input.txt:
5 85 103
Upload 0-input.txt and 1-input.txt to:
oss://your-bucket/sum/inputs/0-input.txt
oss://your-bucket/sum/inputs/1-input.txt
Run the following commands to upload the files:
cd data
bcs oss upload 0-input.txt oss://your-bucket/sum/inputs/
bcs oss upload 1-input.txt oss://your-bucket/sum/inputs/
# Check whether the uploading is successful.
bcs oss ls oss://your-bucket/sum/inputs/
Start a task
bcs sub --file job.cfg
Check the job execution result
The result data is stored in oss://your-bucket/sum/outputs/.
View the data by running the following commands:
bcs o ls oss://your-bucket/sum/outputs/
bcs o cat oss://your-bucket/sum/outputs/0-output.txt
bcs o cat oss://your-bucket/sum/outputs/1-output.txt