All Products
Search
Document Center

How to use SDK

Last Updated: May 11, 2018

1. Example

You must replace user information, such as image_id, access_key_id, and access_key_secret, and OSS-related directories in the codes.

  1. import time
  2. from batchcompute import Cient, ClientError
  3. from batchcompute import CN_SHENZHEN as REGION
  4. from batchcompute.resources import (
  5. JobDescription, TaskDescription, DAG,
  6. GroupDescription, ClusterDescription,
  7. )
  8. # some other codes here
  9. access_key_id = ... # your AccessKey id
  10. access_key_secret = ... # your AccessKey secret
  11. image_id = ... # the id of a image created before
  12. instance_type = ... # instance type
  13. client = Client(REGION, access_key_id, access_key_secret)
  14. try:
  15. # Create cluster.
  16. cluster_desc = ClusterDescription()
  17. group_desc = GroupDescription()
  18. group_desc.DesiredVMCount = 1
  19. group_desc.InstanceType = instance_type
  20. cluster_desc.add_group('group1', group_desc)
  21. cluster_desc.Name = "BatchcomputeCluster"
  22. cluster_desc.ImageId = image_id
  23. cluster_desc.Description = "Python SDK test"
  24. cluster_id = client.create_cluster(cluster_desc).Id
  25. # Create job.
  26. job_desc = JobDescription()
  27. echo_task = TaskDescription()
  28. # Create map task.
  29. echo_task.Parameters.Command.CommandLine = "echo Batchcompute service"
  30. echo_task.Parameters.Command.PackagePath = ""
  31. echo_task.Parameters.StdoutRedirectPath = "oss://xxx/xxx/" # Better replace this path
  32. echo_task.Parameters.StderrRedirectPath = "oss://xxx/xxx/" # Better replace this path
  33. echo_task.InstanceCount = 3
  34. echo_task.ClusterId = cluster_id
  35. # Create task dag.
  36. task_dag = DAG()
  37. task_dag.add_task(task_name="Echo", task=echo_task)
  38. # Create job description.
  39. job_desc.DAG = task_dag
  40. job_desc.Priority = 99 # 0-1000
  41. job_desc.Name = "PythonSDKDemo"
  42. job_desc.JobFailOnInstanceFail = True
  43. job_id = client.create_job(job_desc).Id
  44. # Wait for job finished.
  45. errs = client.poll(job_id)
  46. if errs: print ("Some errors occur: %s" % '\n'.join(errs))
  47. # Delete cluster
  48. client.delete_cluster(cluster_id)
  49. except ClientError, e:
  50. print (e.get_status_code(), e.get_code(), e.get_requestid(), e.get_msg())

2. Classes and constants

2.1 Interface class

The interface class provides Python implementation for all APIs of the BatchCompute service and some useful auxiliary interfaces.

SN Name Serializable Description
1. Client No Client class that interacts with the BatchCompute service

2.2 Description class

The description class provides parameter classes when resources are created, or returns classes of the service when resource status information is obtained.

SN Name Serializable Description
1. JobDescription Yes Class that describes a user job
2. DAG Yes Class that describes tasks of a job and dependencies between these tasks
3. TaskDescription Yes Class that describes a task
4. Parameters Yes Class that describes task running parameters
5. Command Yes Class that is used to configure the execution environment of the task command line
6. ClusterDescription Yes Class that describes a user cluster
7. GroupDescription Yes Class that describes instance configurations of a cluster
8. Job Yes Class that describes the current status of a specified job
9. Task Yes Class that describes the current status of a specified task of a job
10. Instance Yes Class that describes the current status of a specified job instance
11. Result Yes Class that describes the running result of a specified job instance
12. InstanceMetrics Yes Class that describes metrics of a specified job or task instance
13. TaskMetrics Yes Class that describes task metrics of a specified job
14. Cluster Yes Class that describes the status of a specified cluster
15. Group Yes Class that describes the status of the VM group in a specified cluster
16. Metrics Yes Class that describes task metrics of a specified cluster

About serialization

The description class is a serializable class. All serializable classes in the SDK are inherited from the internal class Jsonizable.The following describes the Jsonizable class and its subclasses.

Parameter description:

Jsonizable and its subclass objects can be initialized by using the dictionaries, Jsonizable objects, or JSON strings that describe the dictionary.Note that all invalid attribute descriptions in the dictionary or JSON strings are discarded when Jsonizable and its subclass objects are initialized.

Parameter Type Description
properties dict, str, Jsonizable object Attribute descriptions
  • Use a dictionary to initialize the Jsonizable class.

e.g.

  1. from batchcompute.resources import JobDescription
  2. # A dict object.
  3. properties = {
  4. "Name": "PythonSDKDemo",
  5. "Description": "Batchcompute"
  6. }
  7. jsonizable = JobDescription(properties)
  8. print (jsonizable.Name)
  9. print (jsonizable.Description)
  • Use a JSON string to initialize the Jsonizable class.

e.g.

  1. from batchcompute.resources import JobDescription
  2. # A string jsonized from a dict object.
  3. properties = '''{
  4. "Name": "PythonSDKDemo",
  5. "Description": "Batchcompute"
  6. }'''
  7. jsonizable = JobDescription(properties)
  8. print (jsonizable.Name)
  9. print (jsonizable.Description)
  • Use an object of the same class to initialize the Jsonizable class.

e.g.

  1. form batchcompute.resources import JobDescription
  2. # A JobDescription object.
  3. jsonizable1 = JobDescription()
  4. jsonizable1.Name = "PythonSDKDemo"
  5. jsonizable1.Description = "Batchcompute"
  6. jsonizable2 = JobDescription(jsonizable1)
  7. print(jsonizable2.Name)
  8. print(jsonizable2.Description)

Method description:

SN Method Description
1. update Receives a dictionary object, updates some attributes of the class, and discards invalid attributes.
2. detail Returns a dictionary containing class attributes that are not null.
3. load Receives a string. This string is a jsonized dictionary. All attributes of the class are updated, and invalid attributes are discarded.
4. dump Returns a string. The content of the string is a jsonized dictionary containing all class attributes that are not null.
5. __str__ Built-in function called by print. Within this function, the dump function is called.

About class attributes

All serializable classes contain various attributes. These attributes can be directly read by name. For example, you can use the following code to obtain a job ID:Attribute naming rules are consistent with the class naming rules of the Python specifications PEP8, and different from naming rules of class methods. Attribute names comply with the spelling rules of CamelCase.

  1. # job is a Job object.
  2. job = ...
  3. job_id = job.Id
  4. print (job_id)
  • In addition, you can obtain attributes by assigning dictionary values. For example:
  1. # job is a Job object.
  2. job = ...
  3. job_id = job["Id"]
  4. print (job_id)
  • For the classes like JobDescription, DAG, and TaskDescription, you can change the value of an attribute by assigning a value. For example:
  1. from batchcompute.resources import JobDescription
  2. job_desc = JobDescription()
  3. job_desc.Name = "PythonSDKDemo"
  • For the classes like JobDescription, DAG, and TaskDescription, you can assign a value for a class attribute using a dictionary. For example:
  1. from batchcompute.resources import JobDescription
  2. job_desc = JobDescription()
  3. job_desc["Id"] = "PythonSDKDemo"

2.3 Response class

SN Name Serializable Description
1. CreateResponse No Response class returned by the client after a resource is successfully created
2. GetResponse No Response class returned by the client after the resource status information is obtained
3. ActionResponse No Response class returned by the client after various operations are performed on resources
4. ListResponse No Response class returned by the client after resources are listed

About the response class

All the response classes, including CreateResponse, GetResponse, ActionRespnse, and ListResponse, are inherited from the internal class RawResponse.The following descriptions are applicable to all RawResponse subclasses.

Attribute description:

Attribute Type Description
RequestId str IDs of all client requests
StatusCode int Status codes of all client requests

e.g.

  1. ...
  2. response = client.create_job(job_desc)
  3. print (response.RequestId)
  4. print (response.StatusCode)

2.4 Error class

An exception is thrown when a parameter or request is invalid.

SN Name Serializable Description
1. ClientError No Error class
2. FieldError No Error class
3. ValidationError No Error class
4. JSONError No Error class
5. ConfigError No Error class

2.5 Constants

SN Name Serializable Description
1. CN_QINGDAO No Constant. China North 1 (Qingdao) endpoint of BatchCompute
2. CN_SHENZHEN No Constant. China South 1 (Shenzhen) endpoint of BatchCompute
3. CN_BEIJING No Constant. China North 2 (Beijing) endpoint of BatchCompute
4. CN_HANGZHOU No Constant. China East 1 (Hangzhou) endpoint of BatchCompute