When MaxCompute runs a task — such as an SQL task — it creates an instance to represent that execution. Using PyODPS, you can list, stop, monitor, and inspect instances and their sub-tasks programmatically.
For background on the instance model, see Instance.
Available methods
| Method | Description |
|---|---|
list_instances() | Lists all instances in a project |
exist_instance() | Checks whether an instance exists |
get_instance() | Retrieves an instance by ID |
stop_instance() | Stops a running instance |
get_logview_address() | Returns the Logview URL of an instance |
status | Returns the current status of an instance |
is_terminated() | Returns True when execution is complete |
is_successful() | Returns True only when execution succeeds; returns False if the instance is running or fails |
wait_for_completion() | Blocks until the instance finishes (any outcome) |
wait_for_success() | Blocks until the instance finishes; raises an error if it fails |
get_task_names() | Lists the names of all sub-tasks in an instance |
get_task_result() | Returns the execution result of a named sub-task |
get_task_results() | Returns results for all sub-tasks as an OrderedDict |
get_task_progress() | Returns the current progress of a sub-task while it is running |
List and check instances
List all instance IDs in a project:
for instance in o.list_instances():
print(instance.id)Check whether a specific instance exists:
print(o.exist_instance('my_instance_id'))Stop an instance
Only instances in the Running state can be stopped. Calling stop_instance() on a non-running instance returns an error.
Alternatively, retrieve the instance first and call stop() directly:
o.get_instance('my_instance_id').stop()Get the Logview URL
Logview provides a detailed execution log for debugging and monitoring.
For SQL task instances, call get_logview_address() on the instance:
# Run an SQL statement and get the Logview URL immediately
instance = o.run_sql('desc pyodps_iris')
print(instance.get_logview_address())
# Or retrieve an existing instance by ID
instance = o.get_instance('my_instance_id')
print(instance.get_logview_address())For Machine Learning Platform for AI (PAI) instances, list the sub-tasks first, then get the Logview URL for each:
instance = o.run_xflow('AppendID', 'algo_public', {'inputTableName': 'input_table', 'outputTableName': 'output_table'})
for sub_inst_name, sub_inst in o.get_xflow_sub_instances(instance).items():
print('%s: %s' % (sub_inst_name, sub_inst.get_logview_address()))Check instance status
An instance can be in one of three states: Running, Suspended, or Terminated.
Get the current status of an instance:
instance = o.get_instance('my_instance_id')
print(instance.status) # Returns Status.TERMINATED
print(instance.status.value) # Returns TerminatedCheck whether execution has finished:
instance = o.get_instance('my_instance_id')
from odps.models import Instance
print(instance.status == Instance.Status.TERMINATED) # Returns True if completeWait for an instance to finish
Use wait_for_completion() to block until the instance reaches a terminal state, regardless of outcome. Use wait_for_success() when you want an error raised immediately if the instance fails.
| Method | Behavior on failure |
|---|---|
wait_for_completion() | Returns normally; check status afterward |
wait_for_success() | Raises an error |
Work with sub-tasks
A running instance may contain one or more sub-tasks. Use the following methods to inspect them.
List sub-task names
instance = o.get_instance('my_instance_id')
print(instance.get_task_names())Output:
['jdbc_sql_task']Get sub-task results
Get the name and result of a sub-task from an executed SQL statement:
instance = o.execute_sql('select * from pyodps_iris limit 1')
print(instance.get_task_names())Output:
['AnonymousSQLTask']Get the result of a specific sub-task:
print(instance.get_task_result('AnonymousSQLTask'))Output:
"sepallength","sepalwidth","petallength","petalwidth","name"
4.9,3.0,1.4,0.2,"Iris-setosa"Get results for all sub-tasks at once as an OrderedDict:
print(instance.get_task_results())Output:
OrderedDict([('AnonymousSQLTask', '"sepallength","sepalwidth","petallength","petalwidth","name"\n4.9,3.0,1.4,0.2,"Iris-setosa"\n')])Track sub-task progress
Poll progress while the instance is running:
import time
instance = o.get_instance('20160519101349613gzbzufck2')
while not instance.is_terminated():
for task_name in instance.get_task_names():
print(instance.id, instance.get_task_progress(task_name).get_stage_progress_formatted_string())
time.sleep(10)Output:
20160519101349613gzbzufck2 2016-05-19 18:14:03 M1_Stg1_job0:0/1/1[100%]