Quick start

Last Updated: Jul 06, 2017

Follow the steps below to start using the Log Service Python SDK quickly.

Create an Alibaba Cloud account

An Alibaba Cloud account is required for accessing the Log Service. If you do not have an Alibaba Cloud account, follow the steps below to create one.

  1. Access the Alibaba Cloud website and click Registe.
  2. Follow the on-screen prompts to complete registration and perform real-name authentication.

To better use Alibaba Cloud services, we suggest you complete real-name authentication as soon as possible; otherwise, some Alibaba Cloud services are not available for use.

Obtain an Alibaba Cloud access key

Before using the SDK, apply for an Alibaba Cloud access key.

  1. Log on to the Alibaba Cloud Console.
  2. Go to the Alibaba Cloud key management page.
  3. Select an SDK-specific access key. If you do not have any, create one and ensure that the access key is “active”.

This access key will be used in the following steps. It must be kept confidential. You can refer to SDK configuration to learn more about how to use the access key in the SDK.

Create a Log Service project and LogStore

Currently, the SDK does not support project and LogStore management. You need to create a LogStore on the console before using the SDK.

  1. Log on to the Alibaba Cloud Console.
  2. Click “Log Service” to go to the project management page.
  3. Click “Create Project”. The “Create Project” dialog box is displayed.
  4. Follow the on-screen prompts to specify the project name, notes, and region. Then confirm the creation of the project. The “Project Management” page shows the project after it is successfully created.
  5. Click “Manage” corresponding to the new project. Go to the management page for this project and click the “LogStore Management” tab.
  6. Click “Create LogStore”. The “Create LogStore” dialog box is displayed.
  7. Follow the on-screen prompts to specify the LogStore name and log consumption method. Then confirm the creation of the LogStore. The “LogStore Management” tab shows the LogStore after it is successfully created.
  1. Ensure that the same Alibaba Cloud account is used to obtain the access key and create the project and LogStore.
    1. For details about Log Service projects and LogStores, refer to the Log Service core concepts.
  1. A project name must be globally unique in the Log Service, and a LogStore name must be unique under the corresponding project.
  2. The region of a project is unchangeable once the project is created. Project migration between different regions of Alibaba Cloud is not supported currently.

Install the Python environment

The Python SDK is a pure Python library and supports all Python operating systems, including Linux, Mac OS X, and Windows. Install Python as follows:

  1. Download and install the latest Python 2 installation package.

    Currently, the Python SDK supports the Python 2.6 and 2.7 environments. You can run python -V to query the current Python version.

  2. Download and install the Python package management tool pip.

    After PIP is installed, run pip -V to check whether the installation is successful and query the PIP version.

Install the dependent libraries of the Python SDK

The Python SDK depends on a set of third-party Python libraries. Before using this SDK, install the following dependent libraries:

  • GoogleProtocol Buffer: The Python SDK depends on the Protocol Buffer protocol when writing logs to the Log Service. The installation is as follows:

    1. pip install protobuf

    PIP needs to connect to the Python Package Index website to install Protobuf. Ensure that you can access this website on your machine. If installation fails due to network problems, you can manually download and install Protobuf from Protocol Buffer’s GitHub website. (For details about the installation process, refer to the Readme document contained in the downloaded package.)

  • Python-Requests: The Python SDK depends on the Python-Requests class for HTTP communication. The installation process is as follows:

    1. pip install requests
  • SimpleJSON: The Python SDK depends on SimpleJSON to process the JSON results returned by APIs. The installation process is as follows:

    1. pip install simplejson

Install the Python SDK

Download the Python SDK after you install the Python environment. The steps are as follows:

  1. Download the latest Python SDK package from GitHub.
  2. Decompress the downloaded package to the specified directory.
  3. Run the following command in the preceding directory to install the Python SDK:
    1. python setup.py install

Start a Python program

Now you can start using the Python SDK. To interact with the Log Service and obtain the relevant output, run the following sample code in a text editor or Python IDE:

  1. #!/usr/bin/env python
  2. #encoding: utf-8
  3. import time
  4. from aliyun.log.logitem import LogItem
  5. from aliyun.log.logclient import LogClient
  6. from aliyun.log.getlogsrequest import GetLogsRequest
  7. from aliyun.log.putlogsrequest import PutLogsRequest
  8. from aliyun.log.listtopicsrequest import ListTopicsRequest
  9. from aliyun.log.listlogstoresrequest import ListLogstoresRequest
  10. from aliyun.log.gethistogramsrequest import GetHistogramsRequest
  11. def main():
  12. endpoint = 'cn-hangzhou.sls.aliyuncs.com' # Select the endpoint that matches the region of the project created above
  13. accessKeyId = 'your_access_key_id' # Use your Alibaba Cloud access key ID
  14. accessKey = 'your_access_key' # Use your Alibaba Cloud access key secret
  15. project = 'your_project' # Name of the project created above
  16. logstore = 'your_logstore' # Name of the LogStore created above
  17. # Construct a client
  18. client = LogClient(endpoint, accessKeyId, accessKey)
  19. # List all LogStores
  20. req1 = ListLogstoresRequest(project)
  21. res1 = client.list_logstores(req1)
  22. res1.log_print()
  23. topic = ""
  24. source = ""
  25. # Send 10 packets consecutively, with each packet containing 10 logs
  26. for i in range(10) :
  27. logitemList = [] # LogItem list
  28. for j in range(10):
  29. contents = [('index', str(i * 10 + j))]
  30. logItem = LogItem()
  31. logItem.set_time(int(time.time()))
  32. logItem.set_contents(contents)
  33. logitemList.append(logItem)
  34. req2 = PutLogsRequest(project, logstore, topic, source, logitemList)
  35. res2 = client.put_logs(req2)
  36. res2.log_print()
  37. # List all shards and read the data written during the past minute
  38. listShardRes = client.list_shards(project,logstore);
  39. for shard in listShardRes.get_shards_info():
  40. shard_id = shard["shardID"];
  41. start_time = (int)(time.time() - 60)
  42. end_time = start_time + 60
  43. res = client.get_cursor(project, logstore, shard_id, start_time)
  44. res.log_print()
  45. start_cursor = res.get_cursor()
  46. res = client.get_cursor(project, logstore, shard_id, end_time)
  47. end_cursor = res.get_cursor()
  48. while True :
  49. loggroup_count = 100 # Read 100 packets each time
  50. res = client.pull_logs(project, logstore, shard_id, start_cursor, loggroup_count, end_cursor)
  51. res.log_print()
  52. next_cursor = res.get_next_cursor()
  53. if next_cursor == start_cursor :
  54. break
  55. start_cursor = next_cursor
  56. # Important: The following interface can be used to query data only when the indexing function is enabled.
  57. time.sleep(60)
  58. topic = ""
  59. query = "index"
  60. From = int(time.time()) - 600
  61. To = int(time.time())
  62. res3 = None
  63. # Query the number of logs that match the query criteria during the past 10 minutes. Retry if not all execution results are correct.
  64. while (res3 is None) or (not res3.is_completed()):
  65. req3 = GetHistogramsRequest(project, logstore, From, To, topic, query)
  66. res3 = client.get_histograms(req3)
  67. res3.log_print()
  68. # Obtain the number of logs that match the query criteria
  69. total_log_count = res3.get_total_count()
  70. log_line = 10
  71. # Read 10 logs each time until all log data is queried. Retry three times if not all query results are accurate during each query operation.
  72. for offset in range(0, total_log_count, log_line) :
  73. res4 = None
  74. for retry_time in range(0, 3) :
  75. req4 = GetLogsRequest(project, logstore, From, To, topic, query, log_line, offset, False)
  76. res4 = client.get_logs(req4)
  77. if res4 != None and res4.is_completed():
  78. break
  79. time.sleep(1)
  80. if res4 != None:
  81. res4.log_print()
  82. listShardRes = client.list_shards(project,logstore);
  83. shard = listShardRes.get_shards_info()[0]
  84. #Split shards
  85. if shard["status"] == "readwrite":
  86. shard_id = shard["shardID"]
  87. inclusiveBeginKey = shard["inclusiveBeginKey"]
  88. midKey = inclusiveBeginKey[:-1]+str(((int)(inclusiveBeginKey[-1:]))+1)
  89. client.split_shard(project,logstore,shard_id,midKey)
  90. #Merge shards
  91. shard = listShardRes.get_shards_info()[1]
  92. if shard["status"] == "readwrite":
  93. shard_id = shard["shardID"]
  94. client.merge_shard(project,logstore,shard_id)
  95. #Delete shards
  96. shard = listShardRes.get_shards_info()[-1]
  97. if shard["status"] == "readonly":
  98. shard_id = shard["shardID"]
  99. client.delete_shard(project,logstore,shard_id)
  100. if __name__=='__main__':
  101. main()
Thank you! We've received your feedback.