This topic describes the log export tool that allows you to export Message Service (MNS) logs from Object Storage Service (OSS) to Log Service for query and analysis. This topic also describes how to install the tool on the Windows operating system, use the tool to export logs, and query logs in the Log Service console.

Description

The log export tool supports Python 2.6 and Python 2.7, and can run on the Windows and Linux operating systems.

  • Version: 1.0.0
  • Release date: April 20, 2016
  • New feature: Operations logs of MNS queues and topics can be exported from OSS to Log Service for query and analysis.

Prerequisites

Note For more information about the first three prerequisites, see Preparations.

Preparations

To use the log export tool, you must install the protobuf and simplejson compilers. To prepare the environment in Windows, perform the following steps:

  1. Install Python and pip
    1. Visit the official website of Python and download the Python 2.7 package. Install Python to the D:\Python27 directory.
      Note The D:\Python27 directory is used only for reference. You can specify the directory based on your business requirements.
    2. Right-click This PC, and choose Properties > Advanced System Settings > Environment Variables > Path. Add D:\Python27 to the path and click OK.
    3. Visit the official website of pip. Download and decompress the pip-8.1.2.tar.gz package.
    4. Open the Command Prompt in Windows. You can also open the Windows PowerShell ISE if you are using Windows 7 or later. Go to the directory to which the pip-8.1.2.tar.gz package is decompressed and run the following command:
      python setup.py install
    5. Right-click This PC, and choose Properties > Advanced System Settings > Environment Variables > Path. Add the Scripts directory of Python to the path and click OK. In this example, add D:\Python27\Scripts to the path.
  2. Install the protobuf and simplejson compilers
    1. Run the following command to install protobuf:
      pip install protobuf
    2. Run the following command to install Requests:
      pip install requests
    3. Run the following command to install simplejson:
      pip install simplejson

    If a timeout error or connection failure occurs, perform the following steps:

    1. Go to the C:\Users$username home directory, and create a folder named .pip and a file named pip.conf in the folder.
    2. Open the pip.conf file and enter the following content:
      [global]
      index-url = http://pypi.v2ex.com/simple
    3. Run the commands again to install the protobuf and simplejson compilers. We recommend that you add the following content to each of the commands: --default-timeout 100.

Export logs from OSS to Log Service

Download and decompress the package of the log export tool to the mnslog_export directory.

  • Run the following command to export logs:
    python2.7 mnslog_export.py --access_id=xxx --access_key=xxx --oss_host=xxx --log_bucket=xxx --sls_project=xxx --sls_logstore=xxx --sls_host=xxx --queue_name=xxx
  • Parameters
    Parameter Description
    access_id The AccessKey ID that is provided by Alibaba Cloud. To view the AccessKey ID, you can log on to the Alibaba Cloud Management Console and go to the AccessKey Management page.
    access_key The AccessKey secret that is provided by Alibaba Cloud. To view the AccessKey secret, you can log on to the Alibaba Cloud Management Console and go to the AccessKey Management page.
    oss_host The OSS endpoint of the region where the MNS logging bucket resides. To view the endpoint of the region, you can go to the Overview page of the bucket in the OSS console. For example, the OSS endpoint of the China (Hangzhou) region is oss-cn-hangzhou.aliyuncs.com.
    log_bucket The name of the OSS bucket that the MNS logs are stored. To view the name of the logging bucket, you can log on to the MNS console and go to the Logging Management page.
    start_time You can export the MNS logs that are generated in a specified period from OSS to Log Service. This parameter specifies the start time of the period. For example, 201503270102 indicates that the period starts from 01:02 on March 27, 2015.
    end_time You can export the MNS logs that are generated in a specified period from OSS to Log Service. This parameter specifies the end time of the period. For example, 201603270302 indicates that the period ends at 03:02 on March 27, 2016.
    sls_host The Log Service endpoint of the region where the MNS logging bucket resides. To view the endpoint of the region, you can go to the official website of Log Service. For example, the Log service endpoint of the China (Hangzhou) region is cn-hangzhou.sls.aliyun.com.
    sls_project The name of the Log Service project that you have created.
    sls_logstore The name of the Logstore that you have created.
    queue_name The name of the MNS queue whose logs you want to export from OSS to Log Service. You can specify a single queue or topic.
    topic_name The name of the MNS topic whose logs you want to export from OSS to Log Service. You can specify a single queue or topic.
  • Usage notes
    • You can export only the logs that were generated in the last seven days to Log Service. Otherwise, an error occurs.
    • After you run the mnslog_export.py command and upload MNS logs, we recommend that you wait for 60 seconds before you log on to the Log Service console. Otherwise, you may fail to query the newly uploaded logs.

Query logs

You can log on to the Log Service console to query the logs one minute after MNS logs are exported from OSS to Log Service.

  • Click the name of the Logstore.
  • On the details page of the Logstore, select a time range and click Search & Analyze. All logs collected within the time range are displayed.
  • Click a message ID to query all logs of the specified message.