The log export tool allows you to export MNS logs that are stored in Object Storage Service (OSS) to Log Service for query and analysis. This topic describes how to install the tool on the Windows operating system, use the tool to export logs, and query logs in the Log Service console.

Description

The log export tool supports Python 2.6 and 2.7, and can run on the Windows and Linux operating systems.

  • Version: 1.0.0
  • Updated on April 20, 2016. Click here to download
  • Supported feature

    Supports exporting operations logs of MNS queues and topics from OSS to Log Service for query and analysis.

Prerequisites

Note You must meet the following prerequisites. For more information about the first three prerequisites, see Preparations.

Preparations

To use the log export tool, you must install the protobuf and simplejson compilers. The following section describes how to prepare the environment.

  1. Install Python and pip

    1. Visit the official Python website and download the Python 2.7 package. Install Python to the D:\Python27 directory.
      Note D:\Python27 is used as an example. You can specify the directory based on your needs.
    2. Right-click This PC, and choose Properties > Advanced System Settings > Environment Variables > Path. Add D:\Python27 to the path and click OK.
    3. Visit the official pip website. Download and decompress the pip-8.1.2.tar.gz package.
    4. Go to the Windows command-line tool cmd. You can also use the Windows PowerShell ISE if you are using Windows 7 or a later version. Go to the directory to which pip is decompressed, and run the following command:

      python setup.py install

    5. Right-click This PC, and choose Properties > Advanced System Settings > Environment Variables > Path. Add D:\Python27\Scripts to the path and click OK.
  2. Install the protobuf and simplejson compilers

    1. Run the following command to install protobuf:

      pip install protobuf

    2. Run the following command to install Requests:

      pip install requests

    3. Run the following command to install simplejson:

      pip install simplejson

    If an error such as time-out or connection failure occurs, follow these steps to solve the problem:

    1. Go to the C:\Users$username home directory, create a .pip folder, and create a pip.conf file in the folder.
    2. Open the pip.conf file and enter the following content:
      [global]
      index-url = http://pypi.v2ex.com/simple
    3. Run the commands again to install the protobuf and simplejson compilers. We recommend that you add the following content to each of the commands: --default-timeout 100.

Export logs

Download the package of the log export tool and decompress it to the mnslog_export directory.

  • Command syntax

    python2.7 mnslog_export.py --access_id=xxx --access_key=xxx --oss_host=xxx --log_bucket=xxx --sls_project=xxx --sls_logstore=xxx --sls_host=xxx --queue_name=xxx

  • Parameters
    Parameter Description
    access_id The AccessKey ID provided to you by Alibaba Cloud. You can log on to the Alibaba Cloud console and go to the Security Management page to view the AccessKey ID.
    access_key The AccessKey secret provided to you by Alibaba Cloud. You can log on to the Alibaba Cloud console and go to the Security Management page to view the AccessKey secret.
    oss_host The OSS endpoint of the region in which the logging bucket resides. You can go to the bucket overview page to view the endpoint. For example, the OSS endpoint in China (Hangzhou) is oss-cn-hangzhou.aliyuncs.com.
    log_bucket The name of the logging bucket in which MNS logs are stored. You can log on to the MNS console and go to the Logging Management page to view the name of the logging bucket.
    start_time You can export the MNS logs that are generated in the specified time range from OSS to Log Service. This parameter specifies the beginning of the time range. For example, 201503270102 indicates that the time range starts at 01:02 on March 27, 2015.
    end_time You can export the MNS logs that are generated in the specified time range from OSS to Log Service. This parameter specifies the end of the time range. For example, 201603270302 indicates that the time range ends at 03:02 on March 27, 2016. Note: The logs that are generated at the specified end time point are not exported.
    sls_host The endpoint of Log Service. You can visit the official website of Log Service to view the endpoint. For example, the Log Service endpoint in China (Hangzhou) is cn-hangzhou.sls.aliyun.com.
    sls_project The name of the Log Service project that you have created.
    sls_logstore The name of the Logstore that you have created.
    queue_name The name of the MNS queue whose logs are to be imported. You can only specify a single queue or topic.
    topic_name The name of the MNS topic whose logs are to be imported. You can only specify a single queue or topic.
Notice
  • You can only export the logs that are generated in the last seven days to Log Service. Otherwise, an error occurs
  • After you run the mnslog_export.py command to upload MNS logs to Log Service, we recommend that you wait 60 seconds before logging on to the Log Service console to query logs. Otherwise, you may fail to query newly uploaded logs.

Query logs

After one minute has elapsed since MNS logs are exported from OSS to Log Service, you can log on to the Log Service console to query logs.

  • Click Search&Analysis next to a Logstore to go to the log query page.
  • Select a time range and click Search&Analytics. All logs within the time range are displayed.
  • Click a message ID to query all logs that contain the message ID.