5-minute quick start

Last Updated: Jun 27, 2017

Log Service is a platform service provided by Alibaba Cloud to handle massive log collection, storage, and query. You can use Log Service to collect logs from the service cluster. Log Service also supports real-time consumption, real-time query and shipping logs to OSS+Spark or MaxCompute (coming soon) for further analysis.

The following sample introduces how to collect text logs by using Logtail, preview logs, query the logs collected and ship logs to OSS.

Workflow

The basic workflow of using Log Service is as follows.

1

1. Preparations

1.1 Create an Alibaba Cloud account

For details, refer to FAQ about account management.

1.2 Activate Log Service

Use a registered Alibaba Cloud account to log on to the Log Service product page and click Get it Free.

1.3 Create an Access Key

Note: Logtail requires an access key to collect log data.

  1. On the Log Service console, move the cursor above your username in the upper-right corner and click accesskeys in the displayed menu.

  2. In the pop-up confirmation box, click Continue to manage AccessKey.

  3. On the Access Key Management page, click Create Access Key in the upper-right corner.

  4. In the pop-up box, click Agree and create.

2 Create project

2.1 Create a project

When you first log on to the Log Service console, the system will prompt you to create a project. To create projects on subsequent logins, click Create Project in the upper-right corner.

Note: For details about creating a project, refer to Create a project.

When creating a project, you must specify the Project Name and Region.

2.2 Create a Logstore

After creating a project, you will be prompted to create a Logstore. You can also go to the project and click Create in the upper-right corner.

Note: For details about creating a Logstore, refer to Create a Logstore.

When creating a Logstore, you must specify how you are going to use these logs.

3 Collect logs

Log Service supports various log sources and collection modes; for details, refer to Collection modes.

3.1 Install the Logtail client

  1. Download the installation package.

    Download the Logtail installation package onto the ECS instance. The download address of Windows installation package is http://logtail-release.oss-cn-hangzhou.aliyuncs.com/win/logtail_installer.zip.

    For your relevant installation method, please refer to the Install Logtail on Windows or Install Logtail on Linux.

  2. Install Logtail.

    Unzip the installation package into the current directory and enter the logtail_installer directory. Run cmd as the administrator and run the installation command .\logtail_installer.exe install cn_hangzhou.

    Note: You must run different installation commands according to the network environment and the region of Log Service. This quick start uses China East 1 (Hangzhou) and classic network as an example. For the installation commands of other regions, refer to Install Logtail on Windows.

3.2 Create a Logtail configuration

  1. Log on to the Log Service Console.

  2. Select the desired project, and click the project name or click Manage on the right.

  3. Select the desired Logstore and click Logtail Config Manage.

  4. Click Create in the upper-right corner.

  5. Select a data source.

    4

  6. Specify the collection mode.

    This example uses full mode. For details about Logtail configuration, refer to Use logtail to collect text files.

    5

  7. Apply the configuration to a machine group.

    Note: If no machine group is available, you must first create a machine group.

    6

Note:

  • It can take up to 3 minutes for the Logtail configuration to take effect.
  • If you need to collect IIS access logs, you must first refer to the IIS Log collection best practices to configure IIS.

Use API to write logs

Log Service provides the RESTful API to help write logs. You can use the API’s PostLogStoreLogs interface to write data. For complete API references, refer to API Reference.

Use SDKs to write logs

In addition to APIs, Log Service also provides SDKs in multiple languages (Java, .NET, PHP, and Python) to facilitate log writes. For a complete SDK reference, refer to SDK Reference.

4 Consume logs

Log service provides various ways to consume your collected logs, including previewing, querying, shipping logs to other products.

4.1 Preview logs

After you collect log data through Log Service, you can preview the collected logs. By specifying the Shard ID and time, you can preview the first 10 packets.

1

In the following example, the preview Shard ID is 0 and the time range is the first 10 packets from the previous 15 minutes.

preview

API log consumption

Similar to log writing, API also provides log consumption interfaces GetCursor and PullLogs. For a complete API reference, refer to API Reference.

SDK log consumption

In addition to APIs, Log Service also provides SDKs in multiple languages (Java, .NET, PHP, and Python) that allow you to easily consume logs. For a complete SDK reference, refer to Log Service SDKs.

4.2 Query logs

After you collect log data in Log Service, you can click Search in the Log Consumption Mode column to go to the query interface.

3

Queries are performed by the specified log topic (at present, only logs written using API have topics), keyword, or time. In the histogram, green indicates that the data are precise during this time period, and yellow indicates the data are imprecise. You can click on or drag the yellow portion to perform further queries. Imprecise log data will not be returned in “Match Logs”.

In the following example, the query conditions are blank and the time range is set to the previous 15 minutes.

rawlogs

Use APIs to query logs

Similar to log writing, API also provides the log query interfaces GetLogs and GetHistograms. For a complete API reference, refer to API Reference.

Use SDKs to query logs

In addition to APIs, Log Service also provides SDKs in multiple languages (Java, .NET, PHP, and Python) that allow for easy log querying. For a complete SDK reference, refer to Log Service SDKs.

4.3 Ship logs to OSS (with EMR computing)

  1. Prepare the OSS environment.

    In order to ship logs in Log Service to OSS, you must first prepare an OSS environment as explained in the following process.

    1. Activate the OSS service.

    2. Create an OSS bucket and perform the relevant authorization operations.

      Note: The OSS bucket should be in the same region as the Log Service Project. Data can only be shipped to OSS buckets in the same region as the Log Service project.

  2. Create OSS shipping rules.

    Log Service provides the function to ship log data to OSS. In the Logstore list, click OSS in the Log Consumption Mode column, and then click Enable to set the OSS shipping rules.

    2

    You must specify the OSS bucket to post to, the permission console role name (ARN), whether or not the data are compressed, and other attributes.

    7

  3. View OSS shipping tasks.

    In the OSS shipping task management on the console, you can view the shipping task statuses. After a log has been imported, you can view its data on the OSS console. In addition, the Ship logs to OSS explains how to use the imported data.

Thank you! We've received your feedback.