When you access OSS, a large number of access logs are generated. After you enable and configure logging for a bucket, OSS generates objects based on the predefined naming conventions. This way, hourly access logs are written to the specified bucket as objects.

You can use Alibaba Cloud Data Lake Analytics or build a Spark cluster to analyze access logs. You can also set lifecycle management rules for the bucket that stores access logs to convert the storage class of log objects to Archive for long-term archiving.

Note

Implementation modes

Implementation mode Description
Console User-friendly and intuitive web application
ossutil High-performance command-line tool
OSS SDK for Java SDK demos for various programming languages
OSS SDK for Python
OSS SDK for PHP
OSS SDK for Go
OSS SDK for C
OSS SDK for .NET
OSS SDK for Node.js
OSS SDK for Ruby

Naming conventions for access logs

<TargetPrefix><SourceBucket>YYYY-mm-DD-HH-MM-SS-UniqueString
In the naming conventions:
  • TargetPrefix: the name prefix of the access logs. This field is user-defined and can be left unspecified.
  • YYYY-mm-DD-HH-MM-SS: the year, month, day, hour, minute, and second when the object was created. Note the number of digits.
  • UniqueString: the string (UUID) generated by OSS to identify the object.
An example of the name of an access log: 
MyLog-oss-example2017-09-10-04-00-00-0000
In the preceding example,
  • MyLog- indicates the object prefix specified by the user.
  • oss-example indicates the name of the source bucket.
  • 2017-09-10-04-00-00 indicates the time when the object was created.
  • 0000 indicates the string generated by OSS to identify the object.

Log object formats

The following table describes the fields that compose a log object. In such an object, these fields are combined in order from left to right and are separated with spaces.

Parameter Example Description
Remote IP 119.xxx.xx.11 The IP address from which the request is initiated. The proxy or user firewall may block this field.
Reserved - The reserved field.
Reserved - The reserved field.
Time [02/May/2012:00:00:04 +0800] The time when OSS received the request.
Request-URI "GET /aliyun-logo.png HTTP/1.1" The URI of the user request, including query-string.
HTTP Status 200 The HTTP status code returned by OSS.
SentBytes 5576 The amount of data downloaded from OSS by the user, in bytes.
RequestTime (ms) 71 The length of time used to complete the request, in milliseconds.
Referer http://www.aliyun.com/product/oss The HTTP Referer of the request.
User-Agent curl/7.15.5 The User-Agent field in the HTTP header.
HostName oss-example.oss-cn-hangzhou.aliyuncs.com The domain name to access.
Request ID 505B016950xxxxxx032593A4 The UUID used to identify the request.
LoggingFlag true Indicates whether logging is enabled.
Requester Aliyun ID 16571xxxxxx83691 The RAM user ID. This value is a hyphen (-) for access from anonymous users.
Operation GetObject The type of the request that is sent to perform operations on the object or bucket.
Bucket oss-example The name of the bucket to access.
Key /aliyun-logo.png The name of the object requested by the user.
ObjectSize 5576 The size of the object.
Server Cost Time (ms) 17 The length of time for the OSS server to process this request, in milliseconds.
Error Code NoSuchBucket The error code returned by OSS.
Request Length 302 The length of the user request, in bytes.
UserID 16571xxxxxx83691 The ID of the bucket owner.
Delta DataSize 280 The change to the bucket size. The value is a hyphen (-) if there are no changes to the bucket size.
Sync Request - Indicates whether the request is a CDN back-to-origin request. The value is a hyphen (-) if the request is not a back-to-origin request.
StorageClass Standard The storage class of the current object. Valid values: Standard/IA/Archive/-. The value is a hyphen (-) if the storage class information cannot be obtained or the target object is a bucket.
TargetStorageClass Standard The destination storage class to which the object is converted after a lifecycle rule is triggered or CopyObject is called for the object. Valid values: Standard/IA/Archive/-. The value is a hyphen (-) if the storage class information about the destination object cannot be obtained or the storage class fails to be converted by using a lifecycle rule or by calling CopyObject.

Detail analysis

  • The source and destination buckets can be the same bucket, or different buckets within the same region owned by the same Alibaba Cloud account. You can also store the access logs of multiple source buckets in the same destination bucket. In this case, we recommend that you specify different TargetPrefix values for the log objects of different source buckets.
  • OSS generates an object that stores bucket access logs on an hourly basis. However, requests in the previous hour may be recorded in the object generated for subsequent hour.
  • Each time OSS generates an access log, OSS performs a PUT operation and records the storage space that the operation occupies. However, OSS does not record the traffic generated by the PUT operation. After a log object is generated, you can perform operations on it as a common object.
  • OSS ignores all query-string parameters whose values contain the x- prefix. However, these parameters are recorded in access logs. To identify a special request from a large number of access logs, you can add a query-string parameter whose value contains the x- prefix to the URL of the request. Examples:

    http://oss-example.oss-cn-hangzhou.aliyuncs.com/aliyun-logo.png

    http://oss-example.oss-cn-hangzhou.aliyuncs.com/aliyun-logo.png?x-user=admin

    OSS returns the same result for the preceding two requests. However, you can search for access logs by x-user=admin to locate the marked request.

  • A hyphen (-) may be contained in any field in OSS logs. The hyphen (-) indicates that data is unknown or the field is invalid for the current request.
  • More fields will be added to the end of OSS logs in the future as needed. We recommend that developers consider potential compatibility issues when they develop log processing tools.