When you access OSS, a large number of access logs are generated. After you enable and configure logging for a bucket, OSS generates log objects in accordance with a predefined naming convention. This way, hourly access logs are written to the specified bucket as objects.

You can use Alibaba Cloud Data Lake Analytics (DLA) or build a Spark cluster to analyze the access logs. You can also configure lifecycle rules for the bucket to convert the storage class of log objects to Archive for long-term archiving.

Note

Implementation modes

Implementation mode Description
Console A user-friendly and intuitive web application
ossutil A high-performance command-line tool
Java SDK SDK demos for various programming languages
Python SDK
PHP SDK
Go SDK
C SDK
.NET SDK
Node.js SDK
Ruby SDK

Naming conventions for access log objects

<TargetPrefix><SourceBucket>YYYY-mm-DD-HH-MM-SS-UniqueString
In the naming conventions:
  • TargetPrefix: the prefix of the access log object name. This field is user-defined and can be left unspecified.
  • YYYY-mm-DD-HH-MM-SS: the year, month, day, hour, minute, and second when the object is created. Note the number of digits.
  • UniqueString: the string (UUID) generated by OSS to identify the object.
An example of the name of an access log object: 
MyLog-oss-example2017-09-10-04-00-00-0000
In the preceding example,
  • MyLog- indicates the prefix specified by the user.
  • oss-example indicates the name of the source bucket.
  • 2017-09-10-04-00-00 indicates the time when the object is created.
  • 0000 indicates the unique string generated by OSS to identify the object.

Log object format

The following table describes the fields that compose a log object. In such an object, these fields are combined in order from left to right and are separated with spaces.

Parameter Example Description
Remote IP 119.xxx.xx.11 The IP address from which the request is initiated. The proxy or user firewall may block this field.
Reserved - The reserved field.
Reserved - The reserved field.
Time [02/May/2012:00:00:04 +0800] The time when OSS receives the request.
Request-URI "GET /aliyun-logo.png HTTP/1.1" The URI of the user request, including query-string.
HTTP Status 200 The HTTP status code returned by OSS.
SentBytes 5576 The amount of data in bytes downloaded from OSS by the user.
RequestTime (ms) 71 The length of time in milliseconds used to complete the request.
Referer http://www.aliyun.com/product/oss The HTTP Referer of the request.
User-Agent curl/7.15.5 The User-Agent field in the HTTP header.
HostName oss-example.oss-cn-hangzhou.aliyuncs.com The domain to be accessed.
Request ID 505B016950xxxxxx032593A4 The UUID used to identify the request.
LoggingFlag true Indicates whether logging is enabled.
Requester Aliyun ID 16571xxxxxx83691 The RAM user ID. This value is a hyphen (-) for access from anonymous users.
Operation GetObject The type of the request that is sent to perform operations on the object or bucket.
Bucket oss-example The name of the bucket to access.
Key /aliyun-logo.png The name of the object requested by the user.
ObjectSize 5576 The size of the object.
Server Cost Time (ms) 17 The length of time in milliseconds for OSS to process the request.
Error Code NoSuchBucket The error code returned by OSS.
Request Length 302 The length in bytes of the user request.
UserID 16571xxxxxx83691 The ID of the bucket owner.
Delta DataSize 280 The change to the bucket size. The value is a hyphen (-) if the bucket size does not change.
Sync Request - Indicates whether the request is a CDN back-to-origin request. The value is a hyphen (-) if the request is not a back-to-origin request.
StorageClass Standard The storage class of the current object. Valid values: Standard, IA, Archive, and -. The value is a hyphen (-) if the storage class information cannot be obtained or if the requested object is a bucket.
TargetStorageClass Standard The storage class converted to after a lifecycle rule is triggered or the CopyObject operation is called for an object. Valid values: Standard, IA, Archive, and -. The value is a hyphen (-) if the storage class information for the destination object cannot be obtained or the storage class fails to be converted by using a lifecycle rule or by calling the CopyObject operation.
oss_acc_src_oms_region us-east-1 The endpoint when transfer acceleration is enabled. For example, if the access request is from the US (Virginia) region, the value is us-east-1. If transfer acceleration is disabled, or the endpoint is in the same region as the bucket, the value is a hyphen (-).

Detail analysis

  • The source and destination buckets can be the same bucket, or different buckets within the same region owned by the same Alibaba Cloud account. You can also store the access logs of multiple source buckets in the same destination bucket. In this case, we recommend that you specify different TargetPrefix values for the log objects of different source buckets.
  • OSS generates bucket access logs on an hourly basis. However, requests in the previous hour may be recorded in the log generated for the subsequent hour.
  • Each time OSS generates a bucket access log, OSS performs a PUT operation and records the storage space that the operation occupies. However, OSS does not record the traffic generated by the PUT operation. After a log is generated, you can perform operations on the log as a common object.
  • OSS ignores all query-string parameters whose values contain the x- prefix. However, these parameters are recorded in access logs. To identify a special request from a large number of access logs, you can add a query-string parameter whose value contains the x- prefix to the URL of the request. Examples:

    http://oss-example.oss-cn-hangzhou.aliyuncs.com/aliyun-logo.png

    http://oss-example.oss-cn-hangzhou.aliyuncs.com/aliyun-logo.png?x-user=admin

    OSS returns the same result for both of the preceding requests. However, you can search for access logs by x-user=admin to locate the marked request.

  • A hyphen (-) may be contained in any field in OSS logs. The hyphen (-) indicates that data is unknown or the field is invalid for the current request.
  • More fields are added to OSS logs in the future. We recommend that developers consider potential compatibility issues when they develop log processing tools.