When you access OSS, a large number of access logs are generated. After you enable and configure logging for a bucket, OSS generates log objects in accordance with a predefined naming convention. This way, hourly access logs are written to the specified bucket as objects.
You can use Alibaba Cloud Data Lake Analytics (DLA) or build a Spark cluster to analyze the access logs. You can also configure lifecycle rules for the bucket to convert the storage class of log objects to Archive for long-term archiving.
- For more information about logging-related API operations, see the following topics:
- Enable logging: PutBucketLogging
- Delete logging configurations: DeleteBucketLogging
- View logging configurations: GetBucketLogging
- If you use logging, you are charged for Log Service. For more information, see Pay-as-you-go.
Implementation modes
Implementation mode | Description |
---|---|
Console | A user-friendly and intuitive web application |
ossutil | A high-performance command-line tool |
Java SDK | SDK demos for various programming languages |
Python SDK | |
PHP SDK | |
Go SDK | |
C SDK | |
.NET SDK | |
Node.js SDK | |
Ruby SDK |
Naming conventions for access log objects
<TargetPrefix><SourceBucket>YYYY-mm-DD-HH-MM-SS-UniqueString
- TargetPrefix: the prefix of the access log object name. This field is user-defined and can be left unspecified.
- YYYY-mm-DD-HH-MM-SS: the year, month, day, hour, minute, and second when the object is created. Note the number of digits.
- UniqueString: the string (UUID) generated by OSS to identify the object.
MyLog-oss-example2017-09-10-04-00-00-0000
MyLog-
indicates the prefix specified by the user.oss-example
indicates the name of the source bucket.2017-09-10-04-00-00
indicates the time when the object is created.0000
indicates the unique string generated by OSS to identify the object.
Log object format
The following table describes the fields that compose a log object. In such an object, these fields are combined in order from left to right and are separated with spaces.
Parameter | Example | Description |
---|---|---|
Remote IP | 119.xxx.xx.11 | The IP address from which the request is initiated. The proxy or user firewall may block this field. |
Reserved | - | The reserved field. |
Reserved | - | The reserved field. |
Time | [02/May/2012:00:00:04 +0800] | The time when OSS receives the request. |
Request-URI | "GET /aliyun-logo.png HTTP/1.1" | The URI of the user request, including query-string. |
HTTP Status | 200 | The HTTP status code returned by OSS. |
SentBytes | 5576 | The amount of data in bytes downloaded from OSS by the user. |
RequestTime (ms) | 71 | The length of time in milliseconds used to complete the request. |
Referer | http://www.aliyun.com/product/oss |
The HTTP Referer of the request. |
User-Agent | curl/7.15.5 | The User-Agent field in the HTTP header. |
HostName | oss-example.oss-cn-hangzhou.aliyuncs.com | The domain to be accessed. |
Request ID | 505B016950xxxxxx032593A4 | The UUID used to identify the request. |
LoggingFlag | true | Indicates whether logging is enabled. |
Requester Aliyun ID | 16571xxxxxx83691 | The RAM user ID. This value is a hyphen (-) for access from anonymous users. |
Operation | GetObject | The type of the request that is sent to perform operations on the object or bucket. |
Bucket | oss-example | The name of the bucket to access. |
Key | /aliyun-logo.png | The name of the object requested by the user. |
ObjectSize | 5576 | The size of the object. |
Server Cost Time (ms) | 17 | The length of time in milliseconds for OSS to process the request. |
Error Code | NoSuchBucket | The error code returned by OSS. |
Request Length | 302 | The length in bytes of the user request. |
UserID | 16571xxxxxx83691 | The ID of the bucket owner. |
Delta DataSize | 280 | The change to the bucket size. The value is a hyphen (- ) if the bucket size does not change.
|
Sync Request | - | Indicates whether the request is a CDN back-to-origin request. The value is a hyphen
(- ) if the request is not a back-to-origin request.
|
StorageClass | Standard | The storage class of the current object. Valid values: Standard , IA , Archive , and - . The value is a hyphen (- ) if the storage class information cannot be obtained or if the requested object is
a bucket.
|
TargetStorageClass | Standard | The storage class converted to after a lifecycle rule is triggered or the CopyObject
operation is called for an object. Valid values: Standard , IA , Archive , and - . The value is a hyphen (- ) if the storage class information for the destination object cannot be obtained or
the storage class fails to be converted by using a lifecycle rule or by calling the
CopyObject operation.
|
oss_acc_src_oms_region | us-east-1 | The endpoint when transfer acceleration is enabled. For example, if the access request
is from the US (Virginia) region, the value is us-east-1 . If transfer acceleration is disabled, or the endpoint is in the same region as the
bucket, the value is a hyphen (- ).
|
Detail analysis
- The source and destination buckets can be the same bucket, or different buckets within the same region owned by the same Alibaba Cloud account. You can also store the access logs of multiple source buckets in the same destination bucket. In this case, we recommend that you specify different TargetPrefix values for the log objects of different source buckets.
- OSS generates bucket access logs on an hourly basis. However, requests in the previous hour may be recorded in the log generated for the subsequent hour.
- Each time OSS generates a bucket access log, OSS performs a PUT operation and records the storage space that the operation occupies. However, OSS does not record the traffic generated by the PUT operation. After a log is generated, you can perform operations on the log as a common object.
- OSS ignores all query-string parameters whose values contain the
x-
prefix. However, these parameters are recorded in access logs. To identify a special request from a large number of access logs, you can add a query-string parameter whose value contains thex-
prefix to the URL of the request. Examples:http://oss-example.oss-cn-hangzhou.aliyuncs.com/aliyun-logo.png
http://oss-example.oss-cn-hangzhou.aliyuncs.com/aliyun-logo.png?x-user=admin
OSS returns the same result for both of the preceding requests. However, you can search for access logs by
x-user=admin
to locate the marked request. - A hyphen (
-
) may be contained in any field in OSS logs. The hyphen (-) indicates that data is unknown or the field is invalid for the current request. - More fields are added to OSS logs in the future. We recommend that developers consider potential compatibility issues when they develop log processing tools.