When you access OSS, a large number of access logs are generated. After you enable and configure logging for a bucket, OSS generates log objects in accordance with a predefined naming convention every hour. Then, the access logs are stored in a specified bucket as objects. You can use Alibaba Cloud Log Service or build a Spark cluster to analyze the logs.
Naming conventions of log objects
Log objects that are stored in OSS have the following naming conventions:
|TargetPrefix||The prefix of the log object name.|
|SourceBucket||The name of the source bucket for which logs are generated.|
|YYYY-mm-DD-HH-MM-SS||The time when the log object was created. The items of this field indicates the year, month, day, hour, minute, and second in sequence.|
|UniqueString||The string generated by OSS to uniquely identify the log object.|
Log formats and examples
- Log format
OSS logs include the information about the requester and accessed resources in the following format:
RemoteIP Reserved Reserved Time "RequestURL" HTTPStatus SentBytes RequestTime "Referer" "UserAgent" "HostName" "RequestID" "LoggingFlag" "RequesterAliyunID" "Operation" "BucketName" "ObjectName" ObjectSize ServerCostTime "ErrorCode RequestLength "UserID" DeltaDataSize "SyncRequest" "StorageClass" "TargetStorageClass" "TransmissionAccelerationAccessPoint" "AccessKeyID"
Field Example Description RemoteIP 192.168.0.1 The IP address of the requester. Reserved - The reserved field. Fixed value: -. Reserved - The reserved field. Fixed value: -. Time 03/Jan/2021:14:59:49 +0800 The time when OSS receives a request. RequestURL GET /example.jpg HTTP/1.0 The request URL that contains a query string.
OSS ignores the query string parameter that start with
x-. However, this parameter is recorded in logs. Therefore, you can tag a request by using a query string parameter that starts with
x-. Then, you can use this tag to query the log corresponding to the request.
HTTPStatus 200 The HTTP status code that OSS returns. SentBytes 999131 The downstream traffic generated by the request. Unit: bytes. RequestTime 127 The time consumed to complete the request. Unit: milliseconds. Referer http://www.aliyun.com/product/oss The HTTP header that identifies the address of the web page linked to the resource being requested. UserAgent curl/7.15.5 The user-agent header in an HTTP request. HostName examplebucket.oss-cn-hangzhou.aliyuncs.com The destination domain name to access in the request. RequestID 5FF16B65F05BC932307A3C3C The ID of the request. LoggingFlag true Indicates whether log storage is enabled. Valid values:
- true: indicates that log storage is enabled.
- false: indicates that log storage is disabled.
RequesterAliyunID 16571836914537**** The user ID of the requester. A value of - indicates anonymous access. Operation GetObject The type of the request. BucketName examplebucket The name of the destination bucket to which the requests are sent. ObjectName example.jpg The name of the requested object. ObjectSize 999131 The size of the requested object. Unit: bytes. ServerCostTime 88 The time that OSS takes to process the request. Unit: milliseconds. ErrorCode - The error code returned by OSS. A value of - indicates that no error code was returned. RequestLength 302 The length of the request. Unit: bytes. UserID 16571836914537**** The ID of the bucket owner. DeltaDataSize - The variation of the bucket size. A value of - indicates that this request does not involve write operations of objects. SyncRequest cdn Indicates whether the request is an Alibaba Cloud CDN (CDN) back-to-origin request. Valid values:
- cdn: indicates that the request is a CDN back-to-origin request.
- -: indicates that the request is not a CDN back-to-origin request.
StorageClass Standard The storage class of the uploaded object. Valid values:
- Cold Archive
- -: indicates that the storage class of the object is not obtained.
TargetStorageClass - The storage class that the object is converted to when a lifecycle rule is triggered or the CopyObject operation is called for an object. Valid values:
- Cold Archive
- -: indicates that the request does not involve operations to convert the storage class of the object.
TransmissionAccelerationAccessPoint - The accelerate endpoint used to access the destination bucket by using transfer acceleration. For example, if the requester accesses the destination bucket by using an accelerate endpoint in the China (Hangzhou) region, cn-hangzhou is used as the TransmissionAccelerationAccessPoint value.
A value of - indicates that no accelerate endpoint is used or the accelerate endpoint is in the same region as the destination bucket.
AccessKeyID LTAI4FrfJPUSoKm4JHb5**** The AccessKey ID of the requester. A value of - indicates anonymous requests.
- Sample log entry
192.168.0.1 - - [03/Jan/2021:14:59:49 +0800] "GET /example.jpg HTTP/1.0" 200 999131 127 "http://www.aliyun.com/product/oss" "curl/7.15.5" "examplebucket.oss-cn-hangzhou.aliyuncs.com" "5FF16B65F05BC932307A3C3C" "true" "16571836914537****" "GetObject" "examplebucket" "example.jpg" 999131 88 "-" 302 "16571836914537****" - "cdn" "standard" "-" "-" "LTAI4FrfJPUSoKm4JHb5****"
- The source bucket for which logs are generated and the destination bucket where the generated logs are stored can be the same bucket or different buckets. However, the destination bucket must belong to the same account in the same region as the source bucket.
- OSS generates bucket access logs on an hourly basis. However, requests in the previous hour may be recorded in the log generated for the subsequent hour.
- Before you disable logging, OSS keeps generating log objects. Delete log objects you
no longer need to reduce storage costs.
You can configure lifecycle rules to delete log objects at regular intervals. For more information, see Lifecycle rules.
- More fields are added to OSS logs in the future. We recommend that developers consider potential compatibility issues when they develop log processing tools.
|Console||A user-friendly and intuitive web application|
|ossutil||A high-performance command-line tool|
|Java SDK||SDK demos for a variety of programming languages|
Can I query interrupted requests in OSS logs?
Currently, OSS does not record interrupted requests in logs. If you initiate a request by using an SDK, you can identify the cause of the request interruption based on the returned value of the SDK.