When you access Object Storage Service (OSS), large numbers of access logs are generated. After you enable and configure logging for a bucket, OSS generates log objects every hour in accordance with predefined naming conventions and then stores the access logs as objects in a specified bucket. You can use Alibaba Cloud Log Service or build a Spark cluster to analyze the logs.

Note For more information about logging-related API operations, see the following topics:

Naming conventions of log objects

The following naming conventions apply to log objects:

<TargetPrefix><SourceBucket>YYYY-mm-DD-HH-MM-SS-UniqueString
Field Description
TargetPrefix The prefix of the log object name.
SourceBucket The name of the source bucket for which the access logs are generated.
YYYY-mm-DD-HH-MM-SS The time when the log object was created. The items of this field indicate the year, month, day, hour, minute, and second in sequence.
UniqueString The string generated by OSS to uniquely identify the log object.

Log formats and examples

  • Log formats

    OSS access logs include the information about the requester and accessed resources in the following format:

    RemoteIP Reserved Reserved Time "RequestURL" HTTPStatus SentBytes RequestTime "Referer" "UserAgent" "HostName" "RequestID" "LoggingFlag" "RequesterAliyunID" "Operation" "BucketName" "ObjectName" ObjectSize ServerCostTime "ErrorCode RequestLength "UserID" DeltaDataSize "SyncRequest" "StorageClass" "TargetStorageClass" "TransmissionAccelerationAccessPoint" "AccessKeyID"
    Field Example Description
    RemoteIP 192.168.0.1 The IP address of the requester.
    Reserved - The reserved field. Retain the default value -.
    Reserved - The reserved field. Retain the default value -.
    Time 03/Jan/2021:14:59:49 +0800 The time when OSS received an access request.
    RequestURL GET /example.jpg HTTP/1.0 The request URL that contains a query string.

    OSS ignores the query string parameter that starts with x-. However, this parameter is recorded in logs. Therefore, you can tag a request by using a query string parameter that starts with x-. Then, you can use this tag to query the log that corresponds to the request.

    HTTPStatus 200 The HTTP status code that OSS returns.
    SentBytes 999131 The downstream traffic generated by the request. Unit: bytes.
    RequestTime 127 The time consumed to complete the request. Unit: milliseconds.
    Referer http://www.aliyun.com/product/oss The Rerferer header in the HTTP request.
    UserAgent curl/7.15.5 The User-Agent header in the HTTP request.
    HostName examplebucket.oss-cn-hangzhou.aliyuncs.com The destination domain name that the request attempts to access.
    RequestID 5FF16B65F05BC932307A3C3C The ID of the request.
    LoggingFlag true Indicates whether logging is enabled. Valid values:
    • true: indicates that logging is enabled.
    • false: indicates that logging is disabled.
    RequesterAliyunID 16571836914537**** The user ID of the requester. The value of - indicates anonymous access.
    Operation GetObject The type of the request.
    BucketName examplebucket The name of the destination bucket that the request attempts to access.
    ObjectName example.jpg The name of the destination object that the request attempts to access.
    ObjectSize 999131 The size of the destination object. Unit: bytes.
    ServerCostTime 88 The time that OSS takes to process the request. Unit: milliseconds.
    ErrorCode - The error code returned by OSS. The value of - indicates that no error code is returned.
    RequestLength 302 The length of the request. Unit: bytes.
    UserID 16571836914537**** The ID of the bucket owner.
    DeltaDataSize - The size change of an object. The value of - indicates that this request does not involve write operations on objects.
    SyncRequest cdn Indicates whether the request is an Alibaba Cloud Content Delivery Network (CDN) back-to-origin request. Valid values:
    • cdn: indicates that the request is a CDN back-to-origin request.
    • -: indicates that the request is not a CDN back-to-origin request.
    StorageClass Standard The storage class of the destination object. Valid values:
    • Standard
    • IA
    • Archive
    • Cold Archive
    • -: indicates that the storage class of the object is not obtained.
    TargetStorageClass - Indicates whether the storage class of the object is converted based on a lifecycle rule or the CopyObject operation. Valid values:
    • Standard
    • IA
    • Archive
    • Cold Archive
    • -: indicates that the request does not involve operations to convert the storage class of the object.
    TransmissionAccelerationAccessPoint - The accelerate endpoint used when transfer acceleration is used to access the destination bucket. For example, if the requester accesses the destination bucket by using an accelerate endpoint in the China (Hangzhou) region, the value of TransmissionAccelerationAccessPoint is cn-hangzhou.

    The - value indicates that no accelerate endpoint is used or the accelerate endpoint is in the same region as the destination bucket.

    AccessKeyID LTAI4FrfJPUSoKm4JHb5**** The AccessKey ID of the requester. The - value indicates anonymous requests.
  • Sample logs
    192.168.0.1 - - [03/Jan/2021:14:59:49 +0800] "GET /example.jpg HTTP/1.0" 200 999131 127 "http://www.aliyun.com/product/oss" "curl/7.15.5" "examplebucket.oss-cn-hangzhou.aliyuncs.com" "5FF16B65F05BC932307A3C3C" "true" "16571836914537****" "GetObject" "examplebucket" "example.jpg" 999131 88 "-" 302 "16571836914537****" - "cdn" "standard" "-" "-" "LTAI4FrfJPUSoKm4JHb5****"

    After the log objects are stored in the specified bucket in OSS, you can use Alibaba Cloud Log Service to analyze the log objects. Before you analyze the log objects, you must import the log objects to Alibaba Cloud Log Service. For more information about how to import data, see Import data from OSS to Log Service. For more information about the analysis feature of Log Service, see Log analysis overview.

Usage notes

  • The source bucket for which logs are generated and the destination bucket in which the logs are stored can be the same bucket or different buckets. However, the destination bucket must belong to the same Alibaba Cloud account in the same region as the source bucket.
  • OSS generates access logs for a bucket on an hourly basis. However, requests during a specific hour may be recorded in the logs generated for the previous or the subsequent hour.
  • Before you disable logging, OSS keeps generating log objects. Delete log objects that you no longer need to reduce your storage costs.

    You can configure lifecycle rules to regularly delete log objects. For more information, see Lifecycle rules based on the last modified time.

  • More fields may be added to OSS logs in the future. Therefore, we recommend that developers consider potential compatibility issues when they develop log processing tools.

Implementation methods

Implementation method Description
Console A user-friendly and intuitive web application.
ossutil A high-performance command-line tool.
Java SDK SDK demos for a variety of programming languages.
Python SDK
PHP SDK
Go SDK
C SDK
.NET SDK
Node.js SDK
Ruby SDK

FAQ

Can I query interrupted requests in OSS access logs?

No, OSS does not record interrupted requests in access logs. If you send a request by using an SDK, you can identify the cause of the request interruption based on the returned value of the SDK.