Alibaba Cloud CDN (CDN) supports the log storage feature, which is integrated with Object Storage Service (OSS) and Data Lake Analytics (DLA). After the log storage feature is enabled, CDN automatically stores logs in a specified OSS bucket. Logs can be persisted in OSS buckets. You can use DLA to analyze the logs. This topic describes how to enable log storage.

Prerequisites

Before you enable log storage, make sure that you have completed the following operations:

  • To enable OSS, go to the product page of OSS.
  • To enable DLA, go to the product page of DLA.
  • Log storage uses DLA to deliver logs and uses OSS to store logs. To ensure that log storage works as expected, you must ensure that DLA and OSS in the corresponding region are enabled.

Background information

When you enable log storage, the system automatically creates the service-link role (SLR) AliyunServiceRoleForCDNLogDelivery. The SLR is granted permissions to access OSS and DLA. This allows CDN to automatically store logs in OSS buckets. For more information about the SLR, see Manage the SLR for log storage.

Billing rules

After log storage is enabled, you are charged for both OSS and DLA.
  • For more information about the billing rules of OSS, see OSS Pricing.
  • For more information about the billing rules of DLA, see Billing methods.
    Notice After log storage is enabled, logs are delivered by DLA. You are charged for DLA even if you do not use DLA to analyze data.

Procedure

Notice
  • Logs are collected from DCDN nodes and then directly delivered to the OSS bucket. The integrity of log data is not guaranteed. If you want to acquire complete log data, we recommend that you go to the Log Download tab to download logs. Make sure that the fields that you specify are the same as those in the logs.
  • By default, Resource Access Management (RAM) users are not allowed to enable or manage log storage. You must first grant the RAM users the required permissions. For more information, see Grant a RAM user permissions on log storage.
  • If you want to grant a RAM user permissions to manage log storage, make sure that the authorized scope is set to Alibaba Cloud Account. For more information, see Step 2: Grant permissions to the RAM user.
  • If you want to manage log storage as a RAM user, make sure that the authorized scope of the RAM user is set to Alibaba Cloud Account . Otherwise, log storage may be disabled for some domain names.
  1. Log on to the Alibaba Cloud CDN console.
  2. In the left-side navigation pane, choose Logs > Offline Logs.
  3. Click the Log Storage tab.
  4. Click Activate Now.
  5. Set Storage Location.
    1. In the Activate Log Storage dialog box, set Storage Location.
      Note
      • An OSS bucket is created in the specified region. Logs of Alibaba Cloud CDN are stored in this OSS bucket.
      • The region cannot be changed.
      • If log storage is already enabled in Dynamic Route for CDN (DCDN) with a region specified, you cannot specify a region when you enable log storage in Alibaba Cloud CDN. By default, the region for the log storage feature in Alibaba Cloud CDN and DCDN is the same.
      Service country and region Storage country and region Description
      Mainland China
      • China (Shanghai)
      • China (Beijing)
      • China (Hangzhou)
      Three storage regions are supported: China (Shanghai), China (Beijing), and China (Hangzhou). Select a region based on your business requirements.
      Other countries and regions Singapore (Singapore) Only this region is supported.
      India India (Mumbai) Only this region is supported.
      Europe Germany (Frankfurt) Only this region is supported.
      US US (Silicon Valley) Only this region is supported.
    2. Click Activate Data Lake Analytics and go to the next step .
  6. Set Delivery Rule.
    1. In the Activate Log Storage dialog box, set Log Fields , and select the accelerated domain names for which you want to enable log storage.
      Note You can view and select up to 500 domain names in the console. If you want to enable log storage for more domain names, submit a ticket.
      You can specify one or more log fields. The log fields cannot be modified after they are specified. The following table describes the supported fields.
      Field Description Example
      contentType Indicates a file type. text/html
      domain Indicates an accelerated domain name. www.aliyun.com
      hitInfo Indicates cache hit information. hit
      http2 Indicates the HTTP/2 protocol. HTTP2
      httpCode Indicates HTTP status codes. 504 , 404 , 302 , 200
      method Indicates request methods. Valid values: GET and POST. GET or POST
      refer Indicates the referer header in HTTP requests. "-"
      remoteIP Indicates the IP address of the client that initiated the request. 192.168.15.75
      reqSize Indicates the size of the request. Unit: bytes. 129
      respSize Indicates the size of the response. Unit: bytes. 129
      rt Indicates the response time. Unit: milliseconds. 1
      schema Indicates the protocol over which the request was transmitted. HTTP and HTTPS are supported. HTTP or HTTPS
      traceID Indicates the unique ID of the request. d35ba34115550716522547264e
      ua Indicates the information about the proxy of the client. Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36
      unixtime Indicates the timestamp of the request. Unit: seconds. 1607340145
      urlPath Indicates the Uniform Resource Identifier (URI) of the request. No domain name information is included. /index.html
      urlRawQuery Indicates the query parameters that follow the question mark (?)in a URL. x=1&y=1
      userlnfo Indicates custom fields. None
    2. Click Activate.

      If you want to modify log storage rules or disable log storage after log storage is enabled, you can perform relevant operations on the Log Storage tab.

      After log storage is disabled, you can delete the SLR AliyunServiceRoleForCDNLogDelivery based on business requirements. For more information, see Delete AliyunServiceRoleForCDNLogDelivery.

FAQ

What is the format of the files that are delivered to OSS after log storage is enabled?

Files delivered to OSS are in the Optimized Row Columnar (ORC) format. We recommend that you use DLA to analyze ORC files.