After you configure mirroring-based back-to-origin rules for a bucket, if the object that a requester accesses does not exist in your bucket, Object Storage Service (OSS) obtains the object from the origin specified by the back-to-origin rules. OSS returns the object obtained from the origin to the requester and stores the object in the bucket.

Limits

  • Number of rules

    You can configure up to 20 back-to-origin rules for a bucket. The rules are sequentially matched with requests based on the values of RuleNumber. If a request matches a rule, the subsequent rules are not matched. OSS determines whether a request matches a back-to-origin rule based on whether the request meets the conditions specified in the rule. OSS does not check whether the requested object is obtained from the origin.

  • QPS and traffic limits

    In regions within China, the default number of queries per second (QPS) for mirroring-based back-to-origin is 2,000 and the default bandwidth is 2 Gbit/s. In regions outside China, the default QPS value for mirroring-based back-to-origin is 1,000 and the default bandwidth is 1 Gbit/s.

  • Back-to-origin URL

    An internal endpoint cannot be used as a back-to-origin URL. You can add image processing parameters to a back-to-origin URL. Video snapshot parameters cannot be added to a back-to-origin URL.

  • Default timeout period

    By default, the timeout period of a mirroring-based back-to-origin request is 10 seconds.

Scenarios

Mirroring-based back-to-origin is used to seamlessly migrate data to OSS. You can use mirroring-based back-to-origin to migrate services from a self-managed origin or from another cloud service to OSS without service interruption. You can use mirroring-based back-to-origin to obtain the data that is not migrated to OSS during migration. This ensures service continuity. For more information about use cases, see Migrate data of an Internet-based application to OSS.

Procedure

The following figure shows how mirroring-based back-to-origin works.

Configure back-to-origin rules

  • Trigger conditions

    OSS obtains an object from the origin by using the mirroring-based back-to-origin feature only if 404 is returned for the GetObject operation.

  • Naming conventions

    The URL used by OSS to obtain an object from the origin is in the following format: http(s)://MirrorURL/ObjectName. ObjectName specifies the name of the requested object. For example, if the back-to-origin URL configured for a bucket is https://aliyun.com and the requested object named example.jpg does not exist in the bucket, OSS obtains the object from https://aliyun.com/example.jpg and stores the object as example.jpg.

  • Rules for failed back-to-origin requests

    If the requested object is not found in the origin, the origin returns HTTP status code 404 to OSS. Then, OSS returns the same HTTP status code to the requester. If the origin returns a non-200 HTTP status code to OSS, which indicates an error occurred, such as an object retrieval failure due to network errors, OSS returns 424 MirrorFailed to the requester.

  • x-oss-tag response header

    When OSS returns an object that is obtained from the origin, OSS adds the x-oss-tag header to the response and sets the value of the header to MIRROR. The header is in the following format: x-oss-tag:MIRROR.

    After OSS obtains an object from the origin, this header is added to the response when the object is downloaded unless the object is overwritten. The header indicates that the object is obtained by using mirroring-based back-to-origin.

  • Rules to update objects obtained from the origin

    After OSS obtains an object by using mirroring-based back-to-origin, if the object on the origin is modified, OSS does not update the obtained object.

  • Metadata of objects obtained from the origin
    OSS stores the following HTTP headers that are returned from the origin as the object metadata:
    Content-Type
    Content-Encoding
    Content-Disposition
    Cache-Control
    Expires
    Content-Language
    Access-Control-Allow-Origin
  • HTTP request rules
    • Headers that are contained in the request sent to OSS are not contained in the request sent by OSS to the origin. OSS determines whether to send the QueryString information to the origin based on the back-to-origin rules that you configure.
    • If the origin returns chunked-encoded data, OSS returns chunked-encoded data to you.

Use the OSS console

If you configure multiple back-to-origin rules for a bucket in the OSS console, the rules are used to match a request in the sequence that they are configured. You can click Move Up or Move Down on the right side of the rules to change the priority of the rules.

3

When a requester accesses an object in the specified bucket but the object does not exist, you can specify the URL of the object in the origin and back-to-origin conditions to obtain the object. For example, you have a bucket named example that is located in the China (Hangzhou) region. You want your requester to obtain the required object in the examplefolder folder from https://www.example.com/ when the requester accesses the object that does not exist in the examplefolder folder of the bucket root folder. To configure a back-to-origin rule, perform the following steps:

  1. In the left-side navigation pane, choose Basic Settings > Back-to-Origin.
  2. Click Configure. Click Create Rule.
  3. In the Create Rule panel, set Mode to Mirroring.
  4. Configure Prerequisites and Origin URL.
    Parameter Description
    Prerequisite Select File Name Prefix, and then set File Name Prefix to examplefolder/.
    Note File Name Prefix and File Name Suffix are optional. When you configure multiple back-to-origin rules, you must set different file name prefixes or suffixes to differentiate back-to-origin rules.
    Origin URL Set the first column to https, the second column to www.example.com, and the third column to examplefolder.
  5. Click OK.
    Access process after the back-to-origin rule is configured:
    1. A requester accesses https://examplebucket.oss-cn-hangzhou.aliyuncs.com/examplefolder/example.txt for the first time.
    2. If the examplefolder/example.txt object does not exist in examplebucket, OSS sends a request to https://www.example.com/examplefolder/example.txt.
    3. If the required object is obtained, OSS writes the example.txt object to the examplefolder folder in examplebucket and then returns the object to the requester. If the required object is not obtained, HTTP status code 404 is returned to the requester.

You can follow the preceding steps to configure mirroring-based back-to-origin rules for basic scenarios. If you want to configure mirroring-based back-to-origin rules for specific scenarios, see Special configurations of mirroring-based back-to-origin.

Use OSS SDKs

The following sample code provides examples on how to configure mirroring-based back-to-origin rules by using OSS SDKs for common programming languages. For more information about how to configure mirroring-based back-to-origin rules by using OSS SDKs for other programming languages, see Overview.

import com.aliyun.oss.ClientException;
import com.aliyun.oss.OSS;
import com.aliyun.oss.OSSClientBuilder;
import com.aliyun.oss.OSSException;
import com.aliyun.oss.model.RoutingRule;
import com.aliyun.oss.model.SetBucketWebsiteRequest;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class Demo {

    public static void main(String[] args) throws Exception {
        // In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint. 
        String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
        // The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
        String accessKeyId = "yourAccessKeyId";
        String accessKeySecret = "yourAccessKeySecret";
        // Specify the bucket name. Example: examplebucket. 
        String bucketName = "examplebucket";

        // Create an OSSClient instance. 
        OSS ossClient = new OSSClientBuilder().build(endpoint, accessKeyId, accessKeySecret);

        try {
            SetBucketWebsiteRequest request = new SetBucketWebsiteRequest(bucketName);
            // Specify the operation to perform if the default homepage is specified, the name of the accessed object does not end with a forward slash (/), and the object does not exist. 
            //request.setSubDirType(null);
            // Specify whether to redirect the requester to the default homepage in the subdirectory when the subdirectory is accessed. 
            //request.setSupportSubDir(false);

            List<RoutingRule> routingRules = new ArrayList<RoutingRule>();

            RoutingRule rule = new RoutingRule();
            rule.setNumber(1);
            // Specify the prefix of object names. Only the objects whose names contain the specified prefix match the rule. 
            rule.getCondition().setKeyPrefixEquals("examplebucket");
            // Specify the HTTP status code. The rule is matched only when the specified object is accessed and HTTP status code 404 is returned. 
            rule.getCondition().setHttpErrorCodeReturnedEquals(404);
            // Specify the redirect type. 
            rule.getRedirect().setRedirectType(RoutingRule.RedirectType.Mirror);
            // Specify the origin URL for the mirroring-based back-to-origin rule. Example: https://www.example.com/. 
            rule.getRedirect().setMirrorURL("<yourMirrorURL>");
            //rule.getRedirect().setMirrorRole("AliyunOSSMirrorDefaultRole");
            // Specify whether to include parameters of the original request in the redirect request when the system uses the redirection rule or the mirroring-based back-to-origin rule. 
            rule.getRedirect().setPassQueryString(true);
            // This parameter is used in the same manner as the PassQueryString parameter and is assigned a higher priority level than the PassQueryString parameter. This parameter takes effect only when RedirectType is set to Mirror. 
            rule.getRedirect().setMirrorPassQueryString(true);
            // Specify the HTTP status code in the response. This parameter takes effect only when RedirectType is set to External or AliCDN. 
            //rule.getRedirect().setHttpRedirectCode(302);
            // Specify the domain name that is used for redirection, which must comply with the naming conventions for domains. 
            //rule.getRedirect().setHostName("oss.aliyuncs.com");
            // Specify the protocol that is used for redirection. This parameter takes effect only when RedirectType is set to External or AliCDN. 
            //rule.getRedirect().setProtocol(RoutingRule.Protocol.Https);
            // Specify the string that is used to replace the object name when the request is redirected. This parameter can be set to a variable. 
            //rule.getRedirect().setReplaceKeyWith("${key}.jpg");
            // If this parameter is set to true, the prefix of the object name is replaced with the value specified by ReplaceKeyPrefixWith. 
            rule.getRedirect().setEnableReplacePrefix(true);
            // Specify the string that is used to replace the prefix of the object name when the request is redirected. 
            rule.getRedirect().setReplaceKeyPrefixWith("examplebucket");
            // Specify whether to check the MD5 hash of the response body that is returned by the origin. This parameter takes effect only when RedirectType is set to Mirror. 
            rule.getRedirect().setMirrorCheckMd5(true);

            RoutingRule.MirrorHeaders mirrorHeaders = new RoutingRule.MirrorHeaders();
            // Specify whether to pass through all request headers to the origin. This parameter takes effect only when RedirectType is set to Mirror. 
            mirrorHeaders.setPassAll(false);
            List passes = new ArrayList<String>();
            passes.add("cache-control");
            // Specify the headers to pass through to the origin. This parameter takes effect only when RedirectType is set to Mirror. 
            mirrorHeaders.setPass(passes);
            List removes = new ArrayList<String>();
            removes.add("content-type");
            // Specify the headers that are not allowed to pass through to the origin. This parameter takes effect only when RedirectType is set to Mirror. 
            mirrorHeaders.setRemove(removes);
            List sets = new ArrayList<Map<String, String>>();
            Map header1 = new HashMap<String, String>();
            header1.put("Key", "key1");
            header1.put("Value", "value1");
            Map header2 = new HashMap<String, String>();
            header2.put("Key", "key2");
            header2.put("Value", "value2");
            sets.add(header1);
            sets.add(header2);
            // Specify the headers that are passed to the origin. The specified headers are passed to the origin regardless of whether they are included in the request. 
            mirrorHeaders.setSet(sets);
            // Specify the headers included in the request when you use mirroring-based back-to-origin. This parameter takes effect only when RedirectType is set to Mirror. 
            rule.getRedirect().setMirrorHeaders(mirrorHeaders);

            routingRules.add(rule);
            request.setRoutingRules(routingRules);
            ossClient.setBucketWebsite(request);
        } catch (OSSException oe) {
            System.out.println("Caught an OSSException, which means your request made it to OSS, "
                    + "but was rejected with an error response for some reason.");
            System.out.println("Error Message:" + oe.getErrorMessage());
            System.out.println("Error Code:" + oe.getErrorCode());
            System.out.println("Request ID:" + oe.getRequestId());
            System.out.println("Host ID:" + oe.getHostId());
        } catch (ClientException ce) {
            System.out.println("Caught an ClientException, which means the client encountered "
                    + "a serious internal problem while trying to communicate with OSS, "
                    + "such as not being able to access the network.");
            System.out.println("Error Message:" + ce.getMessage());
        } finally {
            if (ossClient != null) {
                ossClient.shutdown();
            }
        }
    }
}
# -*- coding: utf-8 -*-
import oss2
from oss2.models import BucketWebsite

# Security risks may arise if you use the AccessKey pair of an Alibaba Cloud account to access OSS because the account has permissions on all API operations. We recommend that you use a RAM user to call API operations or perform routine operations and maintenance. To create a RAM user, log on to the RAM console. 
auth = oss2.Auth('yourAccessKeyId', 'yourAccessKeySecret')
# In this example, the endpoint of the China (Hangzhou) region is used. Specify the endpoint based on your business requirements. 
# Specify the name of the bucket. Example: examplebucket. For more information about the naming conventions for buckets, see Bucket naming conventions. 
bucket = oss2.Bucket(auth, 'https://oss-cn-hangzhou.aliyuncs.com', 'examplebucket')

# Enable static website hosting and set the default homepage to index.html and the default 404 page to error.html. 
index_file = 'index.html'
error_file = 'error.html'
# Specify the matching conditions. 
condition1 = Condition(key_prefix_equals='examplefolder',
                       http_err_code_return_equals=404)

# Specify the headers included in the request when you use mirroring-based back-to-origin. 
mirror_headers_set_1 = MirrorHeadersSet("myheader-key5","myheader-value5")
mirror_headers_set_2 = MirrorHeadersSet("myheader-key6","myheader-value6")
set_list = [mirror_headers_set_1, mirror_headers_set_2]
pass_list = ['myheader-key1', 'myheader-key2']
remove_list = ['myheader-key3', 'myheader-key4']
mirror_header = RedirectMirrorHeaders(pass_all=True, pass_list=pass_list, remove_list=remove_list, set_list=set_list)
# Specify the operation to perform after the rule is matched. 

redirect1 = Redirect(redirect_type=REDIRECT_TYPE_MIRROR, mirror_url='https://www.example.com/',
                     mirror_pass_query_string=True, mirror_follow_redirect=True, mirror_check_md5=True, mirror_headers=mirror_header)

rule1 = RoutingRule(rule_num=1, condition=condition1, redirect=redirect1)

# Configure mirroring-based back-to-origin. 
website_set = BucketWebsite(index_file, error_file, [rule1])
package main

import (
    "fmt"
    "os"
    "github.com/aliyun/aliyun-oss-go-sdk/oss"
)

func main() {
    // Create an OSSClient instance. 
    // Set yourEndpoint to the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set yourEndpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify the endpoint based on your business requirements. 
    // Security risks may arise if you use the AccessKey pair of an Alibaba Cloud account to access OSS because the account has permissions on all API operations. We recommend that you use a RAM user to call API operations or perform routine operations and maintenance. To create a RAM user, log on to the RAM console. 
    client, err := oss.New("yourEndpoint", "yourAccessKeyId", "yourAccessKeySecret")
    if err != nil {
        fmt.Println("Error:", err)
        os.Exit(-1)
    }

    // Specify the name of the bucket. Example: examplebucket. 
    bucketName := "examplebucket"
    putXml := `<WebsiteConfiguration>
      <IndexDocument>
        <Suffix>index.html</Suffix>
        <SupportSubDir>true</SupportSubDir>
        <Type>0</Type>
      </IndexDocument>
      <ErrorDocument>
        <Key>error.html</Key>
        <HttpStatus>404</HttpStatus>
      </ErrorDocument>
      <RoutingRules>
        <RoutingRule>
          <RuleNumber>1</RuleNumber>
          <Condition>
            <! -- Specify the prefix of object names. Only the objects whose names contain the specified prefix match the rule. -->
            <KeyPrefixEquals>examplefolder/</KeyPrefixEquals>
            <! -- Specify the HTTP status code. The rule is matched only when the specified object is accessed and HTTP status code 404 is returned. -->
            <HttpErrorCodeReturnedEquals>404</HttpErrorCodeReturnedEquals>
          </Condition>
          <Redirect>
            <! -- Specify the redirect type. -->
            <RedirectType>Mirror</RedirectType>
            <PassQueryString>true</PassQueryString>
            <! -- Specify the origin URL for the mirroring-based back-to-origin rule. In this example, the origin URL is https://www.example.com/. -->
            <MirrorURL>http://example.com/</MirrorURL>-->
            <! -- Specify the status codes such as 4xx and 5xx that OSS must pass through with the response body to the client after OSS receives the status codes from the origin. -->            
            <PassQueryString>true</PassQueryString>
            <! -- This parameter plays the same role as PassQueryString and has a higher priority than PassQueryString. This parameter takes effect only when RedirectType is set to Mirror. -->
            <MirrorPassQueryString>true</MirrorPassQueryString>
            <! -- Specify the HTTP status code in the response. This parameter takes effect only when RedirectType is set to External or AliCDN. -->
            <HttpRedirectCode>302</HttpRedirectCode>
            <! -- Specify the domain name that is used for redirection, which must comply with the naming conventions for domains. -->
            <HostName>oss.aliyuncs.com</HostName>
            <! -- Specify the protocol that is used for redirection. This parameter takes effect only when RedirectType is set to External or AliCDN. -->
            <Protocol>https</Protocol>
            <! -- Specify the string that is used to replace the object name when the request is redirected. This parameter can be set to a variable. -->
            <ReplaceKeyWith>key.jpg</ReplaceKeyWith>
            <! -- If this parameter is set to true, the prefix of the object name is replaced with the value specified by ReplaceKeyPrefixWith. -->
            <EnableReplacePrefix>true</EnableReplacePrefix>
            <! -- Specify the string that is used to replace the prefix of the object name when the request is redirected. -->
            <ReplaceKeyPrefixWith>examplebucket</ReplaceKeyPrefixWith>
            <! -- Specify whether to check the MD5 hash of the body of the response that is returned by the origin. This parameter takes effect only when RedirectType is set to Mirror. -->
            <MirrorCheckMd5>false</MirrorCheckMd5>
            <! -- Specify whether to redirect the access to the address that is specified by Location if the origin returns HTTP status code 3xx.  This parameter takes effect only when RedirectType is set to Mirror. -->
            <MirrorFollowRedirect>true</MirrorFollowRedirect>
            <MirrorHeaders>
              <! -- Specify whether to pass through all request headers to the origin. This parameter takes effect only when RedirectType is set to Mirror. -->
              <PassAll>true</PassAll>
              <! -- Specify the headers to pass through to the origin. This parameter takes effect only when RedirectType is set to Mirror. -->
              <Pass>myheader-key1</Pass>
              <Pass>myheader-key2</Pass>
              <! -- Specify the headers that are not allowed to pass through to the origin. This parameter takes effect only when RedirectType is set to Mirror. -->
              <Remove>myheader-key3</Remove>
              <Remove>myheader-key4</Remove>
               <! -- Specify the headers that are passed to the origin. The specified headers are passed to the origin regardless of whether they are included in the request. -->
              <Set>
                <Key>myheader-key5</Key>
                <Value>myheader-value5</Value>
              </Set>
            </MirrorHeaders>
          </Redirect>
        </RoutingRule>
      </RoutingRules>
    </WebsiteConfiguration>    `

    err = client.SetBucketWebsiteXml(bucketName,putXml)
    if err != nil {
        fmt.Println("Error:", err)
        os.Exit(-1)
    }
}                    

Use ossutil

For more information about how to configure mirroring-based back-to-origin rules by using ossutil, see Add or modify website-related configurations.

Use the RESTful API

If your business requires a high level of customization, you can directly call RESTful APIs. To directly call an API, you must include the signature calculation in your code. For more information, see PutBucketWebsite