Cross-region replication (CRR) enables the automatic and asynchronous (near real-time) replication of objects across Object Storage Service (OSS) buckets in different regions. Operations such as the creation, overwriting, and deletion of objects can be synchronized from a source bucket to a destination bucket.

Scenarios

CRR can meet your requirements for cross-region disaster recovery and data replication. The objects in a destination bucket are exact replicas of objects in a source bucket. The objects have the same names, versioning information, content, and metadata, such as the creation time, owner, user metadata, and access control lists (ACLs). You can configure CRR rules in the following scenarios to meet your business requirements:

  • Compliance requirements

    OSS stores multiple replicas of objects in physical disks. However, to meet compliance requirements, the replicas must be stored in multiple data centers that are located at a geographical distance from each other. CRR allows you to replicate data between geographically distant data centers to meet compliance requirements.

  • Minimal latency

    You have customers who are located in two geographical locations. To minimize the latency that occurs when the customers access objects, you can maintain the replicas of objects in data centers that are geographically closer to the customers.

  • Data backup and disaster recovery

    You have strict requirements for data security and availability. You want to replicate all data in one data center to another data center. If one data center is damaged because of a catastrophic event, such as an earthquake or a tsunami, you can use data that is backed up in the other data center.

  • Data migration

    For business reasons, you may need to migrate data from one data center to another data center.

  • Operational reasons

    You have compute clusters that are deployed in two data centers to analyze the same group of objects. You can maintain the replicas of the objects in the two regions.

Features

CRR provides the following features:

  • Replication Time Control (RTC)

    After the RTC feature is enabled, OSS replicates most of the objects that you uploaded to OSS within a few seconds and replicates 99.99% of objects within 10 minutes. In addition, the RTC feature provides real-time monitoring of data replication. This way, you can view various metrics of replication tasks.

  • Near real-time replication of data

    You can configure CRR rules to monitor data that is added, removed, or modified in near real time and synchronize these changes to a destination bucket. This ensures data consistency between the source and destination buckets.

  • Historical data migration

    Historical data can be replicated from a source bucket to a destination bucket. This way, two data replicas are individually stored in the source and destination buckets.

  • Display of the replication progress

    You can view the most recent replication time for the replicated data and the progress of the replication for historical data migration in percentage.

  • Versioning

    CRR ensures eventual consistency between the data in the source and destination buckets for which versioning is enabled. If you configure a CRR rule to replicate only the added and modified data, delete operations performed on the specified version of an object in the source bucket are not replicated to the destination bucket. However, the delete markers created in the source bucket are replicated to the destination bucket.

  • Transfer acceleration

    You can use the transfer acceleration feature to accelerate data transfer when CRR tasks are performed across regions inside and outside the Chinese mainland. For more information about the transfer acceleration feature, see Enable transfer acceleration.

  • Replication of encrypted data

    CRR allows you to replicate objects that are not encrypted and objects that are encrypted by using SSE-KMS or SSE-OSS on the OSS server. For more information, see CRR in specific scenarios.

  • Event notification and real-time log query
    You can use the following methods to receive notifications when changes are made to objects in source and destination buckets during CRR. The changes include adding, modifying, removing, and overwriting objects.
    • Set the event type to one of the following values in the event notification rule: ObjectReplication:ObjectCreated, ObjectReplication:ObjectRemoved, and ObjectReplication:ObjectModified. For more information, see Overview.
    • Enable real-time log query in the OSS console to obtain the statistics of operations that are performed on objects. For more information, see Query real-time logs.

Usage notes

  • Billable items
    • You are charged for the traffic that is generated when you use CRR to replicate objects in OSS. For more information, see Traffic fees.
    • Each time an object is replicated, OSS accumulates the number of requests, and you are charged for the requests. For more information, see API operation calling fees.
    • If you enable the transfer acceleration feature, you are charged transfer acceleration fees. For more information, see Transfer acceleration fees.
    • If you enable the RTC feature, you are charged additional RTC fees. For more information, see RTC traffic fees.
  • Replication time

    In CRR, data is asynchronously replicated in near real time. The time that is required to replicate data from the source bucket to the destination bucket may range from a few minutes to a few hours. The replication time varies based on the data size.

Limits

  • Limits on regions
    • You can configure CRR rules only in the following regions: China (Hangzhou), China (Shanghai), China (Nanjing - Local Region), China (Fuzhou-Local Region), China (Qingdao), China (Beijing), China (Zhangjiakou), China (Hohhot), China (Ulanqab), China (Shenzhen), China (Heyuan), China (Guangzhou), China (Chengdu), China (Hong Kong), US (Silicon Valley), US (Virginia), Japan (Tokyo), Singapore, Australia (Sydney), Malaysia (Kuala Lumpur), Indonesia (Jakarta), Philippines (Manila), India (Mumbai), Germany (Frankfurt), UK (London), and UAE (Dubai).
    • You must enable the transfer acceleration feature when you perform CRR tasks between the regions inside and outside the Chinese mainland.
    • CRR rules based on object tags can be configured only in the following scenarios:
      • The source region is China (Hangzhou), and the destination region is a region except for China (Hangzhou).
      • The source region is Australia (Sydney), and the destination region is a region outside the Chinese mainland except for Australia (Sydney).
  • Number of CRR rules
    Data in a source bucket can be replicated to multiple destination buckets. You can configure up to 100 CRR rules for a bucket. You can specify the bucket as a source bucket or a destination bucket. 1

    If you want to configure more than 100 CRR rules for a bucket, contact technical support.

  • Limits on operations
    • You can configure CRR between two unversioned buckets or two versioned buckets.
    • The versioning status of two buckets between which a CRR rule is configured cannot be changed.
    • If you configure a CRR rule for two buckets, an object replicated from the source bucket may overwrite an object that has the same name in the destination bucket.
    • Data in a source bucket can be replicated to multiple destination buckets. You can configure up to 100 CRR rules for a bucket. You can specify the bucket as a source bucket or a destination bucket. If you want to configure more than 100 CRR rules for a bucket, contact technical support.
    • Cold Archive objects in a source bucket cannot be replicated to a destination bucket.
    • Appendable objects in a source bucket cannot be replicated to a destination bucket whose storage class is Cold Archive.

Use the OSS console

  1. Log on to the OSS console.
  2. In the left-side navigation pane, click Buckets. On the Buckets page, click the name of the bucket for which you want to enable CRR.
  3. In the left-side navigation tree, choose Data Management > Cross-Region Replication.
  4. On the Cross-Region Replication tab, click Cross-Region Replication.
  5. In the Cross-Region Replication panel, configure the parameters. The following table describes the parameters.
    Section Parameter Description
    Configure Destination Bucket Source Region The region in which the source bucket is located.
    Source Bucket The name of the current bucket.
    Destination Region Specify the region in which the destination bucket is located.
    Destination Bucket Specify the name of the bucket to which you want to replicate data.
    Configure Replication Policy Objects to Replicate Select the objects that you want to replicate.
    • All Files in Source Bucket: OSS replicates all objects from the source bucket to the destination bucket.
    • Files with Specified Prefix: OSS replicates the objects whose names contain a specific prefix from the source bucket to the destination bucket. You can specify up to 10 prefixes.
    Object Tagging Specify the tags of the objects that you want to replicate to the destination bucket. Objects that have the specified tags are replicated to the destination bucket. Select Configure Rules and add tags in key-value pairs. You can add up to 10 tags.

    To configure this parameter, make sure that the following conditions are met:

    • Tags are configured for objects. For more information, see Configure object tagging.
    • Versioning is enabled for the source and destination buckets.
    • The Replication Policy parameter is set to Add/Change.
    • If the source region is China (Hangzhou), the destination region can be any region except for China (Hangzhou). If the source region is Australia (Sydney), the destination region can be any region outside the Chinese mainland except for Australia (Sydney).
    Replication Policy Configure the data replication mode.
    • Add/Change: OSS replicates only the added or changed data from the source bucket to the destination bucket.
    • Add/Delete/Change: OSS replicates all data changes including the create, overwrite, and delete operations on objects from the source bucket to the destination bucket.

    If you use the multipart upload method to upload an object to the source bucket, each uploaded part is replicated to the destination bucket. The complete object that is obtained by calling the CompleteMultipartUpload operation is also replicated to the destination bucket.

    For more information about how to configure CRR for objects in versioned buckets, see CRR in specific scenarios.

    Replicate Historical Data Specify whether to replicate historical data in the source bucket before you enable CRR for the source bucket.
    • Yes: Historical data is replicated to the destination bucket.
      Important When historical data is replicated, objects in the source bucket may overwrite objects that have the same names in the destination bucket. To prevent data loss, we recommend that you enable versioning for the source and destination buckets.
    • No: OSS replicates only objects that are uploaded or updated after the CRR rule takes effect to the destination bucket.
    Replicate Objects Encrypted based on KMS If KMS-based encryption is configured for the source objects or destination bucket, you must select Yes and configure the following parameters:
    • CMK ID: The customer master key (CMK) that is used to encrypt the destination objects.

      If you want to use a CMK to encrypt objects, you must create a CMK in the same region as the destination bucket in the Key Management Service (KMS) console. For more information, see Create a CMK.

    • RAM Role Name: The RAM role that is authorized to perform KMS-based encryption on the destination object.
      • New RAM Role: A RAM role is created to perform KMS-based encryption on the destination object. The name of the RAM role is in the following format: kms-replication-source bucket name-destination bucket name.
      • AliyunOSSRole: The AliyunOSSRole role is used to perform KMS-based encryption on the destination object. If the AliyunOSSRole role does not exist, OSS automatically creates the AliyunOSSRole role when you select this option.
    Note
    Configure Replication Speed Acceleration Type Specify the acceleration type. Only Transfer Acceleration is supported. You can use the transfer acceleration feature to accelerate data transfer when you replicate data across regions inside the Chinese mainland and outside the Chinese mainland. If you enable the transfer acceleration feature, you are charged transfer acceleration fees. For more information, see Transfer acceleration fees.
    Replication Time Control (RTC) After the RTC feature is enabled, OSS replicates most of the objects that you uploaded to OSS within a few seconds and replicates 99.99% of objects within 10 minutes. For more information about RTC, see RTC. You are charged when you enable RTC. For more information, see RTC traffic fees.
  6. Click OK.
    • After you create a CRR rule, the rule cannot be modified or deleted.
    • After you configure a CRR rule, the replication task starts in 3 to 5 minutes. To view the replication progress, choose Data Management > Cross-Region Replication on the Overview page of the source bucket.
    • In CRR, data is asynchronously replicated. Depending on the amount of data, the time that is required to replicate data from the source bucket to the destination bucket may range from a few minutes to a few hours.

Use OSS SDKs

The following code provides examples on how to enable CRR by using OSS SDKs for common programming languages. For more information about how to enable CRR by using OSS SDKs for other programming languages, see Overview. 

import com.aliyun.oss.ClientException;
import com.aliyun.oss.OSS;
import com.aliyun.oss.OSSClientBuilder;
import com.aliyun.oss.OSSException;
import com.aliyun.oss.model.AddBucketReplicationRequest;

public class Demo {

    public static void main(String[] args) throws Exception {
        // In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint. 
        String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
        // The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
        String accessKeyId = "yourAccessKeyId";
        String accessKeySecret = "yourAccessKeySecret";
        // Specify the name of the bucket. Example: examplebucket. 
        String bucketName = "examplebucket";
        // Specify the destination bucket to which you want to replicate the data. 
        String targetBucketName = "yourTargetBucketName";
        // Specify the ID of the region in which the destination bucket is located. Example: oss-cn-beijing. 
        String targetBucketLocation = "oss-cn-beijing";

        // Create an OSSClient instance. 
        OSS ossClient = new OSSClientBuilder().build(endpoint, accessKeyId, accessKeySecret);

        try {
            AddBucketReplicationRequest request = new AddBucketReplicationRequest(bucketName);

            request.setTargetBucketName(targetBucketName);
            request.setTargetBucketLocation(targetBucketLocation);
            // Specify whether to replicate historical data. By default, historical data is replicated. In this example, this parameter is set to false, which indicates that historical data is not replicated. 
            request.setEnableHistoricalObjectReplication(false);
            // Specify the role that you authorize OSS to use to replicate data. If you want to use SSE-KMS to encrypt the objects that are replicated to the destination bucket, you must specify a role. 
            //request.setSyncRole("yourRole");
            // Specify whether to replicate the objects that are encrypted by using SSE-KMS. 
            //request.setSseKmsEncryptedObjectsStatus("Enabled");
            // Specify the key ID used in SSE-KMS. If Status is set to Enabled, you must specify a key ID. 
            //request.setReplicaKmsKeyID("3542abdd-5821-4fb5-a425-90adca***");
            //List prefixes = new ArrayList();
            //prefixes.add("image/");
            //prefixes.add("video");
            //prefixes.add("a");
            //prefixes.add("A");
            // Specify the prefix that is contained in the names of the objects that you want to replicate. After you specify the prefix, only objects whose names contain the prefix are replicated to the destination bucket. 
            //request.setObjectPrefixList(prefixes);
            //List actions = new ArrayList();
            //actions.add(AddBucketReplicationRequest.ReplicationAction.ALL);
            // Specify the operations that can be synchronized to the destination bucket. By default, the value is ALL, which indicates that all operations on the source bucket are synchronized to the destination bucket. 
            //request.setReplicationActionList(actions);
            ossClient.addBucketReplication(request);
        } catch (OSSException oe) {
            System.out.println("Caught an OSSException, which means your request made it to OSS, "
                    + "but was rejected with an error response for some reason.");
            System.out.println("Error Message:" + oe.getErrorMessage());
            System.out.println("Error Code:" + oe.getErrorCode());
            System.out.println("Request ID:" + oe.getRequestId());
            System.out.println("Host ID:" + oe.getHostId());
        } catch (ClientException ce) {
            System.out.println("Caught an ClientException, which means the client encountered "
                    + "a serious internal problem while trying to communicate with OSS, "
                    + "such as not being able to access the network.");
            System.out.println("Error Message:" + ce.getMessage());
        } finally {
            if (ossClient != null) {
                ossClient.shutdown();
            }
        }
    }
}        
# -*- coding: utf-8 -*-
import oss2
from oss2.models import ReplicationRule
# The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
auth = oss2.Auth('yourAccessKeyId', 'yourAccessKeySecret')
# Specify the endpoint of the region in which the source bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. For more information about the endpoints of other regions, see Regions and endpoints. 
# Specify the name of the source bucket. Example: srcexamplebucket. 
bucket = oss2.Bucket(auth, 'https://oss-cn-hangzhou.aliyuncs.com', 'srcexamplebucket')
replica_config = ReplicationRule(
    # Specify the destination bucket to which data is replicated. 
    target_bucket_name='destexamplebucket',
    # Specify the region in which the destination bucket is located. 
    target_bucket_location='oss-cn-beijing'
)

# Specify the prefixes that are contained in the names of the objects that you want to replicate. After you specify the prefixes, only the objects whose names contain the prefixes are replicated to the destination bucket. 
# prefix_list = ['prefix1', 'prefix2']
# Specify CRR rules. 
# replica_config = ReplicationRule(
     # prefix_list=prefix_list,
     # Specify the operations that can be synchronized to the destination bucket. The default value is ALL. This indicates that all operations on the source bucket are synchronized to the destination bucket. 
     # action_list=[ReplicationRule.ALL],
     # Specify the destination bucket to which data is replicated. 
     # target_bucket_name='destexamplebucket1',
     # Specify the region in which the destination bucket is located. 
     # target_bucket_location='oss-cn-shanghai',
     # By default, historical data is replicated. In this example, this parameter is set to False. This indicates that historical data is not replicated. 
     # is_enable_historical_object_replication=False,
     # Specify the link that is used to transfer data during data replication. 
     # target_transfer_type='oss_acc',
     # Specify the role that is assumed by OSS to replicate data. If you want to use SSE-KMS to encrypt the objects that are replicated to the destination bucket, you must specify this parameter. 
     # sync_role_name='roleNameTest',
     # Replicate the objects that are encrypted by using SSE-KMS. 
     # sse_kms_encrypted_objects_status=ReplicationRule.ENABLED
     # Specify the customer master key (CMK) ID that is used in SSE-KMS. If you want to use SSE-KMS to encrypt the objects that are replicated to the destination bucket, you must configure this parameter. 
     # replica_kms_keyid='9468da86-3509-4f8d-a61e-6eab1eac****',
  #)

# Enable CRR. 
bucket.put_bucket_replication(replica_config)
package main

import (
    "fmt"
    "github.com/aliyun/aliyun-oss-go-sdk/oss"
    "os"
)

func HandleError(err error) {
    fmt.Println("Error:", err)
    os.Exit(-1)
}
// Enable CRR for a source bucket. 
func main()  {
    // Specify the endpoint of the region in which the source bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify your actual endpoint. 
    // The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
    client, err := oss.New("yourEndpoint", "yourAccessKeyId", "yourAccessKeySecret")
    if err != nil {
        HandleError(err)
    }
    // Specify the name of the source bucket. 
    srcbucketName := "srcexamplebucket"

    // Specify that only the new data that is written to the bucket after the rule is created is copied. The historical data of the source bucket is not copied. 
    putXml := `<?xml version="1.0" encoding="UTF-8"?>
    <ReplicationConfiguration>
      <Rule>
        <PrefixSet>
            <! -- Specify that objects whose names contain one of the following prefixes are replicated to the destination bucket: prefix_1 and prefix_2. After you specify the prefixes, only the objects whose names contain the prefixes are replicated to the destination bucket. -->
            <! -- To replicate all objects from the source bucket to the destination bucket, do not configure the Prefix parameter. 
            <Prefix>prefix_1</Prefix>
            <Prefix>prefix_2</Prefix>
        </PrefixSet>
        <! -- Specify the operations that can be synchronized to the destination bucket. The default value is ALL. This indicates that all operations on the source bucket are synchronized to the destination bucket. -->
        <Action>ALL</Action>
        <Destination>
            <! -- Specify the destination bucket to which the data is replicated. -->
            <Bucket>destexamplebucket</Bucket>
            <! -- Specify the region in which the destination bucket is located. -->
            <Location>oss-cn-beijing</Location>
            <! -- Specify the link that is used to transfer data during data replication. In this example, this parameter is set to oss_acc. This indicates that the link used to transfer data is accelerated. -->
            <TransferType>oss_acc</TransferType>
        </Destination>
        <! -- By default, historical data is copied. In this example, this parameter is set to disabled. This indicates that the historical data is not copied. -->
        <HistoricalObjectReplication>disabled</HistoricalObjectReplication>
        <! -- Specify the role that you authorize OSS to use to replicate data. If you want to use SSE-KMS to encrypt the objects that are replicated to the destination bucket, you must specify this parameter. -->
        <SyncRole>aliyunramrole</SyncRole>
        <SourceSelectionCriteria>
          <SseKmsEncryptedObjects>
            <! -- Specify whether to replicate objects that are encrypted by using SSE-KMS. -->
            <Status>Enabled</Status>
          </SseKmsEncryptedObjects>
        </SourceSelectionCriteria>
        <EncryptionConfiguration>
            <! -- Specify the customer master key (CMK) ID that is used in SSE-KMS. If Status is set to Enabled, you must specify this parameter. -->
           <ReplicaKmsKeyID>c4d49f85-ee30-426b-a5ed-95e9139d****</ReplicaKmsKeyID>
       </EncryptionConfiguration>
     </Rule>
   </ReplicationConfiguration>`

    err = client.PutBucketReplication(srcbucketName,putXml)
    if err != nil {
        HandleError(err)
    }
}

Use ossutil

For more information about how to enable CRR by using ossutil, see replication.

Use RESTful APIs

If your business requires a high level of customization, you can directly call RESTful APIs. To directly call an API, you must include the signature calculation in your code. For more information, see PutBucketReplication.

FAQ

Do versioning-suspended buckets support CRR?

No. You can configure CRR between two unversioned buckets or two versioned buckets.

Does CRR support the replication of objects across more than two buckets?

No. For example, if you configure a CRR rule to replicate objects from Bucket A to Bucket B and configure another CRR rule to replicate objects from Bucket B to Bucket C, objects that are replicated from Bucket A to Bucket B are not replicated to Bucket C.

If you want to replicate data from Bucket A to Bucket C, you must configure a CRR rule to replicate data from Bucket A to Bucket C.

Why is the replication progress of historical data displayed as 0% for a long period of time?

The replication progress of historical data is not updated in real time. You must wait until all objects are scanned. If a large number of objects are stored in your bucket, such as hundreds of millions of objects, several hours is required before the replication progress of historical data is updated. If the replication progress of historical data is not updated, it does not mean that historical data is not replicated to the destination bucket.

You can check whether historical data in the source bucket is replicated to the destination bucket by viewing the storage capacity of the destination bucket and traffic usage, such as inbound and outbound traffic. For more information about how to view the storage capacity of the destination bucket and traffic usage, see View resource usage.