All Products
Search
Document Center

Object Storage Service:Overview

Last Updated:Aug 14, 2023

Cross-region replication (CRR) enables the automatic and asynchronous (near real-time) replication of objects across buckets in different Object Storage Service (OSS) regions. CRR synchronizes operations such as the creation, overwriting, and deletion of objects from a source bucket to a destination bucket.

Scenarios

CRR can meet your requirements for cross-region disaster recovery and data replication. Objects in destination buckets are exact replicas of objects in the source bucket. The objects have the same object names, versioning information, object content, and object metadata, such as the creation time, owner, user metadata, and access control list (ACL). You can configure CRR rules to meet your business requirements:

  • Compliance requirements

    OSS stores multiple replicas of objects in physical disks. However, to meet compliance requirements, the replicas must be stored in multiple data centers that are located at a geographical distance from each other. CRR allows you to replicate data between geographically distant data centers to meet compliance requirements.

  • Minimal latency

    You have customers who are located in two geographical locations. To minimize the latency that occurs when the customers access objects, you can maintain the replicas of objects in data centers that are geographically closer to the customers.

  • Data backup and disaster recovery

    You have strict requirements for data security and availability. You want to replicate all data in a data center to another data center. If one data center is damaged because of a catastrophic event, such as an earthquake or a tsunami, you can use the data that is backed up in the other data center.

  • Data migration

    For business reasons, you may need to migrate data from one data center to another data center.

  • Operational reasons

    You have compute clusters that are deployed in two data centers to analyze the same group of objects. You can maintain the replicas of the objects in the two regions.

Features

CRR provides the following features:

  • RTC

    After Replication Time Control (RTC) is enabled, OSS replicates most of the objects that you uploaded to OSS within a few seconds and replicates 99.99% of objects within 10 minutes. In addition, RTC provides real-time monitoring of data replication. This allows you to easily monitor replication tasks by using different metrics.

  • Near real-time data replication

    You can configure CRR rules to monitor data that is added, removed, or modified in near real time and synchronize these changes to a destination bucket. This ensures data consistency between the source and destination buckets.

  • Historical data migration

    Historical data can be replicated from a source bucket to a destination bucket. This way, two data replicas are individually stored in the source and destination buckets.

  • Replication progress query

    You can view the most recent replication time of the replicated data and the progress of the replication for historical data migration in percentage.

  • Versioning

    CRR ensures eventual consistency between the data in the source and destination buckets for which versioning is enabled. If you configure a CRR rule to replicate only the added and modified data, deletion operations performed on the specified version of an object in the source bucket are not replicated to the destination bucket. However, the delete markers created in the source bucket are replicated to the destination bucket.

  • Transfer acceleration

    You can use transfer acceleration to accelerate data transfer when CRR tasks are performed across regions inside and outside the Chinese mainland. For more information, see Enable transfer acceleration.

  • Replication of encrypted data

    CRR allows you to replicate objects that are not encrypted and objects that are encrypted by using SSE-KMS or SSE-OSS. For more information, see CRR in specific scenarios.

  • Event notification and real-time log query

    You can use the following methods to receive notifications of changes that are made to objects in source and destination buckets during CRR. The changes include adding, modifying, removing, and overwriting objects.

    • Set the event type to the following values in the event notification rule: ObjectReplication:ObjectCreated, ObjectReplication:ObjectRemoved, and ObjectReplication:ObjectModified. For more information, see Overview.

    • Enable real-time log query in the OSS console to obtain the statistics of operations that are performed on objects. For more information, see Query real-time logs.

Usage notes

  • Billing

    • You are charged for the traffic that is generated when you use CRR to replicate objects in OSS. For more information, see Traffic fees.

    • Each time an object is replicated, OSS calculates the number of requests. You are charged for the requests. For more information, see API operation calling fees.

    • If you enable transfer acceleration, you are charged transfer acceleration fees. For more information, see Transfer acceleration fees.

    • If you enable RTC, you are charged additional RTC fees. For more information, see RTC traffic fees.

    • If you use CRR to replicate Archive objects from a source bucket to a destination bucket, no data retrieval operations are involved in the process, and you are not charged data retrieval fees.

  • Replication time

    In CRR, data is asynchronously replicated in near real time. The period of time that is required to replicate data from the source bucket to the destination bucket may range from a few minutes to a few hours. The replication time varies based on the data size.

Limits

  • Regions

    • CRR is supported in the following regions: China (Hangzhou), China (Shanghai), China (Nanjing - Local Region), China (Fuzhou - Local Region), China (Qingdao), China (Beijing), China (Zhangjiakou), China (Hohhot), China (Ulanqab), China (Shenzhen), China (Heyuan), China (Guangzhou), China (Chengdu), China (Hong Kong), US (Silicon Valley), US (Virginia), Japan (Tokyo), South Korea (Seoul), Singapore, Australia (Sydney), Malaysia (Kuala Lumpur), Indonesia (Jakarta), Philippines (Manila), Thailand (Bangkok), India (Mumbai), Germany (Frankfurt), UK (London), and UAE (Dubai).

    • You must enable transfer acceleration when you perform CRR between the regions inside and outside the Chinese mainland.

    • CRR rules based on object tags can be configured only in the following scenarios:

      • The source region is China (Hangzhou), and the destination region is a region other than China (Hangzhou).

      • The source region is Australia (Sydney), and the destination region is a region outside the Chinese mainland, except for Australia (Sydney).

  • Number of CRR rules

    Data in a source bucket can be replicated to multiple destination buckets. By default, you can configure up to 100 CRR rules for a bucket. A bucket can be specified as a source bucket in a CRR rule and a destination bucket in another CRR rule at the same time.1

    If you want to configure more than 100 CRR rules for a bucket, contact technical support.

  • Operations

    • You can configure CRR between two unversioned buckets or between two versioning-enabled buckets.

    • You cannot change the versioning status of two buckets for which a CRR rule is configured.

    • If you configure a CRR rule for two buckets, an object replicated from the source bucket may overwrite an object that has the same name in the destination bucket.

    • Data in a source bucket can be replicated to multiple destination buckets. By default, you can configure up to 100 CRR rules for a bucket. A bucket can be specified as a source bucket in a CRR rule and a destination bucket in another CRR rule at the same time. If you want to configure more than 100 CRR rules for a bucket, contact technical support.

    • Cold Archive objects in a source bucket cannot be replicated to a destination bucket.

    • You cannot replicate Appendable objects in a source bucket to a destination bucket whose storage class is Cold Archive.

Use the OSS console

  1. Log on to the OSS console.

  2. In the left-side navigation pane, click Buckets. On the Buckets page, click the name of the bucket for which you want to enable CRR.

  3. In the left-side navigation tree, choose Data Management > Cross-Region Replication.

  4. On the Cross-Region Replication tab, click the Cross-Region Replication button.

  5. In the Cross-Region Replication panel, configure the parameters. The following table describes the parameters.

    Section

    Parameter

    Description

    Configure Destination Bucket

    Region

    The region in which the source bucket is located.

    Source Bucket

    The name of the current bucket.

    Region

    The region in which the destination bucket is located.

    Destination Bucket

    The bucket to which you want to replicate data.

    Configure Replication Policy

    Objects to Replicate

    The objects that you want to replicate.

    • All Files in Source Bucket: OSS replicates all objects from the source bucket to the destination bucket.

    • Files with Specified Prefix: OSS replicates the objects whose names contain a specific prefix from the source bucket to the destination bucket. You can specify up to 10 prefixes.

    Object Tagging

    The tags of the objects that you want to replicate to the destination bucket. Objects that contain the specified tags are replicated to the destination bucket. Select Configure Rules and add tags in key-value pairs. You can add up to 10 tags.

    To configure this parameter, make sure that the following conditions are met:

    • Tags are configured for objects. For more information, see Configure object tagging.

    • Versioning is enabled for the source bucket and the destination bucket.

    • The Replication Policy parameter is set to Add/Change.

    • If the source region is China (Hangzhou), the destination region can be a region other than China (Hangzhou). If the source region is Australia (Sydney), the destination region can be a region outside the Chinese mainland, except for Australia (Sydney).

    Replication Policy

    The data replication mode.

    • Add/Change: OSS replicates only the added or changed data from the source bucket to the destination bucket.

    • Add/Delete/Change: OSS replicates object creation, modification, and deletion operations from the source bucket to the destination bucket.

    If you use the multipart upload method to upload an object to the source bucket, each uploaded part is replicated to the destination bucket. The complete object that is obtained by calling the CompleteMultipartUpload operation is also replicated to the destination bucket.

    For more information about how to configure CRR for objects in versioning-enabled buckets, see CRR in specific scenarios.

    Replicate Historical Data

    Specifies whether to replicate historical data (data that exists in the source bucket before you enable CRR) to the destination bucket.

    • Yes: Historical data is replicated to the destination bucket.

      Important

      When historical data is replicated, objects that are replicated from the source bucket may overwrite objects that have the same names in the destination bucket. To prevent data loss, we recommend that you enable versioning for the source and destination buckets.

    • No: OSS replicates only objects that are uploaded or updated after the CRR rule takes effect to the destination bucket.

    Replicate Objects Encrypted based on KMS

    If Key Management Service (KMS)-based encryption is configured for the objects in the source or destination bucket, you must set the Replicate Objects Encrypted based on KMS parameter to Yes and configure the following parameters:

    • CMK ID: The customer master key (CMK) that is used to encrypt the objects in the destination bucket.

      If you want to use a CMK to encrypt objects, you must create a CMK in the same region as the destination bucket in the KMS console. For more information, see Create a CMK.

    • RAM Role Name: The RAM role that is authorized to perform KMS-based encryption on the destination objects.

      • New RAM Role: A RAM role is created by OSS to encrypt the destination objects by using CMKs. The RAM role is in the kms-replication-sourceBucketName-destinationBucketName format.

      • AliyunOSSRole: The AliyunOSSRole role is used to perform KMS-based encryption on the destination objects. If the AliyunOSSRole role does not exist, OSS automatically creates the AliyunOSSRole role when you select this option.

    Note
    • If you create a RAM role or modify the permissions of an existing RAM role, make sure that you attach the AliyunOSSFullAccess policy to the role. Otherwise, data may fail to be replicated.

    • You can call the HeadObject operation to query the encryption status of objects in the source bucket and the GetBucketEncryption operation to query the encryption status of objects in the destination bucket.

    • For more information about how to use CRR with server-side encryption, see CRR in specific scenarios.

    Configure Replication Speed

    Acceleration Type

    The acceleration type. Only Transfer Acceleration is supported. You can use transfer acceleration to accelerate data transfer when you replicate data between regions inside and outside the Chinese mainland. If you enable transfer acceleration, you are charged transfer acceleration fees. For more information, see Transfer acceleration fees.

    Replication Time Control (RTC)

    After the RTC feature is enabled, OSS replicates most of the objects that you uploaded to OSS within a few seconds and replicates 99.99% of objects within 10 minutes. For more information, see RTC. RTC incurs costs. For more information, see RTC traffic fees.

  6. Click OK.

    • After you configure a CRR rule, you cannot modify or delete the rule.

    • After you configure a CRR rule, the replication task starts in 3 to 5 minutes. To view the progress of replication tasks, go to the Cross-Region Replication page of the source bucket.

    • In CRR, data is asynchronously replicated. The period of time that is required to replicate data from the source bucket to the destination bucket varies based on the amount of data. The period of time may range from a few minutes to a few hours.

Use OSS SDKs

The following sample code provides examples on how to enable CRR by using OSS SDKs for common programming languages. For more information about how to enable CRR by using OSS SDKs for other programming languages, see Overview.

import com.aliyun.oss.ClientException;
import com.aliyun.oss.OSS;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.OSSClientBuilder;
import com.aliyun.oss.OSSException;
import com.aliyun.oss.model.AddBucketReplicationRequest;

public class Demo {

    public static void main(String[] args) throws Exception {
        // In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint. 
        String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
        // We recommend that you do not save access credentials in the project code. Otherwise, access credentials may be leaked. As a result, the security of all resources in your account is compromised. In this example, access credentials are obtained from environment variables. You need to configure environment variables before you run the sample code. 
        EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
        // Specify the name of the bucket. Example: examplebucket. 
        String bucketName = "examplebucket";
        // Specify the destination bucket to which you want to replicate the data. 
        String targetBucketName = "yourTargetBucketName";
        // Specify the region in which the destination bucket is located. 
        // If you want to enable CRR, the source and destination buckets must be located in different regions. If you want to enable SRR, the source and destination buckets must be located in the same region. 
        String targetBucketLocation = "yourTargetBucketLocation";

        // Create an OSSClient instance. 
        OSS ossClient = new OSSClientBuilder().build(endpoint, credentialsProvider);

        try {
            AddBucketReplicationRequest request = new AddBucketReplicationRequest(bucketName);

            request.setTargetBucketName(targetBucketName);
            request.setTargetBucketLocation(targetBucketLocation);
            // Specify whether to replicate historical data. By default, historical data is replicated. In this example, this parameter is set to false. This indicates that historical data is not replicated. 
            request.setEnableHistoricalObjectReplication(false);
            // Specify the name of the role that you want to authorize OSS to use to replicate data. If you want to use SSE-KMS to encrypt objects in the source bucket, you must specify a role. 
            //request.setSyncRole("yourRole");
            // Specify whether to replicate objects that are encrypted by using SSE-KMS. 
            //request.setSseKmsEncryptedObjectsStatus("Enabled");
            // Specify the customer master key (CMK) ID used in SSE-KMS. If Status is set to Enabled, you must specify this parameter. 
            //request.setReplicaKmsKeyID("3542abdd-5821-4fb5-a425-90adca***");
            //List prefixes = new ArrayList();
            //prefixes.add("image/");
            //prefixes.add("video");
            //prefixes.add("a");
            //prefixes.add("A");
            // Specify the prefixes that are contained in the names of the objects that you want to replicate. After you specify the prefixes, only objects whose names contain the prefixes are replicated to the destination bucket. 
            //request.setObjectPrefixList(prefixes);
            //List actions = new ArrayList();
            //actions.add(AddBucketReplicationRequest.ReplicationAction.ALL);
            // Specify the operations that can be replicated to the destination bucket. The default value is ALL. This indicates that all operations performed on objects in the source bucket are replicated to the destination bucket. 
            //request.setReplicationActionList(actions);
            ossClient.addBucketReplication(request);
        } catch (OSSException oe) {
            System.out.println("Caught an OSSException, which means your request made it to OSS, "
                    + "but was rejected with an error response for some reason.");
            System.out.println("Error Message:" + oe.getErrorMessage());
            System.out.println("Error Code:" + oe.getErrorCode());
            System.out.println("Request ID:" + oe.getRequestId());
            System.out.println("Host ID:" + oe.getHostId());
        } catch (ClientException ce) {
            System.out.println("Caught an ClientException, which means the client encountered "
                    + "a serious internal problem while trying to communicate with OSS, "
                    + "such as not being able to access the network.");
            System.out.println("Error Message:" + ce.getMessage());
        } finally {
            if (ossClient != null) {
                ossClient.shutdown();
            }
        }
    }
}        
# -*- coding: utf-8 -*-
import oss2
from oss2.models import ReplicationRule
# The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
auth = oss2.Auth('yourAccessKeyId', 'yourAccessKeySecret')
# Specify the endpoint of the region in which the source bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
# Specify the name of the source bucket. Example: srcexamplebucket. 
bucket = oss2.Bucket(auth, 'https://oss-cn-hangzhou.aliyuncs.com', 'srcexamplebucket')
replica_config = ReplicationRule(
    # Specify the destination bucket to which you want to replicate data. 
    target_bucket_name='destexamplebucket',
    # Specify the region in which the destination bucket is located. 
    # If you want to enable CRR, the source and destination buckets must be located in different regions. If you want to enable SRR, the source and destination buckets must be located in the same region. 
    target_bucket_location='yourTargetBucketLocation'
)

# Specify the prefixes that are contained in the names of the objects that you want to replicate. After you specify the prefixes, only objects whose names contain the prefixes are replicated to the destination bucket. 
# prefix_list = ['prefix1', 'prefix2']
# Specify the data replication rule. 
# replica_config = ReplicationRule(
     # prefix_list=prefix_list,
     # Specify the operations that can be replicated to the destination bucket. The default value is ALL, which indicates that all operations performed on objects in the source bucket are replicated to the destination bucket. 
     # action_list=[ReplicationRule.ALL],
     # Specify the destination bucket to which you want to replicate data. 
     # target_bucket_name='destexamplebucket1',
     # Specify the region in which the destination bucket is located. 
     # target_bucket_location='yourTargetBucketLocation',
     # By default, historical data is replicated. In this example, this parameter is set to False. This indicates that historical data is not replicated. 
     # is_enable_historical_object_replication=False,
     # Specify the link that is used to transfer data during data replication. 
     # target_transfer_type='oss_acc',
     # Specify the role that you want to authorize OSS to use to replicate data. If you want to use SSE-KMS to encrypt the objects that are replicated to the destination bucket, you must specify this parameter. 
     # sync_role_name='roleNameTest',
     # Replicate the objects that are encrypted by using SSE-KMS. 
     # sse_kms_encrypted_objects_status=ReplicationRule.ENABLED
     # Specify the customer master key (CMK) ID that is used in SSE-KMS. If you want to use SSE-KMS to encrypt the objects that are replicated to the destination bucket, you must specify this parameter. 
     # replica_kms_keyid='9468da86-3509-4f8d-a61e-6eab1eac****',
  #)

# Enable data replication. 
bucket.put_bucket_replication(replica_config)
package main

import (
    "fmt"
    "github.com/aliyun/aliyun-oss-go-sdk/oss"
    "os"
)

func HandleError(err error) {
    fmt.Println("Error:", err)
    os.Exit(-1)
}
// Enable data replication. 
func main()  {
    // Specify the endpoint of the region in which the source bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify your actual endpoint. 
    // The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
    client, err := oss.New("yourEndpoint", "yourAccessKeyId", "yourAccessKeySecret")
    if err != nil {
        HandleError(err)
    }
    // Specify the name of the source bucket. 
    srcbucketName := "srcexamplebucket"

    // Specify that only the data that is written to the bucket after the replication rule is created is replicated. The historical data of the source bucket is not replicated. 
    putXml := `<?xml version="1.0" encoding="UTF-8"?>
    <ReplicationConfiguration>
      <Rule>
        <PrefixSet>
            <! -- Specify that objects whose names contain one of the following prefixes are replicated to the destination bucket: prefix_1 and prefix_2. After you specify the prefixes, only objects whose names contain one of the prefixes are replicated to the destination bucket. -->
            <! -- To replicate all objects from the source bucket to the destination bucket, do not specify a prefix. 
            <Prefix>prefix_1</Prefix>
            <Prefix>prefix_2</Prefix>
        </PrefixSet>
        <! -- Specify the operations that can be replicated to the destination bucket. The default value is ALL, which indicates that all operations performed on objects in the source bucket are replicated to the destination bucket. -->
        <Action>ALL</Action>
        <Destination>
            <! -- Specify the destination bucket to which you want to replicate data. -->
            <Bucket>destexamplebucket</Bucket>
            <! -- Specify the region in which the destination bucket is located. -->
            <! -- If you want to enable CRR, the source and destination buckets must be located in different regions. If you want to enable SRR, the source and destination buckets must be located in the same region. -->
            <Location>oss-cn-beijing</Location>
            <! -- Specify the link that is used to transfer data during data replication. In this example, this parameter is set to oss_acc. This indicates that the link used to transfer data is accelerated. -->
            <TransferType>oss_acc</TransferType>
        </Destination>
        <! -- By default, historical data is replicated. In this example, this parameter is set to disabled. This indicates that historical data is not replicated. -->
        <HistoricalObjectReplication>disabled</HistoricalObjectReplication>
        <! -- Specify the role that you want to authorize OSS to use to replicate data. If you want to use SSE-KMS to encrypt the objects that are replicated to the destination bucket, you must specify this parameter. -->
        <SyncRole>aliyunramrole</SyncRole>
        <SourceSelectionCriteria>
          <SseKmsEncryptedObjects>
            <! -- Specify whether to replicate the objects that are encrypted by using SSE-KMS. -->
            <Status>Enabled</Status>
          </SseKmsEncryptedObjects>
        </SourceSelectionCriteria>
        <EncryptionConfiguration>
            <! -- Specify the customer master key (CMK) ID that is used in SSE-KMS. If Status is set to Enabled, you must specify this parameter. -->
           <ReplicaKmsKeyID>c4d49f85-ee30-426b-a5ed-95e9139d****</ReplicaKmsKeyID>
       </EncryptionConfiguration>
     </Rule>
   </ReplicationConfiguration>`

    err = client.PutBucketReplication(srcbucketName,putXml)
    if err != nil {
        HandleError(err)
    }
}            

Use ossutil

You can use ossutil to enable CRR. For more information, see replication.

Use RESTful APIs

If your business requires a high level of customization, you can directly call RESTful APIs. To directly call an API, you must include the signature calculation in your code. For more information, see PutBucketReplication.

FAQ

Does SSR synchronize lifecycle rule-based deletions from the source bucket to the destination bucket?

It depends on whether the replication policy of the SSR rule is Add/Change or Add/Delete/Change.

  • Add/Change: When objects are deleted from the source bucket based on a lifecycle rule, OSS does not delete their copies from the destination bucket.

  • Add/Delete/Change: When objects are deleted from the source bucket based on a lifecycle rule, OSS deletes their copies from the destination bucket.

    Note

    In the destination bucket, you may find objects with the same names as those that are deleted from the source bucket based on a lifecycle rule. This does not indicate that the Add/Delete/Change replication policy fails to take effect. The same-name objects in the destination bucket are not real copies of objects that are deleted from the source bucket. They may be separately written to the destination bucket and given the same names as the objects that have been deleted from the source bucket based on a lifecycle rule.

Do versioning-suspended buckets support CRR?

No, CRR is not supported by versioning-suspended buckets. You can configure CRR between two unversioned buckets or between two versioning-enabled buckets.

Does OSS support chained replication?

No, OSS does not support chained replication. For example, if a data replication rule is configured to replicate data from Bucket A to Bucket B and another data replication rule is configured to replicate data from Bucket B to Bucket C, data in bucket A is only replicated to Bucket B and is not replicated to Bucket C.

If you want to replicate data from Bucket A to Bucket C, you must configure a data replication rule to replicate data from Bucket A to Bucket C.

Why is the replication progress of historical data displayed as 0% for a long period of time?

The replication progress of historical data is not updated in real time. You must wait until all objects are scanned. If a large number of objects are stored in your bucket, such as hundreds of millions of objects, several hours is required before the replication progress of historical data is updated. If the replication progress of historical data is not updated, it does not mean that historical data is not replicated to the destination bucket.

You can check whether historical data in the source bucket is replicated to the destination bucket by viewing the storage capacity of the destination bucket and traffic usage, such as inbound and outbound traffic. For more information about how to view the storage capacity of the destination bucket and CRR traffic, see View the resource usage of a bucket.

If the destination bucket uses KMS to encrypt data, am I charged for the encryption algorithm operations?

If the destination bucket uses KMS to encrypt data, you are charged for encryption algorithm operations. For more information about the fees, see Billing of KMS.

Can I disable CRR?

Yes, you can click Disable Replication in the Actions column on the Cross-Region Replication page to disable CRR.

After you disable CRR, the replicated data is stored in the destination bucket. The incremental data in the source bucket is not replicated to the destination bucket.