Implement cross-region replication for a bucket by using Python SDK - Object Storage Service

Data replication asynchronously copies objects and their creation, update, and deletion operations from a source bucket to a destination bucket in near real-time. OSS supports cross-region replication (CRR) and same-region replication (SRR).

Usage notes

In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS from other Alibaba Cloud services in the same region as OSS, use an internal endpoint. For more information about OSS regions and endpoints, see Regions and Endpoints.
In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials using OSS SDK for Python 1.0.
This topic demonstrates creating an OSSClient instance with an OSS endpoint. For alternative configurations, such as using a custom domain or authenticating with credentials from Security Token Service (STS), see Initialization.
To enable data replication, you must have the oss:PutBucketReplication permission. To query data replication rules, you must have the oss:GetBucketReplication permission. To query the regions to which data can be replicated, you must have the oss:GetBucketReplicationLocation permission. To query the progress of a data replication task, you must have the oss:GetBucketReplicationProgress permission. To disable data replication, you must have the oss:DeleteBucketReplication permission. For more information, see Grant a custom policy.

Enable data replication

Important

Before you enable data replication, make sure that the source and destination buckets are both unversioned or have versioning enabled.

The following code enables data replication from srcexamplebucket in the China (Hangzhou) region to destexamplebucket in the same or a different region.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import ReplicationRule
# Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region information that corresponds to the endpoint, for example, cn-hangzhou. Note that this parameter is required for V4 signatures.
region = "cn-hangzhou"

# Specify the bucket name, for example, examplebucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

replica_config = ReplicationRule(
    # Specify the destination bucket to which you want to replicate data.
    target_bucket_name='destexamplebucket',
    # Specify the region where the destination bucket is located.
    # If you want to enable cross-region replication, the source and destination buckets must be in different regions. If you want to enable same-region replication, the source and destination buckets must be in the same region.
    target_bucket_location='yourTargetBucketLocation'
)

# Specify the prefixes of the objects to replicate. After you specify prefixes, only objects that match the prefixes are replicated to the destination bucket.
# prefix_list = ['prefix1', 'prefix2']
# Set the data replication rule.
# replica_config = ReplicationRule(
     # prefix_list=prefix_list,
     # Replicate object creation and update operations in the source bucket to the destination bucket.
     # action_list=[ReplicationRule.PUT],
     # Specify the destination bucket to which you want to replicate data.
     # target_bucket_name='destexamplebucket1',
     # Specify the region where the destination bucket is located.
     # target_bucket_location='yourTargetBucketLocation',
     # By default, historical data is replicated. In this example, this parameter is set to False to disable historical data replication.
     # is_enable_historical_object_replication=False,
     # Specify the data transmission link for data replication.
     # target_transfer_type='oss_acc',
     # Specify the role that OSS is authorized to use for data replication. This element is required if you use SSE-KMS to encrypt destination objects.
     # sync_role_name='roleNameTest',
     # Replicate objects that are created with SSE-KMS encryption.
     # sse_kms_encrypted_objects_status=ReplicationRule.ENABLED
     # Specify the SSE-KMS key ID. This element is required if you replicate objects that are created with SSE-KMS encryption.
     # replica_kms_keyid='9468da86-3509-4f8d-a61e-6eab1eac****',
  #)

# Enable data replication.
bucket.put_bucket_replication(replica_config)

View data replication rules

The following code queries the data replication rules for examplebucket.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import ReplicationRule

# Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region information that corresponds to the endpoint, for example, cn-hangzhou. Note that this parameter is required for V4 signatures.
region = "cn-hangzhou"

# Specify the bucket name, for example, examplebucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# View the data replication rules.
result = bucket.get_bucket_replication()
# Print the returned information.
for rule in result.rule_list:
    print(rule.rule_id)
    print(rule.target_bucket_name)
    print(rule.target_bucket_location)

View destination regions available for replication

The following code lists the destination regions to which data in examplebucket can be replicated.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import ReplicationRule

# Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region information that corresponds to the endpoint, for example, cn-hangzhou. Note that this parameter is required for V4 signatures.
region = "cn-hangzhou"

# Specify the bucket name, for example, examplebucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# View the destination regions available for replication.
result = bucket.get_bucket_replication_location()
for location in result.location_list:
    print(location)

View data replication progress

You can query the progress of historical data replication tasks and incremental data replication tasks.

The progress of historical data replication tasks is expressed as a percentage. You can query the progress of historical data replication tasks only for buckets for which historical data replication is enabled.
The progress of incremental data replication tasks is expressed as a point in time. Data that is stored in the source bucket before the point in time is replicated.

The following code queries the data replication progress for the rule with the ID `test_replication_1` in examplebucket.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import ReplicationRule

# Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region information that corresponds to the endpoint, for example, cn-hangzhou. Note that this parameter is required for V4 signatures.
region = "cn-hangzhou"

# Specify the bucket name, for example, examplebucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# View the data replication progress.
# Specify the replication rule ID, for example, test_replication_1.
result = bucket.get_bucket_replication_progress('test_replication_1')
print(result.progress.rule_id)
# Check whether historical data replication is enabled.
print(result.progress.is_enable_historical_object_replication)
# Historical data replication progress.
print(result.progress.historical_object_progress)
# Real-time data replication progress.
print(result.progress.new_object_progress)

Disable data replication

To disable data replication between a source bucket and a destination bucket, delete the replication rule for the bucket.

The following code deletes the replication rule with the ID `test_replication_1` from examplebucket.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import ReplicationRule

# Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region information that corresponds to the endpoint, for example, cn-hangzhou. Note that this parameter is required for V4 signatures.
region = "cn-hangzhou"

# Specify the bucket name, for example, examplebucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# Disable data replication. After this operation, replicated objects remain in the destination bucket, but new changes from the source bucket stop replicating.
# Specify the replication rule ID, for example, test_replication_1.
result = bucket.delete_bucket_replication('test_replication_1')

References

For the complete sample code for data replication, see GitHub.
For more information about the API operation to enable data replication, see PutBucketReplication.
For more information about the API operation to view data replication rules, see GetBucketReplication.
For more information about the API operation to view destination regions available for replication, see GetBucketReplicationLocation.
For more information about the API operation to view data replication progress, see GetBucketReplicationProgress.
For more information about the API operation to disable data replication, see DeleteBucketReplication.