Implement cross-region replication for a bucket by using Python SDK - Object Storage Service

Data replication asynchronously copies objects from a source bucket to a destination bucket. It also replicates operations such as object creation, updates, and deletions in near real-time. Object Storage Service (OSS) supports cross-region replication (CRR) and same-region replication (SRR).

Usage notes

In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS from other Alibaba Cloud services in the same region as OSS, use an internal endpoint. For more information about OSS regions and endpoints, see Regions and endpoints.
In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials using OSS SDK for Python 1.0.
In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Initialization.
To enable data replication, you must have the oss:PutBucketReplication permission. To query data replication rules, you must have the oss:GetBucketReplication permission. To query the regions to which data can be replicated, you must have the oss:GetBucketReplicationLocation permission. To query the progress of a data replication task, you must have the oss:GetBucketReplicationProgress permission. To disable data replication, you must have the oss:DeleteBucketReplication permission. For more information, see Attach a custom policy to a RAM user.

Enable data replication

Important

Before you enable data replication, make sure that the source and destination buckets are both unversioned or have versioning enabled.

The following code shows how to enable data replication. It replicates data from srcexamplebucket in the China (Hangzhou) region to destexamplebucket in the same or a different region.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import ReplicationRule
# Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region information that corresponds to the endpoint, for example, cn-hangzhou. Note that this parameter is required for V4 signatures.
region = "cn-hangzhou"

# Specify the bucket name, for example, examplebucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

replica_config = ReplicationRule(
    # Specify the destination bucket to which you want to replicate data.
    target_bucket_name='destexamplebucket',
    # Specify the region where the destination bucket is located.
    # If you want to enable cross-region replication, the source and destination buckets must be in different regions. If you want to enable same-region replication, the source and destination buckets must be in the same region.
    target_bucket_location='yourTargetBucketLocation'
)

# Specify the prefixes of the objects to replicate. After you specify prefixes, only objects that match the prefixes are replicated to the destination bucket.
# prefix_list = ['prefix1', 'prefix2']
# Set the data replication rule.
# replica_config = ReplicationRule(
     # prefix_list=prefix_list,
     # Replicate object creation and update operations in the source bucket to the destination bucket.
     # action_list=[ReplicationRule.PUT],
     # Specify the destination bucket to which you want to replicate data.
     # target_bucket_name='destexamplebucket1',
     # Specify the region where the destination bucket is located.
     # target_bucket_location='yourTargetBucketLocation',
     # By default, historical data is replicated. In this example, this parameter is set to False to disable historical data replication.
     # is_enable_historical_object_replication=False,
     # Specify the data transmission link for data replication.
     # target_transfer_type='oss_acc',
     # Specify the role that OSS is authorized to use for data replication. This element is required if you use SSE-KMS to encrypt destination objects.
     # sync_role_name='roleNameTest',
     # Replicate objects that are created with SSE-KMS encryption.
     # sse_kms_encrypted_objects_status=ReplicationRule.ENABLED
     # Specify the SSE-KMS key ID. This element is required if you replicate objects that are created with SSE-KMS encryption.
     # replica_kms_keyid='9468da86-3509-4f8d-a61e-6eab1eac****',
  #)

# Enable data replication.
bucket.put_bucket_replication(replica_config)

View data replication rules

The following code shows how to view the data replication rules for examplebucket.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import ReplicationRule

# Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region information that corresponds to the endpoint, for example, cn-hangzhou. Note that this parameter is required for V4 signatures.
region = "cn-hangzhou"

# Specify the bucket name, for example, examplebucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# View the data replication rules.
result = bucket.get_bucket_replication()
# Print the returned information.
for rule in result.rule_list:
    print(rule.rule_id)
    print(rule.target_bucket_name)
    print(rule.target_bucket_location)

View destination regions available for replication

The following code shows how to view the list of destination regions to which data in examplebucket can be replicated.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import ReplicationRule

# Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region information that corresponds to the endpoint, for example, cn-hangzhou. Note that this parameter is required for V4 signatures.
region = "cn-hangzhou"

# Specify the bucket name, for example, examplebucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# View the destination regions available for replication.
result = bucket.get_bucket_replication_location()
for location in result.location_list:
    print(location)

View data replication progress

You can query the progress of historical data replication tasks and incremental data replication tasks.

The progress of historical data replication tasks is expressed as a percentage. You can query the progress of historical data replication tasks only for buckets for which historical data replication is enabled.
The progress of incremental data replication tasks is expressed as a point in time. Data that is stored in the source bucket before the point in time is replicated.

The following code shows how to view the data replication progress for the rule with the ID `test_replication_1` for examplebucket.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import ReplicationRule

# Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region information that corresponds to the endpoint, for example, cn-hangzhou. Note that this parameter is required for V4 signatures.
region = "cn-hangzhou"

# Specify the bucket name, for example, examplebucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# View the data replication progress.
# Specify the replication rule ID, for example, test_replication_1.
result = bucket.get_bucket_replication_progress('test_replication_1')
print(result.progress.rule_id)
# Check whether historical data replication is enabled.
print(result.progress.is_enable_historical_object_replication)
# Historical data replication progress.
print(result.progress.historical_object_progress)
# Real-time data replication progress.
print(result.progress.new_object_progress)

Disable data replication

You can disable the data replication relationship between a source bucket and a destination bucket by deleting the replication rule for the bucket.

The following code shows how to delete the replication rule with the ID `test_replication_1` from examplebucket.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import ReplicationRule

# Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the region information that corresponds to the endpoint, for example, cn-hangzhou. Note that this parameter is required for V4 signatures.
region = "cn-hangzhou"

# Specify the bucket name, for example, examplebucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# Disable data replication. After data replication is disabled, the objects that have been replicated to the destination bucket remain. However, subsequent changes to objects in the source bucket are no longer replicated to the destination bucket.
# Specify the replication rule ID, for example, test_replication_1.
result = bucket.delete_bucket_replication('test_replication_1')

References

For the complete sample code for data replication, see GitHub.
For more information about the API operation to enable data replication, see PutBucketReplication.
For more information about the API operation to view data replication rules, see GetBucketReplication.
For more information about the API operation to view destination regions available for replication, see GetBucketReplicationLocation.
For more information about the API operation to view data replication progress, see GetBucketReplicationProgress.
For more information about the API operation to disable data replication, see DeleteBucketReplication.