Implement cross-region replication for a bucket by using Python SDK - Object Storage Service

Data replication asynchronously copies objects from a source bucket to a destination bucket. It replicates object creation, updates, and deletions in near real-time. Object Storage Service (OSS) supports cross-region replication (CRR) and same-region replication (SRR).

Prerequisites

Before you begin, make sure that you have:

The required permissions for each operation: For more information, see Attach a custom policy to a RAM user.

Operation	Required permission
Enable data replication	`oss:PutBucketReplication`
Query replication rules	`oss:GetBucketReplication`
Query available destination regions	`oss:GetBucketReplicationLocation`
Query replication progress	`oss:GetBucketReplicationProgress`
Disable data replication	`oss:DeleteBucketReplication`

The OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables configured. For more information, see Configure access credentials using OSS SDK for Python 1.0.
An OSS endpoint for the region where your bucket is located. To access OSS from other Alibaba Cloud services in the same region, use an internal endpoint. For more information, see Regions and endpoints.
(Optional) A custom domain name or Security Token Service (STS) configuration if not using an OSS endpoint directly. For more information, see Initialization.

Enable data replication

Important

Before you enable data replication, make sure that the source and destination buckets are both unversioned or both have versioning enabled.

The following example enables data replication from examplebucket in the China (Hangzhou) region to destexamplebucket in the same or a different region.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import ReplicationRule

# Obtain access credentials from environment variables.
# Make sure that OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint for the China (Hangzhou) region.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Specify the region. Required for V4 signatures.
region = "cn-hangzhou"

# Specify the source bucket name.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

replica_config = ReplicationRule(
    # Specify the destination bucket name.
    target_bucket_name='destexamplebucket',
    # Specify the destination bucket region.
    # CRR: source and destination must be in different regions.
    # SRR: source and destination must be in the same region.
    target_bucket_location='yourTargetBucketLocation'
)

# Enable data replication.
bucket.put_bucket_replication(replica_config)
print("Data replication enabled successfully.")

ReplicationRule parameters

Parameter	Type	Required	Description
`target_bucket_name`	str	Yes	Name of the destination bucket
`target_bucket_location`	str	Yes	Region of the destination bucket
`prefix_list`	list	No	Object name prefixes to replicate. Only objects matching these prefixes are replicated.
`action_list`	list	No	Operations to replicate. For example, `[ReplicationRule.PUT]` replicates only creation and update operations.
`is_enable_historical_object_replication`	bool	No	Whether to replicate historical data. Default: `True`.
`target_transfer_type`	str	No	Transfer type. Set to `'oss_acc'` to use transfer acceleration.
`sync_role_name`	str	No	Role authorized for data replication. Required when using SSE-KMS to encrypt destination objects.
`sse_kms_encrypted_objects_status`	str	No	Set to `ReplicationRule.ENABLED` to replicate SSE-KMS-encrypted objects.
`replica_kms_keyid`	str	No	KMS key ID for the destination. Required when replicating SSE-KMS-encrypted objects.

The following example creates a replication rule with advanced options:

# Replicate only objects with the specified prefixes.
prefix_list = ['prefix1', 'prefix2']

replica_config = ReplicationRule(
    prefix_list=prefix_list,
    # Replicate only object creation and update operations.
    action_list=[ReplicationRule.PUT],
    target_bucket_name='destexamplebucket',
    target_bucket_location='yourTargetBucketLocation',
    # Disable historical data replication.
    is_enable_historical_object_replication=False,
    # Use transfer acceleration.
    target_transfer_type='oss_acc',
    # Specify the role authorized for data replication.
    # Required for SSE-KMS encryption.
    sync_role_name='roleNameTest',
    # Replicate SSE-KMS-encrypted objects.
    sse_kms_encrypted_objects_status=ReplicationRule.ENABLED,
    # Specify the KMS key ID for the destination.
    replica_kms_keyid='9468da86-3509-4f8d-a61e-6eab1eac****',
)

bucket.put_bucket_replication(replica_config)
print("Data replication enabled with advanced options.")

Query replication rules

The following example retrieves all data replication rules for examplebucket.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

# Obtain access credentials from environment variables.
# Make sure that OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint and region.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
region = "cn-hangzhou"

bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# Query replication rules.
result = bucket.get_bucket_replication()

# Print each rule's details.
for rule in result.rule_list:
    print("Rule ID:", rule.rule_id)
    print("Destination bucket:", rule.target_bucket_name)
    print("Destination region:", rule.target_bucket_location)

The response contains a rule_list. Each rule includes the following fields:

Field	Description
`rule_id`	Unique identifier of the replication rule
`target_bucket_name`	Name of the destination bucket
`target_bucket_location`	Region of the destination bucket

Query available destination regions

The following example lists the regions to which data in examplebucket can be replicated.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

# Obtain access credentials from environment variables.
# Make sure that OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint and region.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
region = "cn-hangzhou"

bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# Query available destination regions.
result = bucket.get_bucket_replication_location()

print("Available destination regions:")
for location in result.location_list:
    print(" -", location)

The response contains a location_list of region identifiers (for example, oss-cn-beijing) where data can be replicated.

Query replication progress

Replication progress is tracked in two ways:

Historical data replication -- expressed as a percentage. Available only for buckets with historical data replication enabled.
Incremental data replication -- expressed as a point in time. Data stored in the source bucket before this point in time has been replicated to the destination bucket.

The following example queries the replication progress for the rule with ID test_replication_1 on examplebucket.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

# Obtain access credentials from environment variables.
# Make sure that OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint and region.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
region = "cn-hangzhou"

bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# Query replication progress for a specific rule.
result = bucket.get_bucket_replication_progress('test_replication_1')

print("Rule ID:", result.progress.rule_id)
print("Historical replication enabled:", result.progress.is_enable_historical_object_replication)
print("Historical replication progress:", result.progress.historical_object_progress)
print("Incremental replication progress:", result.progress.new_object_progress)

The response contains a progress object with the following fields:

Field	Description
`rule_id`	Replication rule ID
`is_enable_historical_object_replication`	Whether historical data replication is enabled
`historical_object_progress`	Percentage of historical data replicated
`new_object_progress`	Point-in-time marker for incremental data replication

Disable data replication

Disabling data replication deletes the replication rule for a bucket. Objects already replicated to the destination bucket remain unchanged, but subsequent changes to source objects are no longer replicated.

The following example deletes the replication rule with ID test_replication_1 from examplebucket.

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

# Obtain access credentials from environment variables.
# Make sure that OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET are set.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint and region.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
region = "cn-hangzhou"

bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# Disable data replication by deleting the rule.
bucket.delete_bucket_replication('test_replication_1')
print("Data replication rule deleted. Existing replicated objects are preserved in the destination bucket.")

References

Complete sample code on GitHub
PutBucketReplication -- enable data replication
GetBucketReplication -- query replication rules
GetBucketReplicationLocation -- query available destination regions
GetBucketReplicationProgress -- query replication progress
DeleteBucketReplication -- disable data replication