This topic answers frequently asked questions about same-account and cross-account data replication in Object Storage Service (OSS), including same-region and cross-region scenarios, and provides troubleshooting guidance.
Does OSS regional replication support custom paths or prefixes for the destination bucket?
No. Neither same-region replication nor cross-region replication supports custom storage paths for the destination bucket. The destination path is determined by the source data path that you configure in the data replication rule.
Does OSS regional replication support syncing objects based on their modification time?
No. You cannot configure a data replication rule to sync objects based on a specified modification time range. However, you can use the File Modification Time feature in Data Transport to achieve this. If you specify a time range, Data Transport migrates only the files modified within that period.
Why can't I create a data replication rule?
Check whether you have the required permissions.
Missing RAM user permissions
When you use the console as a RAM user to create a data replication rule, you cannot click OK.
Cause: The oss:PutBucketReplication permission is missing.
Solution: Grant the oss:PutBucketReplication permission to the RAM user.
When you use the console as a RAM user to create a data replication rule, the RAM Role Name list does not display custom RAM roles.
Cause: The ram:ListRoles permission is missing.
Solution: Grant the ram:ListRoles permission to the RAM user.
For more information, see Attach a custom policy to a RAM user.
Missing RAM role permissions
Same-account replication
For same-account replication, you must grant the RAM role permissions to replicate data between the source and destination buckets. For more information about how to create and authorize a RAM role, see Choose a RAM role.
Cross-account replication
For cross-account replication, you must grant permissions in both the source and destination accounts. The source account must grant the RAM role permission to replicate data from the source bucket, and the destination account must grant it permission to receive objects in the destination bucket. For more information about how to create and authorize a RAM role, see Methods.
Check if the versioning status of the buckets is consistent.
The source and destination buckets must have the same versioning status (either enabled or disabled).
Check if the endpoint or access key information is correct.
When you create a data replication rule by using an SDK or ossutil, check the following configurations:
Verify that the endpoints for the regions of the source and destination buckets are correct. For more information, see Regions and endpoints.
Verify that the access key used to establish the replication relationship is correct. For more information, see View access key pair details.
Why aren't objects replicated to the destination bucket?
If objects are not replicating to the destination bucket after you have configured a data replication rule, check the following possible causes.
Verify the source bucket configuration.
Check whether the data replication status is Enabled.
Check whether the prefix is correct.
Replicate specific objects: To replicate specific objects from the source bucket to the destination bucket, set the prefix to the desired object prefix. For example, if you set the prefix to
log, only objects with names starting withlog, such aslog/date1.txtandlog/date2.txt, are replicated. Objects that do not match the prefix, such asdate3.txt, are not replicated.NoteThe prefix cannot end with an asterisk (*), such as
log/*, and it cannot contain the bucket name.Replicate all objects: To replicate all objects from the source bucket to the destination bucket, leave the prefix empty.
Check whether you have configured replication for existing objects. Existing objects are not replicated unless this option is enabled.
Check whether the objects in the source bucket were replicated from another bucket.
If an object in a bucket is a replica created by another replication configuration, Object Storage Service (OSS) does not replicate that object again. For example, if you configure replication from bucket A to bucket B, and then from bucket B to bucket C, OSS does not replicate objects from bucket B to bucket C that were originally replicated from bucket A.
Check if the objects are encrypted with Key Management Service (KMS). If they are, you must enable replication for KMS-encrypted objects.
If server-side encryption with KMS-managed keys (SSE-KMS) is used for the source objects or the destination bucket, you must enable replication of KMS-encrypted objects and configure the following parameters:
KMS Key: Specify the KMS key to encrypt the destination objects.
You must create a KMS key in the same region as the destination bucket in advance. For more information, see Create a CMK.
RAM Role: Authorize a RAM role to perform KMS encryption on the destination objects.
New RAM Role: Create a new RAM role to perform KMS encryption on the destination objects. The role name uses the format
kms-replication-SourceBucketName-DestinationBucketName.AliyunOSSRole: Use the AliyunOSSRole role to perform KMS encryption on the destination objects. If you have not created the AliyunOSSRole role, OSS automatically creates it when you select this option.
NoteIf you create a role or modify role permissions, attach the
AliyunOSSFullAccesspolicy to the role. Otherwise, data replication may fail.
You can call the HeadObject and GetBucketEncryption operations to query the encryption status of the source object and destination bucket, respectively.
Check if the replication progress has reached 100%.
Data replication is asynchronous and occurs in near real time. The time required to replicate data to the destination bucket can range from a few minutes to several hours, depending on the amount of data. If you are replicating large objects, wait for the replication progress to reach 100% before you check whether the objects appear in the destination bucket.
Why are objects not deleted from the destination bucket after they are deleted from the source bucket?
Reason 1: The destination bucket has a retention policy enabled.
Before the retention period of the retention policy expires, no user, including the resource owner, can delete objects from the bucket.
Reason 2: Whether objects are deleted from the destination bucket depends on the versioning status of the source bucket and the configured data replication policy.
Source bucket versioning status
Request type
Data replication policy
Result
Versioning disabled
Delete request is initiated
Add/Change
Objects are deleted only from the source bucket, not from the destination bucket.
Add/Delete/Change
Objects are deleted from both the source and destination buckets.
Versioning enabled
Delete request is initiated without specifying an object version ID
Add/Change
Objects are not deleted from either bucket. Instead, OSS creates a delete marker in the source bucket, and this delete marker is replicated to the destination bucket.
Add/Delete/Change
Delete request is initiated with a specific object version ID
Add/Change
The object is deleted only from the source bucket, not from the destination bucket.
Add/Delete/Change
The object is deleted from both the source and destination buckets.
How do I verify data consistency between the source and destination buckets after replication is complete?
You can run the following code to verify data consistency between the source and destination buckets after replication is complete.
import com.aliyun.oss.OSSClient;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.model.*;
import com.aliyun.oss.OSSException;
import com.aliyuncs.exceptions.ClientException;
public class Demo {
public static void main(String[] args) throws ClientException {
// Obtain access credentials from environment variables. Before you run this code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
// Replace srcEndpoint with the endpoint of the region where the source bucket is located.
String srcEndpoint = "https://oss-cn-hangzhou.aliyuncs.com";
OSSClient srcClient = new OSSClient(srcEndpoint , credentialsProvider);
// Specify the name of the source bucket.
String srcBucketName = "src-replication-bucket";
// Replace destEndpoint with the endpoint of the region where the destination bucket is located.
String destEndpoint = "https://oss-cn-beijing.aliyuncs.com";
OSSClient destClient = new OSSClient(destEndpoint, credentialsProvider);
// Specify the name of the destination bucket.
String destBucketName = "dest-replication-bucket";
// If the source and destination buckets do not have versioning enabled, call listObjectsV2 to list the objects in the source bucket.
// If versioning is enabled or suspended for the source and destination buckets, call listVersions to list the objects in the source bucket.
ListObjectsV2Result result;
ListObjectsV2Request request = new ListObjectsV2Request(srcBucketName);
do {
result = srcClient.listObjectsV2(request);
for (OSSObjectSummary summary : result.getObjectSummaries())
{
String objectName = summary.getKey();
ObjectMetadata srcMeta;
try {
// Get the metadata of the source object.
srcMeta = srcClient.headObject(srcBucketName, objectName);
} catch (OSSException ossException) {
if (ossException.getErrorCode().equals("NoSuchKey")) {
continue;
} else {
System.out.println("head src-object failed: " + objectName);
}
continue;
}
ObjectMetadata destMeta;
try {
// Get the metadata of the replicated object in the destination bucket.
destMeta = destClient.headObject(destBucketName, objectName);
} catch (OSSException ossException) {
if (ossException.getErrorCode().equals("NoSuchKey")) {
System.out.println("dest-object not exist: " + objectName);
} else {
System.out.println("head dest-object failed: " + objectName);
}
continue;
}
// Check if the CRC values of the source and destination objects are consistent.
Long srcCrc = srcMeta.getServerCRC();
String srcMd5 = srcMeta.getContentMD5();
if (srcCrc != null) {
if (destMeta.getServerCRC() != null) {
if (!destMeta.getServerCRC().equals(srcCrc)) {
System.out.println("crc not equal: " + objectName
+ " | srcCrc: " + srcCrc + " | destCrc: " + destMeta.getServerCRC());
}
continue;
}
}
// Check if the MD5 hashes of the source and destination objects are consistent.
if (srcMd5!= null) {
if (destMeta.getContentMD5() != null) {
if (!destMeta.getContentMD5().equals(srcMd5)) {
System.out.println("md5 not equal: " + objectName
+ " | srcMd5: " + srcMd5 + " | destMd5: " + destMeta.getContentMD5());
}
continue;
}
}
// Check if the ETags of the source and destination objects are consistent.
if (srcMeta.getETag() == null || !srcMeta.getETag().equals(destMeta.getETag())) {
System.out.println("etag not equal: " + objectName
+ " | srcEtag: " + srcMeta.getETag() + " | destEtag: " + destMeta.getETag());
}
}
request.setContinuationToken(result.getNextContinuationToken());
request.setStartAfter(result.getStartAfter());
} while (result.isTruncated());
}
}
Is chained replication supported?
No. For example, if you configure a data replication rule to replicate data from bucket A to bucket B, and another rule to replicate data from bucket B to bucket C, data from bucket A is replicated only to bucket B, not to bucket C.
To replicate data from bucket A to bucket C, you must also configure a data replication rule from bucket A to bucket C.
If both bucket A and bucket B have historical object replication enabled, and historical replication is still in progress, the historical replication job may scan some new data written to bucket A and then replicate it to bucket C.
Does two-way replication between OSS buckets create a risk of circular replication?
No. If you configure two-way replication between bucket A and bucket B, data replicated from bucket A to bucket B is not replicated back from B to A. The same rule applies to data replicated from B to A.
If objects in the source bucket are deleted by a lifecycle rule, are they also deleted from the destination bucket?
If the data replication policy for the source bucket is set to Add/Change, objects deleted from the source bucket by a lifecycle rule are not deleted from the destination bucket.
If the data replication policy for the source bucket is set to Add/Delete/Change, objects deleted from the source bucket by a lifecycle rule are also deleted from the destination bucket.
NoteIf you find an object in the destination bucket that was deleted from the source bucket by a lifecycle rule, it does not mean the Add/Delete/Change policy is ineffective. You might have manually written an object with the same name to the destination bucket.
Why does the replication progress for existing data stay at 0% for a long time?
Replication progress is not updated in real time
OSS updates the replication progress for existing data only after scanning all objects. If your bucket contains a large number of objects, such as hundreds of millions, it may take several hours for the progress to update. A progress of 0% does not necessarily mean that replication has not started.
You can check whether existing data from the source bucket has started replicating to the destination bucket by monitoring changes in the destination bucket's storage capacity and replication traffic. For more information about how to monitor these metrics, see View the resource usage of a bucket.
Incorrect permission policy for the source bucket
OSS does not currently validate the source bucket's permissions when you create a data replication rule. This allows you to create the rule successfully, but no data replicates to the destination bucket, causing the replication progress to remain at 0%.
To configure the correct permission policy, see Required permissions for data replication.
What can I do if data replication is slow?
Increase bandwidth
Data replication is asynchronous and occurs in near real time. The time required to transfer data from the source bucket to the destination bucket can range from a few minutes to several hours, depending on the data volume. If replication takes too long, check for bandwidth limitations that may be causing delays. If bandwidth is the issue, contact technical support to request a bandwidth increase and improve replication performance.
Enable Replication Time Control (RTC)
When you enable Replication Time Control (RTC), OSS replicates most objects within seconds and 99.99% of objects within 10 minutes. After you enable RTC, you incur charges for the data replication traffic generated by RTC-enabled tasks. For more information, see Use Replication Time Control (RTC).
How can I track the replication operations performed on the source and destination buckets?
You can configure an event notification rule and set the event types to ObjectReplication:ObjectCreated, ObjectReplication:ObjectRemoved, and ObjectReplication:ObjectModified. This allows you to receive notifications about object changes in the source and destination buckets during replication, such as object creation, updates, deletions, and overwrites. For more information, see Use event notifications to monitor object changes in real time.
Is data replication supported for buckets with versioning suspended?
No. You can only configure data replication between two buckets that both have versioning disabled or both have versioning enabled.
If the destination bucket uses KMS encryption, am I charged for KMS cryptographic API calls?
Yes. If the destination bucket uses KMS encryption, you incur charges for KMS cryptographic API calls. For more information, see Billing of KMS.
Can I disable a data replication rule after it is enabled?
Yes. You can stop data replication by clicking Disable Replication next to the rule.
After you disable the rule, the destination bucket retains all previously replicated data, but OSS no longer replicates new data from the source bucket.
Does replication change the order of objects?
No, replication does not change the modification order of objects.