Object Storage Service:SRR across accounts

Prerequisites

• Create Bucket A in a region under Account A to serve as the source bucket for SRR. Record the UID of Account A, the name of Bucket A, and its region.

• Create Bucket B in the same region under Account B to serve as the destination bucket for SRR. Record the UID of Account B and the name of Bucket B.

Role authorization

An SRR task across accounts involves buckets from two different accounts. Therefore, you must configure the required trust policy and least privilege policy for a Resource Access Management (RAM) role.

1. Use Account A to perform the following operations.

   1. Create a service role.

      Important

      You can use a RAM user to create a role. The RAM user must have the following permissions: ram:CreateRole, ram:GetRole, ram:ListPoliciesForRole, and ram:AttachPolicyToRole. However, granting a RAM user permissions such as ram:CreateRole and ram:GetRole poses a high security threat. We recommend that you use your Alibaba Cloud account to create a RAM role and grant permissions to it. After the authorization is complete, the RAM user can directly use the RAM role created by the Alibaba Cloud account.

      During the role creation process, select Alibaba Cloud Service as the trusted entity type and Object Storage Service as the trusted entity name. For more information, see Create a service role.

      Note

      After the role is created, record the RAM role ARN from the Basic Information section for later use.

   2. Grant the role permissions to perform SRR across accounts on the source bucket.

      You can grant permissions to the role in either of the following ways.

      Grant a system policy to the RAM role

      Warning

      You can grant the AliyunOSSFullAccess system policy to the RAM role. The AliyunOSSFullAccess policy grants full permissions on all buckets under the current account by default. Use this policy with caution.

      Grant a custom policy to the RAM role

      Use a RAM policy to grant the RAM role the least privilege required for replication on the source bucket (src-bucket).

      Note

      When you use the policy, replace the source bucket name as needed.

      {
         "Version":"1",
         "Statement":[
            {
               "Effect":"Allow",
               "Action":[
                  "oss:ReplicateList",
                  "oss:ReplicateGet"
               ],
               "Resource":[
                  "acs:oss:*:*:src-bucket",          	
                  "acs:oss:*:*:src-bucket/*"
               ]
            }
         ]
      }

      If you want to replicate Key Management Service (KMS) encrypted objects to the destination bucket, you must also grant the AliyunKMSFullAccess system policy to the role. For more information, see Grant permissions to a RAM role.

2. Use Account B to grant the role permissions to receive replicated objects in the destination bucket.

   (Recommended) Method 1: Add a policy using the visual editor

   1. Log on to the OSS console.

   2. In the navigation pane on the left, click Buckets, and then click the destination bucket name dest-bucket.

   3. In the navigation pane on the left, choose Permission Control > Bucket Policy.

   4. On the Bucket Policy page, on the Add in GUI tab, click Receive Objects to Replicate.

   5. In the Receive Objects to Replicate panel, for Obtain UID and RAM Role From, select Obtain From Source Replication RAM Role ARN. For Source RAM Role ARN for Replication, enter the role ARN that you recorded in Step 1. For Authorization Purpose, select Cross-account SRR.

   6. Click Generate Policy.

   Method 2: Add by Syntax Policy

   1. In the navigation pane on the left, choose Permission Control > Bucket Policy.

   2. On the Bucket Policy page, on the Add by Syntax tab, click Edit.

   3. In the policy editor, enter the following bucket policy.

      Important

      • When you add a bucket policy using the JSON editor, the new policy overwrites the existing policy. Make sure the new policy includes the content of the existing policy. Otherwise, operations that rely on the existing policy may fail.

      • When you use the policy, replace the custom role name, the destination bucket name (dest-bucket), the UID of the account that owns the source bucket (137918634953xxxx), and the UID of the account that owns the destination bucket (111933544165xxxx) as needed. If the custom role name contains uppercase letters, convert them to lowercase. For example, if the created role is named AliyunOssDrsRole, you must convert it to aliyunossdrsrole. The UID must be the UID of an Alibaba Cloud account.

      {
         "Version":"1",
         "Statement":[
            {
              "Effect":"Allow",
              "Action":[
                  "oss:ReplicateList",
                  "oss:ReplicateGet",
                  "oss:ReplicatePut",
                  "oss:ReplicateDelete"			
               ],
              "Principal": [
                  "arn:sts::137918634953xxxx:assumed-role/aliyunossdrsrole/*"		 
      		 ],
               "Resource":[
                  "acs:oss:*:111933544165xxxx:dest-bucket",          	
                  "acs:oss:*:111933544165xxxx:dest-bucket/*"
               ]
            }
         ]
      }

   4. Click Save.

Replicate KMS-encrypted objects

If you want to replicate KMS-encrypted objects from the source bucket under Account A to the destination bucket under Account B, use Account B to perform the following steps.

1. Log on to the Instance Management page of the KMS console. Purchase and enable a KMS instance in the same region as the destination bucket. When you purchase the KMS instance, make sure that Access Management Quantity is 2 or greater. Keep the default configurations for the other parameters. For more information, see Purchase and enable a KMS instance.

   Note

   Replicating KMS-encrypted objects across accounts relies on KMS. The regions that support this feature are limited by KMS. For more information about the regions supported by KMS, see Regions and endpoints that support software-protected keys.

2. Create a key in the KMS instance. The key type must be a non-default key. We recommend that you use a software-protected key. For more information, see Create a software-protected key.

   Note

   After the key is created, record the key ARN from the Basic Information section for use when you create the replication rule.

3. Set a key policy for the created key. When you set the key policy, select Other Account User and specify the role ARN that you created in the preceding steps as the Other Account User. For more information, see Set a key policy.

   Important

   To replicate KMS-encrypted data across accounts, the key policy must include at least the decryption (kms:Decrypt) and generate data key (kms:GenerateDataKey) permissions. When you set a key policy in the console, these permissions are included by default. If you set a custom key policy using OpenAPI, make sure that the policy includes at least the kms:Decrypt and kms:GenerateDataKey permissions.

Procedure

import com.aliyun.oss.*;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.common.comm.SignVersion;
import com.aliyun.oss.model.AddBucketReplicationRequest;

public class Demo {

    public static void main(String[] args) throws Exception {
        // The following example uses the endpoint of the China (Hangzhou) region. Replace the endpoint with the actual endpoint.
        String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
        // Specify the region ID that corresponds to the endpoint. Example: cn-hangzhou.
        String region = "cn-hangzhou";
        // We strongly recommend that you do not hard-code your access credentials in your project code. This may lead to credential leakage and threaten the security of all resources in your account. The following example shows how to obtain access credentials from environment variables. Before you run the example code, configure the environment variables.
        EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
        // Specify the source bucket name. Example: src-bucket.
        String bucketName = "src-bucket";
        // Specify the destination bucket to which you want to replicate data. The destination bucket must belong to a different account than the source bucket.
        String targetBucketName = "dest-bucket";
        // Specify the region where the destination bucket is located. The destination bucket must be in the same region as the source bucket.
        String targetBucketLocation = "oss-cn-hangzhou";

        // Create an OSSClient instance.
        // When the OSSClient instance is no longer needed, call the shutdown method to release resources.
        ClientBuilderConfiguration clientBuilderConfiguration = new ClientBuilderConfiguration();
        // Explicitly declare the use of the V4 signature algorithm.
        clientBuilderConfiguration.setSignatureVersion(SignVersion.V4);
        OSS ossClient = OSSClientBuilder.create()
                .endpoint(endpoint)
                .credentialsProvider(credentialsProvider)
                .clientConfiguration(clientBuilderConfiguration)
                .region(region)
                .build();

        try {
            AddBucketReplicationRequest request = new AddBucketReplicationRequest(bucketName);

            request.setTargetBucketName(targetBucketName);
            request.setTargetBucketLocation(targetBucketLocation);
            // By default, historical data is replicated. This example sets the value to false to disable historical data replication.
            request.setEnableHistoricalObjectReplication(false);
            // Specify the name of the role that authorizes OSS to replicate data. This role must have been granted permissions to perform SRR on the source bucket and receive replicated objects in the destination bucket.
            request.setSyncRole("yourRole");           
            //List prefixes = new ArrayList();
            //prefixes.add("image/");
            //prefixes.add("video");
            //prefixes.add("a");
            //prefixes.add("A");
            // Specify the prefixes of the objects to replicate. After you specify prefixes, only objects that match the prefixes are replicated to the destination bucket.
            //request.setObjectPrefixList(prefixes);
            //List actions = new ArrayList();
            //actions.add(AddBucketReplicationRequest.ReplicationAction.PUT);
            // Replicates object creation and update operations from the source bucket to the destination bucket.
            //request.setReplicationActionList(actions);
            ossClient.addBucketReplication(request);
        } catch (OSSException oe) {
            System.out.println("Caught an OSSException, which means your request made it to OSS, "
                    + "but was rejected with an error response for some reason.");
            System.out.println("Error Message:" + oe.getErrorMessage());
            System.out.println("Error Code:" + oe.getErrorCode());
            System.out.println("Request ID:" + oe.getRequestId());
            System.out.println("Host ID:" + oe.getHostId());
        } catch (ClientException ce) {
            System.out.println("Caught an ClientException, which means the client encountered "
                    + "a serious internal problem while trying to communicate with OSS, "
                    + "such as not being able to access the network.");
            System.out.println("Error Message:" + ce.getMessage());
        } finally {
            if (ossClient != null) {
                ossClient.shutdown();
            }
        }
    }
}        

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import ReplicationRule
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
auth = oss2.ProviderAuth(EnvironmentVariableCredentialsProvider())
# Specify the endpoint of the region where the source bucket is located. For example, if the source bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
# Specify the source bucket name. Example: src-bucket.
bucket = oss2.Bucket(auth, 'https://oss-cn-hangzhou.aliyuncs.com', 'src-bucket')
replica_config = ReplicationRule(
    # Specify the destination bucket to which you want to replicate data. The destination bucket must belong to a different account than the source bucket.
    target_bucket_name='dest-bucket',
    # Specify the region where the destination bucket is located. The destination bucket must be in the same region as the source bucket.
    target_bucket_location='oss-cn-hangzhou',
    # Specify the name of the role that authorizes OSS to replicate data. This role must have been granted permissions to perform SRR on the source bucket and receive replicated objects in the destination bucket.
    sync_role_name='yourRole',
)

# Specify the prefixes of the objects to replicate. After you specify prefixes, only objects that match the prefixes are replicated to the destination bucket.
# prefix_list = ['prefix1', 'prefix2']
# Set the data replication rule.
# replica_config = ReplicationRule(
     # prefix_list=prefix_list,
     # Replicates object creation and update operations from the source bucket to the destination bucket.
     # action_list=[ReplicationRule.PUT],
     # Specify the destination bucket to which you want to replicate data. The destination bucket must belong to a different account than the source bucket.
     # target_bucket_name='dest-bucket',
     # Specify the region where the destination bucket is located. The destination bucket must be in the same region as the source bucket.
     # target_bucket_location='oss-cn-hangzhou',
     # By default, historical data is replicated. This example sets the value to False to disable historical data replication.
     # is_enable_historical_object_replication=False,      
  #)

# Enable data replication.
bucket.put_bucket_replication(replica_config)

package main

import (
    "encoding/xml"
    "fmt"
    "github.com/aliyun/aliyun-oss-go-sdk/oss"
    "os"
)

func HandleError(err error) {
    fmt.Println("Error:", err)
    os.Exit(-1)
}

// Enable data replication.
func main() {
    // Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
    provider, err := oss.NewEnvironmentVariableCredentialsProvider()
    if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
    }
    // Create an OSSClient instance.
    // Replace yourEndpoint with the endpoint of the bucket. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Replace the endpoint with the actual endpoint.
    client, err := oss.New("yourEndpoint", "", "", oss.SetCredentialsProvider(&provider))
    if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
    }
    // Specify the source bucket name.
    srcbucketName := "yourSrcBucket"
    // Specify the destination bucket to which you want to replicate data. The destination bucket must belong to a different account than the source bucket.
    destBucketName := "yourDestBucket"
    // Specify the prefixes of the objects to replicate, such as prefix_1 and prefix_2. After you specify prefixes, only objects that match the prefixes are replicated to the destination bucket.
    // To replicate all objects from the source bucket to the destination bucket, you do not need to set a prefix.
    prefix1 := "prefix_1"
    prefix2 := "prefix_2"    
    prefixSet := oss.ReplicationRulePrefix{Prefix: []*string{&prefix1, &prefix2}}
    
    reqReplication := oss.PutBucketReplication{
    Rule: []oss.ReplicationRule{
    {
    PrefixSet: &prefixSet,
    //Replicates object creation and update operations from the source bucket to the destination bucket.
    Action: "PUT",    
    Destination: &oss.ReplicationRuleDestination{
    Bucket: destBucketName,
    // Specify the region where the destination bucket is located. The source bucket and destination bucket must be in the same region.    
    Location: "oss-cn-hangzhou",    
    },
    // By default, historical data is replicated. This example sets the value to disabled to disable historical data replication.
    HistoricalObjectReplication: "disabled",
    // Specify the name of the role that authorizes OSS to replicate data. This role must have been granted permissions to perform SRR across accounts on the source bucket and receive replicated objects in the destination bucket.
    SyncRole:                    "yourRole",    
    },
    },
    }

    xmlBody, err := xml.Marshal(reqReplication)
    if err != nil {
    HandleError(err)
    }
    err = client.PutBucketReplication(srcbucketName, string(xmlBody))

    if err != nil {
    HandleError(err)
    }

    fmt.Println("Put Bucket Replication Success!")
}

References

• Same-region replication

• SRR within the same account

• Replication behavior in special scenarios

• Data replication FAQ