Object Storage Service:Mirroring-based back-to-origin

Operation behavior

The following table shows how mirroring-based back-to-origin affects each storage operation.

Metadata preserved from origin

When OSS fetches an object from the origin and stores it in the bucket, note the following about the Last-Modified header:

OSS does not preserve the source object's Last-Modified timestamp. When comparing timestamps to check for updates, keep in mind that the OSS Last-Modified value reflects when the object was cached — not when it was last modified on the origin.

Configure the rule

1. Go to the Buckets page and click the name of the target bucket.

2. In the left-side navigation pane, choose Data Management > Mirroring-based Back-to-origin.

3. Click Create Rule.

4. In the Create Rule panel, set the following parameters. Keep the defaults for all other parameters.

   Parameter	Configuration
   Method	Select Mirroring.
   Condition	Select Object Name Prefix and enter examplefolder/.
   Origin URL	Protocol: https. Domain name: example.com. Path prefix: leave blank.

5. Click OK.

Verify the rule

Access https://examplebucket.oss-cn-hangzhou.aliyuncs.com/examplefolder/example.txt.

If examplefolder/example.txt does not exist in examplebucket, OSS sends a request to https://example.com/examplefolder/example.txt. After fetching the object, OSS saves it as examplefolder/example.txt in examplebucket and returns it to the requester.

Configure the rule

1. Go to the Buckets page and click the bucket name.

2. Choose Data Management > Mirroring-based Back-to-origin, then click Create Rule.

3. In the Create Rule panel, set the following parameters:

   Parameter	Configuration
   Method	Select Mirroring.
   Condition	Select Object Name Prefix and enter examplefolder/.
   Replace or Delete File Prefix	Select Replace or Delete File Prefix and enter destfolder/. This option appears after you set Object Name Prefix.
   Origin URL	Protocol: https. Domain name: example.com. Path prefix: leave blank.
   MD5 Verification	Select Perform MD5 verification.

4. Click OK.

How MD5 verification works:

When the origin response includes the Content-MD5 header, OSS compares the MD5 hash of the fetched object against the header value:

• Match: OSS saves the object and returns it to the client.

• Mismatch: OSS streams the object back to the client (data is returned immediately), but does not save it to the bucket. The checksum calculation requires the complete object data; if it fails, the object is not stored.

When the origin response does not include Content-MD5, OSS saves the object without verification.

Verify the rule

Access https://bucket-01.oss-cn-hangzhou.aliyuncs.com/examplefolder/example.txt.

If examplefolder/example.txt does not exist, OSS fetches from https://example.com/destfolder/example.txt. If the MD5 hash matches (or the response has no Content-MD5), OSS saves the object as examplefolder/example.txt in bucket-01.

Configure the rules

1. Go to the Buckets page and click the bucket name.

2. Choose Data Management > Mirroring-based Back-to-origin, then click Create Rule.

3. Create Rule 1:

   Parameter	Configuration
   Method	Select Mirroring.
   Condition	Select Object Name Prefix and enter dir1/.
   Replace or Delete File Prefix	Select Replace or Delete File Prefix and enter example1/.
   Origin URL	Protocol: https. Domain name: example.com. Path prefix: leave blank.
   3xx Response	Select Follow Origin to Redirect Request. If not selected, OSS returns the redirect address to the requester instead of following it.

4. Repeat to create Rule 2:

   Parameter	Configuration
   Method	Select Mirroring.
   Condition	Select Object Name Prefix and enter dir2/.
   Replace or Delete File Prefix	Select Replace or Delete File Prefix and enter example2/.
   Origin URL	Protocol: https. Domain name: example.org. Path prefix: leave blank.
   3xx Response	Select Follow Origin to Redirect Request.

5. Click OK.

Verify the rules

Access https://bucket-02.oss-cn-beijing.aliyuncs.com/dir1/example.txt.

If dir1/example.txt does not exist, OSS fetches from https://example.com/example1/example.txt. The object is saved as dir1/example1/example.txt in bucket-02. If Origin Server A has a redirect rule for that path, OSS follows the redirect before fetching.

For requests under dir2/, the fetched object is stored under the dir2/example2/ prefix in bucket-02.

Permissions for private bucket origin fetch

When the origin is a private OSS bucket, OSS uses the AliyunOSSMirrorDefaultRole role with the AliyunOSSReadOnlyAccess policy to fetch data in read-only mode.

If a Resource Access Management (RAM) user configures this feature, the RAM user must have the ram:GetRole permission:

• If AliyunOSSMirrorDefaultRole already exists, it is used directly.

• If it does not exist, have the Alibaba Cloud account create AliyunOSSMirrorDefaultRole in advance and grant it AliyunOSSReadOnlyAccess. This avoids giving the RAM user high-risk permissions such as ram:CreateRole and ram:AttachPolicyToRole.

Configure the rule

1. Go to the Buckets page and click the bucket name (bucket-03).

2. Choose Data Management > Mirroring-based Back-to-origin, then click Create Rule.

3. In the Create Rule panel, set the following parameters:

   Important

   When fetching from a private bucket, do not select to pass all HTTP headers. This causes the origin fetch to fail. Pass only the specific headers required.

   Back-to-origin rules do not support passing these headers: authorization, authorization2, range, content-length, date, and any header starting with x-oss-, oss-, or x-drs-.

   Parameter	Configuration
   Method	Select Mirroring.
   Condition	Select Object Name Prefix and enter examplefolder/.
   Origin Type	Select OSS Private Bucket, then select bucket-04 from the Source Bucket drop-down list.
   Origin URL	Protocol: https. Leave other fields blank.
   Origin Parameter	Select Transfer with Query String. OSS passes the query string from the client request to the origin.
   Set Transmission Rule of HTTP Header	Select Transmit Specific HTTP Headers and add header1, header2, and header3.

4. Click OK.

Verify the rule

Access https://bucket-03.oss-cn-shanghai.aliyuncs.com/examplefolder/example.png?caller=lucas&production=oss.

If examplefolder/example.png does not exist in bucket-03, OSS sends a request to https://bucket-04.oss-cn-shanghai.aliyuncs.com/examplefolder/example.png?caller=lucas&production=oss. bucket-04 processes the request using the query string and returns example.png. OSS then saves it as examplefolder/example.png in bucket-03.

If the client request also carries header1, header2, and header3, those headers are forwarded to bucket-04.

Seamless data migration

For a complete migration walkthrough, see Seamlessly migrate services to Alibaba Cloud OSS using mirroring-based back-to-origin.

Origin server load

During the initial migration phase, back-to-origin request volume may spike. Monitor your origin server's bandwidth and processing capacity. Prefetch high-traffic objects during off-peak hours to reduce origin load at peak times.

Cost control

Set up cost alerts in the Alibaba Cloud Management Center to monitor back-to-origin request volume and avoid unexpected charges.

Security configuration

If the origin URL uses HTTPS, make sure the origin server's certificate:

• Is issued by a trusted CA

• Has a domain name that matches the origin URL

• Has not expired

Log query

Use the real-time log query feature to view back-to-origin logs. The User-Agent header in back-to-origin requests contains the string aliyun-oss-mirror.

Why is the fetched object a different size from the source object?

If sizes differ, check your back-to-origin rule's HTTP header configuration for the Accept-Encoding header being passed to the origin. If the origin compresses the response (e.g., with gzip), the stored object is the compressed version — which has a different size from the uncompressed source.

To resolve, check your back-to-origin rule's HTTP header configuration:

• If you set the rule to pass all headers, add accept-encoding to the list of prohibited headers.

• If you set the rule to pass specific headers, make sure accept-encoding is not in the list.

If sizes still differ after ruling out compression, check the Last-Modified timestamps and MD5 or CRC-64 hashes to determine whether the source object has been updated since the fetch:

1. Check `Last-Modified` — use the script below to compare timestamps.

   OSS does not preserve the source object's Last-Modified timestamp. It sets Last-Modified to the time the object was written to OSS via mirroring.

      import oss2
      import requests
      from datetime import datetime
      from oss2.credentials import EnvironmentVariableCredentialsProvider
   
      # Obtain credentials from environment variables.
      # Set OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET before running.
      auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
   
      # Replace with your bucket's endpoint, e.g., https://oss-cn-hangzhou.aliyuncs.com
      endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
      region = "cn-hangzhou"  # Required for V4 signatures
   
      bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)
      object_key = 'yourObjectKey'
      source_url = 'yourSourceUrl'
   
      oss_object_info = bucket.get_object_meta(object_key)
      oss_last_modified = oss_object_info.headers['last-modified']
      print(f"OSS Last-Modified: {oss_last_modified}")
   
      response = requests.head(source_url)
      source_last_modified = response.headers.get('last-modified')
      print(f"Source Last-Modified: {source_last_modified}")
   
      oss_time = datetime.strptime(oss_last_modified, '%a, %d %b %Y %H:%M:%S %Z')
      source_time = datetime.strptime(source_last_modified, '%a, %d %b %Y %H:%M:%S %Z')
   
      if oss_time < source_time:
          print("The source file has been updated.")
      elif oss_time > source_time:
          print("The fetched file is newer.")
      else:
          print("The timestamps of the two files are the same.")

   If the source Last-Modified is later, the source was updated after the fetch. Delete the OSS object to trigger a re-fetch.

2. Compare MD5 and CRC-64 — if timestamps match, compare checksums to confirm whether the content differs.

      # -*- coding: utf-8 -*-
      import oss2
      import hashlib
      import requests
      # Install crcmod for CRC-64 support: pip install crcmod
      import crcmod
      from oss2.credentials import EnvironmentVariableCredentialsProvider
   
      auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
      endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
      region = "cn-hangzhou"
   
      bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)
      object_key = 'yourObjectKey'
      source_url = 'yourSourceUrl'
   
      oss_object_info = bucket.get_object_meta(object_key)
      oss_md5 = oss_object_info.headers.get('etag', '').strip('"')  # ETag is usually the MD5 hash
      oss_crc64 = oss_object_info.headers.get('x-oss-hash-crc64ecma', '')
      print(f"OSS MD5: {oss_md5}")
      print(f"OSS CRC64: {oss_crc64}")
   
      response = requests.get(source_url)
      if response.status_code == 200:
          source_content = response.content
          source_md5 = hashlib.md5(source_content).hexdigest()
          print(f"Source MD5: {source_md5}")
   
          crc64_func = crcmod.predefined.mkCrcFun('crc-64')
          source_crc64 = hex(crc64_func(source_content))[2:].upper().zfill(16)
          print(f"Source CRC64: {source_crc64}")
   
          if oss_md5 == source_md5:
              print("MD5 values match.")
          else:
              print("MD5 values do not match.")
   
          if oss_crc64.upper() == source_crc64:
              print("CRC-64 values match.")
          else:
              print("CRC-64 values do not match.")
      else:
          print(f"Failed to fetch source file. HTTP Status Code: {response.status_code}")

   If checksums match, the content is identical — the size difference is likely due to compression. If they don't match, the source object may have changed; delete the OSS object and allow it to be re-fetched.

How do I troubleshoot a 424 MirrorFailed error?

Run these checks in order:

1. Check reachability of the origin server:

      # Replace with your actual origin server address
      curl -I "https://www.example.com/images/test.jpg"

2. Check DNS resolution:

      nslookup www.example.com

3. Check the HTTPS certificate (if the origin uses HTTPS):

      openssl s_client -connect www.example.com:443 -servername www.example.com

4. Review back-to-origin logs using the OSS real-time log query feature to identify the specific failure.

If the origin is OSS, also check:

• `host` header pass-through: Prohibit the host header from being forwarded. If the host header from the destination bucket is sent to the origin, the origin receives a Host value it doesn't recognize and returns a 403. OSS then surfaces this as a 424 MirrorFailed error.

• Private bucket permissions: If no permissions are configured, check whether the ACL of the destination bucket and its objects is set to public-read. If permissions are configured, check whether the AliyunOSSMirrorDefaultRole role exists and has the AliyunOSSReadOnlyAccess policy. If the policy was changed, restore it.

If the origin is not OSS, inspect server-side logs and verify Server Name Indication (SNI) configuration, back-to-origin parameters, and header pass-through settings. The origin may be returning 401 (Unauthorized), 403 (Forbidden), or a 5xx error.

Why wasn't the object fetched even though the rule is configured?

HEAD requests do not trigger mirroring-based back-to-origin rules. If a client sends a HEAD request for a missing object, OSS does not fetch it from the origin or write it to the bucket.

What status codes does the back-to-origin rule handle?

The rule is triggered only when the object is missing in OSS (HTTP 404). If the origin returns 200 or 206, OSS caches and returns the object. For any other status code from the origin, OSS returns the corresponding error to the client without caching.

If your origin returns an unexpected status code other than 200, 206, or 404, review the origin server configuration. Common causes include misconfigured host headers, SNI issues, or authentication problems.

What order are back-to-origin rules matched in?

Rules are matched in ascending order of RuleNumber. The first matching rule is executed, and the remaining rules are skipped.

Can I fetch from a service inside a VPC or on an internal IP address?

No. The origin server must have a publicly accessible address. To expose a Virtual Private Cloud (VPC) service, use a NAT Gateway or an Internet-facing Server Load Balancer (SLB) instance.

The source object was updated. Why does OSS still serve the old version?

Mirroring-based back-to-origin is a one-time pull — OSS does not monitor or re-fetch objects after they are cached. Delete the object in OSS to trigger a fresh fetch on the next request, or use object name versioning (e.g., style.v2.css) to avoid this issue.