Object Storage Service:Callback

Limits

• Region limits

  The callback feature is supported in the following regions: China (Hangzhou), China (Shanghai), China (Qingdao), China (Beijing), China (Zhangjiakou), China (Hohhot), China (Ulanqab), China (Shenzhen), China (Heyuan), China (Guangzhou), China (Chengdu), China (Hong Kong), US (Silicon Valley), US (Virginia), Japan (Tokyo), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Philippines (Manila), Germany (Frankfurt), UK (London), and UAE (Dubai).

• Callback behavior

  • The application server must respond to a callback request within 5 seconds. If the response times out, the callback is considered to have failed.

  • A failed callback does not affect the status of a successful file upload.

  • Failed callbacks are not automatically retried.

• Interface Support

  The callback feature is supported only for the PutObject, PostObject, and CompleteMultipartUpload operations. File upload managers and presigned URLs that are encapsulated by V2 SDKs and based on these fundamental operations also support the callback feature.

Client implementation

This section describes the logic and process for constructing upload callback parameters. To quickly implement the upload callback feature, see the sample code provided by the SDK.

To have OSS automatically trigger a callback after a file is uploaded, you must pass the `callback` parameter and the optional `callback-var` parameter in the upload request.

1. Construct the `callback` parameter.

   This parameter defines the address of the application server, the format of the request content, and other information. You must construct this parameter in JSON format and then Base64-encode it.

   1. Simple configuration example:

      {
      "callbackUrl":"http://oss-demo.aliyuncs.com:23450",
      "callbackBody":"bucket=${bucket}&object=${object}&my_var=${x:my_var}"
      }

      In this example:

      • callbackUrl: The address of the application server. You must modify this value as required. For example, http://oss-demo.aliyuncs.com:23450.

      • callbackBody: The content of the callback request body. You can use placeholders to dynamically pass upload information. For example, ${bucket} represents the bucket name, ${object} represents the full path of the file, and ${x:xxx} references custom variables. OSS replaces these placeholders with the actual values during the callback. For more information about the available parameters, see System parameters supported by callbackBody.

   2. Advanced configuration example:

      {
      "callbackUrl":"http://oss-demo.aliyuncs.com:23450",
      "callbackHost":"oss-cn-hangzhou.aliyuncs.com",
      "callbackBody":"bucket=${bucket}&object=${object}&my_var=${x:my_var}",
      "callbackBodyType":"application/x-www-form-urlencoded",
      "callbackSNI":false
      }

      For more information about the fields, see Callback parameter details.

2. Construct the `callback-var` parameter (optional).

   Important

   The `callback-var` parameter must be in JSON format. The key of each custom parameter must start with `x:` and contain only lowercase letters, such as x:uid.

   This parameter is used to pass custom business information, such as user IDs or order numbers, to the application server. For example:

   {
     "x:uid": "12345",
     "x:order_id": "67890"
   }

   The `callback-var` parameter must be used with the `callbackBody` parameter. To reference the custom parameters for the user ID (`uid`) and order number (`order_id`) in the preceding example, you can use the `${x:xxx}` placeholder in `callbackBody`. For example:

   {
     "callbackUrl": "http://oss-demo.aliyuncs.com:23450",
     "callbackBody": "uid=${x:uid}&order=${x:order_id}"
   }

   During the callback, OSS sends the following content. This example assumes that `callbackBodyType` is set to `application/x-www-form-urlencoded`.

   uid=12345&order=67890

3. Base64-encode the `callback` and `callback-var` parameters after you construct them.

   • Example: `callback` parameter encoding

     Original `callback` parameter:

     {
         "callbackUrl": "http://oss-demo.aliyuncs.com:23450",
         "callbackHost": "your.callback.com",
         "callbackBody": "bucket=${bucket}&object=${object}&uid=${x:uid}&order=${x:order_id}",
         "callbackBodyType": "application/x-www-form-urlencoded",
         "callbackSNI": false
     }

     Base64-encoded result:

     eyJjYWxsYmFja0hvc3QiOiAieW91ci5jYWxsYmFjay5jb20iLCAiY2FsbGJhY2tVcmwiOiAiaHR0cDovL29zcy1kZW1vLmFsaXl1bmNzLmNvbToyMzQ1MCIsICJjYWxsYmFja0JvZHkiOiAiYnVja2V0PSR7YnVja2V0fSZvYmplY3Q9JHtvYmplY3R9JnVpZD0ke3g6dWlkfSZvcmRlcj0ke3g6b3JkZXJfaWR9IiwgImNhbGxiYWNrQm9keVR5cGUiOiAiYXBwbGljYXRpb24veC13d3ctZm9ybS11cmxlbmNvZGVkIiwgImNhbGxiYWNrU05JIjogZmFsc2V9

   • Example: `callback-var` parameter encoding

     Original `callback-var` parameter:

     {
       "x:uid": "12345",
       "x:order_id": "67890"
     }

     Base64-encoded result:

     eyJ4OnVpZCI6ICIxMjM0NSIsICJ4Om9yZGVyX2lkIjogIjY3ODkwIn0=

4. Attach the encoded parameters to the request.

   After you encode the parameters, you can pass them to OSS using one of the following methods.

   Pass parameters in headers (recommended)

   This method is suitable for uploading objects using SDKs or backend code. This method provides high security and is recommended. You can set the `x-oss-callback` and `x-oss-callback-var` HTTP headers to pass callback parameters.

   • x-oss-callback: The Base64-encoded `callback` parameter.

   • x-oss-callback-var (optional): The Base64-encoded `callback-var` parameter.

   Note: When you calculate the request signature, these two parameters must be included in the canonical headers to ensure the validity of the request.

   Example: Pass callback parameters in headers

   PUT /your_object HTTP/1.1
   Host: callback-test.oss-test.aliyun-inc.com
   Accept-Encoding: identity
   Content-Length: 5
   x-oss-callback-var: eyJ4OnVpZCI6ICIxMjM0NSIsICJ4Om9yZGVyX2lkIjogIjY3ODkwIn0=
   User-Agent: aliyun-sdk-python/0.4.0 (Linux/2.6.32-220.23.2.ali1089.el5.x86_64/x86_64;2.5.4)
   x-oss-callback: eyJjYWxsYmFja0hvc3QiOiAieW91ci5jYWxsYmFjay5jb20iLCAiY2FsbGJhY2tVcmwiOiAiaHR0cDovL29zcy1kZW1vLmFsaXl1bmNzLmNvbToyMzQ1MCIsICJjYWxsYmFja0JvZHkiOiAiYnVja2V0PSR7YnVja2V0fSZvYmplY3Q9JHtvYmplY3R9JnVpZD0ke3g6dWlkfSZvcmRlcj0ke3g6b3JkZXJfaWR9IiwgImNhbGxiYWNrQm9keVR5cGUiOiAiYXBwbGljYXRpb24veC13d3ctZm9ybS11cmxlbmNvZGVkIiwgImNhbGxiYWNrU05JIjogZmFsc2V9
   Host: callback-test.oss-test.aliyun-inc.com
   Expect: 100-Continue
   Date: Wed, 26 Apr 2023 03:46:17 GMT
   Content-Type: text/plain
   Authorization: OSS qn6q**************:77Dv****************
   Test

   Use form fields in the POST request body to pass parameters

   This method applies only to the PostObject operation. You can pass callback parameters only through form fields in the POST request body.

   • `callback` parameter: You can pass this parameter as a separate form item. The value is the Base64-encoded JSON configuration.

     --9431149156168
     Content-Disposition: form-data; name="callback"
     eyJjYWxsYmFja0hvc3QiOiAieW91ci5jYWxsYmFjay5jb20iLCAiY2FsbGJhY2tVcmwiOiAiaHR0cDovL29zcy1kZW1vLmFsaXl1bmNzLmNvbToyMzQ1MCIsICJjYWxsYmFja0JvZHkiOiAiYnVja2V0PSR7YnVja2V0fSZvYmplY3Q9JHtvYmplY3R9JnVpZD0ke3g6dWlkfSZvcmRlcj0ke3g6b3JkZXJfaWR9IiwgImNhbGxiYWNrQm9keVR5cGUiOiAiYXBwbGljYXRpb24veC13d3ctZm9ybS11cmxlbmNvZGVkIiwgImNhbGxiYWNrU05JIjogZmFsc2V9

   • `callback-var` parameter (custom parameters): Each custom field must be passed as a separate form item. Do not encapsulate the custom fields into a single `callback-var` field.

     For example, for the custom parameters `uid` and `order_id`:

     {
       "x:uid": "12345",
       "x:order_id": "67890"
     }

     You must convert them into two separate fields in the form.

     --9431149156168
     Content-Disposition: form-data; name="x:uid"
     12345
     --9431149156168
     Content-Disposition: form-data; name="x:order_id"
     67890

   • Verify the `callback` parameter (optional): You can specify verification conditions for the `callback` parameter in the policy. If you do not set verification conditions, the parameter is not verified during the upload. For example:

     { "expiration": "2021-12-01T12:00:00.000Z",
       "conditions": [
         {"bucket": "examplebucket" },
         {"callback": "eyJjYWxsYmFja0hvc3QiOiAieW91ci5jYWxsYmFjay5jb20iLCAiY2FsbGJhY2tVcmwiOiAiaHR0cDovL29zcy1kZW1vLmFsaXl1bmNzLmNvbToyMzQ1MCIsICJjYWxsYmFja0JvZHkiOiAiYnVja2V0PSR7YnVja2V0fSZvYmplY3Q9JHtvYmplY3R9JnVpZD0ke3g6dWlkfSZvcmRlcj0ke3g6b3JkZXJfaWR9IiwgImNhbGxiYWNrQm9keVR5cGUiOiAiYXBwbGljYXRpb24veC13d3ctZm9ybS11cmxlbmNvZGVkIiwgImNhbGxiYWNrU05JIjogZmFsc2V9"},
         ["starts-with", "$key", "user/eric/"]
       ]
     }

   Pass parameters in the URL

   • This method is commonly used to upload files using presigned URLs. This method implements automatic callbacks by Base64-encoding callback parameters and appending them to the URL. However, this method poses security risks because callback information is exposed in the URL. We recommend that you use this method only for temporary access or in scenarios with low security requirements.

   • If you choose to pass callback parameters in the URL, you must include the callback parameter. The callback-var parameter is optional. When you calculate the signature, these parameters must be included in the canonical query string. For more information, see Signature Version 4.

     Example:

     PUT /your_object?OSSAccessKeyId=LTAI******************&Signature=vjby*************************************&Expires=1682484377&callback-var=eyJ4OnVpZCI6ICIxMjM0NSIsICJ4Om9yZGVyX2lkIjogIjY3ODkwIn0=&callback=eyJjYWxsYmFja0hvc3QiOiAieW91ci5jYWxsYmFjay5jb20iLCAiY2FsbGJhY2tVcmwiOiAiaHR0cDovL29zcy1kZW1vLmFsaXl1bmNzLmNvbToyMzQ1MCIsICJjYWxsYmFja0JvZHkiOiAiYnVja2V0PSR7YnVja2V0fSZvYmplY3Q9JHtvYmplY3R9JnVpZD0ke3g6dWlkfSZvcmRlcj0ke3g6b3JkZXJfaWR9IiwgImNhbGxiYWNrQm9keVR5cGUiOiAiYXBwbGljYXRpb24veC13d3ctZm9ybS11cmxlbmNvZGVkIiwgImNhbGxiYWNrU05JIjogZmFsc2V9 HTTP/1.1
     Host: callback-test.oss-cn-hangzhou.aliyuncs.com
     Date: Wed, 26 Apr 2023 03:46:17 GMT
     Content-Length: 5
     Content-Type: text/plain

Server-side implementation

This section describes the server-side processing flow. For code examples in various languages, see Server-side code examples.

The application server must support the following features:

1. Receive POST requests from OSS

   After a file is successfully uploaded, OSS automatically sends a POST request to your configured application server based on the callback parameters. For example:

   POST /test HTTP/1.1
   Host: your.callback.com
   Connection: close
   Authorization: GevnM3**********3j7AKluzWnubHSVWI4dY3VsIfUHYWnyw==
   Content-MD5: iKU/O/JB***ZMd8Ftg==
   Content-Type: application/x-www-form-urlencoded
   Date: Tue, 07 May 2024 03:06:13 GMT
   User-Agent: aliyun-oss-callback
   x-oss-bucket: your_bucket
   x-oss-pub-key-url: aHR0cHM6Ly9nb3NzcHVi**********vY2FsbGJeV92MS5wZW0=
   x-oss-request-id: 66399AA50*****3334673EC2
   x-oss-requester: 23313******948342006
   x-oss-signature-version: 1.0
   x-oss-tag: CALLBACK
   bucket=your_bucket&object=your_object&uid=12345&order_id=67890

2. Verify the request signature for security (optional)

   To ensure that callback requests originate from OSS, verify their signatures on your application server. For verification instructions, see Recommended configurations.

   Note

   Signature verification is optional. You can enable it as required.

3. Return a callback response

   After the application server receives a callback request, it must return a response to OSS. The callback response must meet the following requirements:

   • The application server must return a response with the `HTTP/1.1 200 OK` status code.

   • The response header must include `Content-Length`.

   • The response body can be in JSON or XML format. This example uses the JSON format. To use the XML format for the response body, you must add Content-Type: application/xml to the response header.

   For example, the application server returns `{"Status": "OK"}`.

   Note: This example uses Python 2.7.6. For actual development, you must use Python 3.

   HTTP/1.0 200 OK
   Server: BaseHTTP/0.3 Python/2.7.6
   Date: Mon, 14 Sep 2015 12:37:27 GMT
   Content-Type: application/json
   Content-Length: 9
   {"Status": "OK"}

   OSS then passes this response content to the client. For example:

   HTTP/1.1 200 OK
   Date: Mon, 14 Sep 2015 12:37:27 GMT
   Content-Type: application/json
   Content-Length: 9
   Connection: keep-alive
   ETag: "D8E8FCA2DC0F896FD7CB4CB0031BA249"
   Server: AliyunOSS
   x-oss-bucket-version: 1442231779
   x-oss-request-id: 55F6BF87207FB30F2640C548
   {"Status": "OK"}

   Important

   For a CompleteMultipartUpload request, if the original response body contains content, such as information in JSON format, enabling the upload callback overwrites this content with the content returned by the callback, such as {"Status": "OK"}.

Verify request signature for security

After you set the callback parameters, OSS sends a POST callback request to your configured application server based on the `callbackUrl` parameter. To ensure that the request originates from OSS, you can verify the signature in the callback request. The following section describes the verification steps in detail.

1. Client-Side Signature Generation (Performed by OSS)

   OSS uses the RSA asymmetric encryption algorithm and the MD5 hash algorithm to generate a signature for the request content and adds the signature to the `authorization` field in the request header.

   • The signature is calculated as follows:

     authorization = base64_encode(rsa_sign(private_key, url_decode(path) + query_string + '\n' + body, md5))

     Note

     In this formula, `private_key` is the private key, `path` is the resource path of the callback request, `query_string` is the query string, and `body` is the callback message body.

   • The following steps describe how to generate a signature:

     1. Create the string to sign. The string consists of the resource path that is obtained by decoding the URL, the original query string, a carriage return, and the callback message body.

     2. Sign the created string using the RSA encryption algorithm and the private key. The hash function that is used to calculate the signature is MD5.

     3. Base64-encode the signed string to obtain the final signature. Then, add the signature to the `Authorization` header in the callback request.

   • Example of signature generation:

     POST /index.php?id=1&index=2 HTTP/1.0
     Host: 172.16.XX.XX
     Connection: close
     Content-Length: 18
     authorization: kKQeGTRccDKyHB3H9vF+xYMSrmhMZj****/kdD1ktNVgbWEfYTQG0G2SU/RaHBovRCE8OkQDjC3uG33esH2t****
     Content-Type: application/x-www-form-urlencoded
     User-Agent: http-client/0.0.1
     x-oss-pub-key-url: aHR0cDovL2dvc3NwdWJsaWMuYWxpY2RuLmNvbS9jYWxsYmFja19wdWJfa2V5X3YxLnsr****
     bucket=examplebucket

     The final signature kKQeGTRccDKyHB3H9vF+xYMSrmhMZjzzl2/kdD1ktNVgbWEfYTQG0G2SU/RaHBovRCE8OkQDjC3uG33esH2t**** is generated from the path /index.php, the query string ?id=1&index=2, and the body bucket=examplebucket.

2. The callback server verifies the signature

   The application server must verify the signature of the request from OSS to confirm the legitimacy of the request source. You can follow these steps:

   1. Obtain the public key:

      You can obtain the Base64-encoded public key URL from the `x-oss-pub-key-url` field in the request header and then decode the URL.

      public_key = urlopen(base64_decode(x-oss-pub-key-url header value))

      Example value before decoding:

      aHR0cDovL2dvc3NwdWJsaWMuYWxpY2RuLmNvbS9jYWxsYmFja19wdWJfa2V5X3YxLnBlbQ==

      After decoding:

      http://gosspublic.alicdn.com/callback_pub_key_v1.pem

      Note

      The public key URL must start with http://gosspublic.alicdn.com/ or https://gosspublic.alicdn.com/. The content of the public key URL does not change. We recommend that you cache the public key to prevent service interruptions that are caused by network fluctuations.

   2. Decode the signature.

      You can obtain the signature from the `authorization` field in the request header and then Base64-decode the signature.

      signature = base64_decode(authorization header value)

   3. Construct the string for verification.

      You can concatenate the resource path, query string, line feed, and callback message body in the following format:

      sign_str = url_decode(path) + query_string + '\n' + body

   4. Execute signature verification.

      You can verify the signature using the MD5 hash algorithm and the RSA public key.

      result = rsa_verify(public_key, md5(sign_str), signature)

3. Signature verification example

   The following Python 3 code provides an example of how to verify a signature on an application server. This example requires the M2Crypto library.

   import http.client
   import base64
   import hashlib
   import urllib.request
   import urllib.parse
   import socket
   from http.server import BaseHTTPRequestHandler, HTTPServer
   from M2Crypto import RSA
   from M2Crypto import BIO
   
   def get_local_ip():
       try:
           csock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
           csock.connect(('8.8.8.8', 80))
           (addr, port) = csock.getsockname()
           csock.close()
           return addr
       except socket.error:
           return ""
   
   class MyHTTPRequestHandler(BaseHTTPRequestHandler):
       '''
       def log_message(self, format, *args):
           return
       '''
       def do_POST(self):
           # Get public key.
           pub_key_url = ''
           try:
               pub_key_url_base64 = self.headers['x-oss-pub-key-url']
               pub_key_url = base64.b64decode(pub_key_url_base64).decode()
               if not pub_key_url.startswith("http://gosspublic.alicdn.com/") and not pub_key_url.startswith("https://gosspublic.alicdn.com/"):
                   self.send_response(400)
                   self.end_headers()
                   return
               url_reader = urllib.request.urlopen(pub_key_url)
               # Cache the public key content based on the public key address.
               pub_key = url_reader.read() 
           except Exception as e:
               print('pub_key_url : ' + pub_key_url)
               print('Get pub key failed! Error:', str(e))
               self.send_response(400)
               self.end_headers()
               return
           
           # Get authorization.
           authorization_base64 = self.headers['authorization']
           authorization = base64.b64decode(authorization_base64)
           # Get callback body.
           content_length = self.headers['content-length']
           callback_body = self.rfile.read(int(content_length))
           # Compose authorization string.
           auth_str = ''
           pos = self.path.find('?')
           if -1 == pos:
               auth_str = urllib.parse.unquote(self.path) + '\n' + callback_body.decode()
           else:
               auth_str = urllib.parse.unquote(self.path[0:pos]) + self.path[pos:] + '\n' + callback_body.decode()
           print(auth_str)
           # Verify authorization.
           auth_md5 = hashlib.md5(auth_str.encode()).digest()
           bio = BIO.MemoryBuffer(pub_key)
           rsa_pub = RSA.load_pub_key_bio(bio)
           try:
               result = rsa_pub.verify(auth_md5, authorization, 'md5')
           except:
               result = False
           if not result:
               print('Authorization verify failed!')
               print('Public key : %s' % (pub_key))
               print('Auth string : %s' % (auth_str))
               self.send_response(400)
               self.end_headers()
               return
           
           # Do something based on callback_body.
           # Respond to OSS.
           resp_body = '{"Status":"OK"}'
           self.send_response(200)
           self.send_header('Content-Type', 'application/json')
           self.send_header('Content-Length', str(len(resp_body)))
           self.end_headers()
           self.wfile.write(resp_body.encode())
   
   class MyHTTPServer(HTTPServer):
       def __init__(self, host, port):
           super().__init__((host, port), MyHTTPRequestHandler)
   
   if __name__ == '__main__':
       server_ip = get_local_ip()
       server_port = 23451
       server = MyHTTPServer(server_ip, server_port)
       server.serve_forever()

   For server-side code examples in other programming languages, see the following table.

   SDK language	Description
   Java	Download link: Java Running method: Decompress the package and run java -jar oss-callback-server-demo.jar 9000. You can replace 9000 with a different port number.
   Python	Download link: Python Running method: Decompress the package and run python callback_app_server.py. This program requires RSA dependencies.
   PHP	Download link: PHP Running method: Deploy to an Apache environment. In PHP, retrieving some headers is environment-dependent. Modify the example code to suit your environment.
   .NET	Download link: .NET Running method: Decompress the package and follow the instructions in README.md.
   Node.js	Download link: Node.js Running method: Decompress the package and run node example.js.
   Ruby	Download link: Ruby Running method: Run ruby aliyun_oss_callback_server.rb

Callback parameter details

The following table provides detailed descriptions of the `callback` parameter, which is used to configure the content and behavior of callback requests after a file is successfully uploaded to OSS.

System parameters supported by callbackBody

The `callbackBody` field in the `callback` parameter supports multiple system parameters that you can reference to pass information about the uploaded file in the callback request. The supported system parameters are listed in the following table.

Does OSS send callback notifications to the application server after a file upload fails?

No. OSS executes callbacks only after a file is successfully uploaded. If a file upload fails, OSS does not execute a callback and instead returns an error message.

How to handle the error 'Response body is not valid json format'?

• The application server throws an exception during processing. This causes the body that is returned to OSS to not be in JSON format, as shown in the following figure:

  Solution:

  • You can run the following command to confirm the content.

    curl -d "<Content>" <CallbackServerURL> -v

  • You can capture packets to confirm the content.

    On Windows, you can use Wireshark to capture packets. On Linux, you can run the tcpdump command.

• The body that is returned by the application server to OSS contains a byte order mark (BOM) header.

  This error is common in application servers that are written using the PHP SDK. Because the PHP SDK returns a BOM header, the body that is received by OSS contains three extra bytes. This results in a body that is not in JSON format. As shown in the following figure, the `ef bb bf` bytes constitute the BOM header.

  

  Solution: Remove the BOM header from the body that is returned by the application server to OSS.