How to handle common errors when you implement upload callback

Introduction of upload callback

After the object upload is complete, OSS provides Callback for your callback server. Include a callback parameter in the upload request. Upload callback can be implemented. API operations that support upload callback include PutObject, PostObject, and CompleteMultipartUpload. For more information, see Upload callback and Callback API in Developer Guide.

Note: Callback servers in this article are also called application servers.

Upload callback process

The following table describes the upload callback process showed in the preceding figure.

SDK or PostObject

Examples

During the object upload, you can set the callback parameters to specify the URL of the callback server, data to be sent to the callback server, and data format. The following table describes the parameters that are available for an upload callback. When the callback server processes a callback, some context information, such as bucket and object, is specified by using system variables. Other context information is specified by using custom variables.

Upload callback parameters are included in the upload request in one of the following two ways:

• The callback parameters are included by x-oss-callback in the message header. This is a common and recommended way.
• The callback parameters are included by callback in QueryString.

The following code provides rules for generating the x-oss-callback or callback values:

Callback := Base64(CallbackJson)
CallbackJson := '{' CallbackUrlItem, CallbackBodyItem [, CallbackHostItem, CallbackBodyTypeItem] '}' 
CallbackUrlItem := '"'callbackUrl'"' ':' '"'CallbackUrlValue'"'
CallbackBodyItem := '"'callbackBody'"' ':' '"'CallbackBodyValue'"'
CallbackHostItem := '"'callbackHost'"' ':' '"'CallbackHostValue'"'
CallbackBodyTypeItem := '"'callbackBodyType'"' : '"'CallbackBodyType'"'
CallbackBodyType := application/x-www-form-urlencoded | application/json

Example 1 of the CallbackJson value:

{
    "callbackUrl" : "http://example.com/test.php",
    "callbackHost" : "oss-cn-hangzhou.aliyuncs.com",
    "callbackBody" : "{\"bucket\":${mimeType}, \"object\":${object},\"size\":${size},\"mimeType\":${mimeType},\"my_var\":${x:my_var}}",
    "callbackBodyType" : "application/json"
}

Example 2:

{
    "callbackUrl" : "http://example.com/test.php",
    "callbackBody" : "bucket=${bucket}&object=${object}&etag=${etag}&size=${size}&mimeType=${mimeType}&my_var=${x:my_var}"
}

System variables and custom variables

Variables for callbackBody, such as ${bucket}, ${object}, and ${size}, in the CallbackJson example are OSS-defined system variables. During the callback, OSS replaces the system variables with actual values. The following table lists OSS-defined system variables.

Note:

• The system variables are case-sensitive.
• The system variables are in the ${bucket} format.
• imageInfo is set for images. For the non-image format, the value of imageInfo is empty.

Variables for callbackBody, such as ${x:my_var}, in the CallbackJson example are the custom variables. During the callback, OSS replaces the custom variables with custom values. Custom variable values are defined and included in the upload request in one of the following two ways:

• The custom variables are included by x-oss-callback-var in the message header. This is a common and recommended way.
• The custom variables are included by callback-var in QueryString.

The following code is the rules for generating x-oss-callback-var or callback-var:

CallbackVar := Base64(CallbackVarJson)
CallbackVarJson := '{' CallbackVarItem [, CallbackVarItem]* '}'
CallbackVarItem := '"''x:'VarName'"' : '"'VarValue'"'

An example of the CallbackVarJson value:

{
    "x:my_var1" : "value1",
    "x:my_var2" : "value2"
}

OSS SDK examples used for upload callback

The code used to perform the preceding steps is encapsulated in OSS SDK for Java and OSS SDK for JavaScript. The preceding rules are used to generate upload callback parameters and custom variables for OSS SDK for Python, OSS SDK for PHP, and OSS SDK for C. The following table describes OSS SDK examples for upload callback.

Note: OSS SDK for Go does not support upload callback.

Implement upload callback for PostObject

PostObject supports upload callback, whose callback parameters are included by the callback form field and custom variables are included by an independent form field. For more information, see PostObject.

Debugging

To perform debugging on upload callback you must debug the client from which an object is uploaded and the callback server that processes the callback request We recommend that you debug the client first and then the callback server. After you debug the client and the callback server, perform the complete upload callback.

• Debugging on the client: You can use the callback server address http://oss-demo.aliyuncs.com:23450 provided by OSS. The callback server address is specified by the callbackUrl parameter. The callback server only verifies the callback request signature, and does not process the callback request. For callback requests with successful signature verification, the callback server returns {"Status":"OK"}. For callback requests with failed signature verification, the callback server returns 400 Bad Request. For non-POST requests, the callback server returns 501 Unsupported method. For more information about the code of the callback server example, see callback_app_server.py.zip.
• Debugging on the callback server: The callback server is an HTTP server that can process the POST request. You can modify the callback server based on the example provided by the OSS or implement it by yourself. The following table describes examples of the callback server provided by OSS.
  

  Programming language	Example	Running method
  Java	AppCallbackServer.zip	Decompress the package and run java -jar oss-callback-server-demo.jar 9000.
  PHP	callback-php-demo.zip	Deploy and run the program in the Apache environment.
  Python	callback_app_server.py.zip	Decompress the package and run python callback_app_server.py.
  C#	callback-server-dotnet.zip	Compile the program and run aliyun-oss-net-callback-server.exe 127.0.0.1 80.
  Go	callback-server-go.zip	Compile the program and run aliyun_oss_callback_server.
  Ruby	oss-callback-server	Run ruby aliyun_oss_callback_server.rb.

The callback server can be debugged by running the cURL command. The following commands may be used:

# Run the following command to send a `POST` request whose message body is `object=test_obj` to the callback server:
curl -d "object=test_obj" http://oss-demo.aliyuncs.com:23450 -v
# Run the following command to send a `POST` request whose message body is `post.txt` to the callback server:
curl -d @post.txt http://oss-demo.aliyuncs.com:23450 -v
# Run the following command to send a `POST` request whose message body is `post.txt` and that includes the specified message header `Content-Type` to the callback server:
curl -d @post.txt -H "Content-Type: application/json" http://oss-demo.aliyuncs.com:23450 -v

Note:

• When you debug the callback server, ignore signature verification because it is difficult for cURL to simulate the signature feature.
• The OSS example already provides the signature verification feature. We recommend that you directly use it.
• We recommend that the callback server provide the logging feature to record all messages, which facilitates debugging and tracking.
• After a callback request is correctly processed, the callback server must return 200 instead of other 20x HTTP status codes.
• The message body sent from the callback server to OSS must be in the JSON format, and Content-Type is set to application/json.

Common errors and causes

InvalidArgument

The callback parameter settings are incorrect, or the parameter format is incorrect. The common error is that the callback parameters in ArgumentValue are not in the valid JSON format. In JSON, \" is an escape character. For example, "callbackBody":"{"bucket":${bucket},"object":${object}}" must be "callbackBody":"{\"bucket\":${bucket},\"object\":${object}}". For more information about the SDKs, see the upload callback examples in the SDK usage example section.

<Error>
  <Code>InvalidArgument</Code>
  <Message>The callback configuration is not json format.</Message>
  <RequestId>587C79A3DD373E2676F73ECE</RequestId>
  <HostId>bucket.oss-cn-hangzhou.aliyuncs.com</HostId>
  <ArgumentName>callback</ArgumentName>
  <ArgumentValue>{"callbackUrl":"8.8.8.8:9090","callbackBody":"{"bucket":${bucket},"object":${object}}","callbackBodyType":"application/json"}</ArgumentValue>
</Error>

CallbackFailed

The body of the response is not in the JSON format.

This error may be caused by the following reasons:

• The body of the response returned by the callback server to OSS is not in the JSON format. OSS reports the error if
  Resp_body is not in valid JSON format. In addition, this error may be caused by other underlying factors, such as the callback server returning a stack trace instead of a normal response to OSS because of exceptions.
  
  
• The body of the response returned by the callback server to OSS includes a BOM header.
  This problem occurs in callback servers coded in PHP, which includes a BOM header in the response returned to OSS. Therefore, OSS reports the error because three additional bytes are included in the response, which does not conform to the JSON format. The following figure shows the content included in the packet sent by the callback server.
  
  

  Note: In the preceding figure, the ef bb bf bytes are the three additional bytes of the BOM header. Remove the BOM header from the response returned by the callback server to OSS.

HTTP status code

HTTP status codes, such as 502 and 400, are returned. The following figure shows an example.

Note:

• An HTTP status code, such as 400, 404, or 403, is returned by the callback server to OSS. If HTTP status code 200 is returned, the operation is successful.
• HTTP status code 502 is returned when the web server is not enabled on the callback server. The server cannot receive the callback request sent by OSS.

Timeout

For security reasons, OSS waits a maximum of 5 seconds to receive the callback response. If the response is not returned after 5 seconds, OSS disconnects from the callback server and returns a timeout error to the client. The IP address included in the error message can be ignored.

References

• Upload callback

Application scope

• OSS