All Products
Search
Document Center

How to handle common errors when you implement upload callback

Last Updated: Aug 02, 2021

Overview

This article describes how to analyze and handle common errors when you implement upload callback.

Description

For more information about common errors and possible causes when you implement upload callback, see Common errors and possible causes. For more information about the introduction of upload callback and how to use this feature, see the following content:

Introduction of upload callback

After the object upload is complete, OSS provides Callback for your callback server. Include a callback parameter in the upload request. Upload callback can be implemented. API operations that support upload callback include PutObject, PostObject, and CompleteMultipartUpload. For more information, see Upload callback and Callback API in Developer Guide.

Note: Callback servers in this article are also called application servers.

Scenarios

  • Notification
    A typical application of upload callback is to send callback requests when an authorized third-party user uploads objects. To implement upload callback in this scenario, the third-party user must specify callback parameters. After an object is uploaded, OSS sends a callback request to the callback server. After the callback server receives the callback request, the callback server records the upload information.
  • Processing, review, and collect statistics
    When the callback server receives a callback request, the callback server processes upload information, reviews the object upload, and collects statistics for the uploaded object.

Upload callback process

The following table describes the upload callback process showed in the preceding figure.

Data stream Meaning Description
1 The client uploads an object and includes a callback parameter. For more information about the format, see PostObject. The upload is implemented by SDK and by calling API operations (PutObject, CompleteMultipartUpload, and PostObject).
2 OSS stores the object and initiates a callback. OSS sends a POST request to the specified CallbackUrl in the upload request. The callback timeout period is 5 seconds, which is a fixed value and cannot be configured. For more information about the format of the POST request, see Initiate a callback request.
3 The callback server returns the processing result.
  • The message body returned by the callback server must be in the JSON format.
  • OSS determines that the callback fails if the returned result is not the 200 HTTP status code. The 40x HTTP status code indicates invalid parameters or callback failures. The 50x HTTP status code indicates timeout or connection failures.
4 OSS returns the upload and callback results.
  • If both the upload and callback succeed, the 200 HTTP status code is returned.
  • If the upload succeeds but the callback fails, the 203 HTTP status code is returned. The value of ErrorCode is CallbackFailed, and ErrorMessage indicates the error cause.

SDK or PostObject

Examples

During the object upload, you can set the callback parameters to specify the URL of the callback server, data to be sent to the callback server, and data format. The following table describes the parameters that are available for an upload callback. When the callback server processes a callback, some context information, such as bucket and object, is specified by using system variables. Other context information is specified by using custom variables.

Field Meaning Description
callbackUrl Callback server address Required.
callbackHost Value of the Host field in the callback request message header Optional. The default value is callbackUrl.
callbackBody Callback request message body Required. It can contain system variables and custom variables.
callbackBodyType Value of Content-Type in the callback request message header, which is the callbackBody data format. Optional. It can be application/x-www-form-urlencoded (default) or application/json.

Upload callback parameters are included in the upload request in one of the following two ways:

  • The callback parameters are included by x-oss-callback in the message header. This is a common and recommended way.
  • The callback parameters are included by callback in QueryString.

The following code provides rules for generating the x-oss-callback or callback values:

Callback := Base64(CallbackJson)
CallbackJson := '{' CallbackUrlItem, CallbackBodyItem [, CallbackHostItem, CallbackBodyTypeItem] '}' 
CallbackUrlItem := '"'callbackUrl'"' ':' '"'CallbackUrlValue'"'
CallbackBodyItem := '"'callbackBody'"' ':' '"'CallbackBodyValue'"'
CallbackHostItem := '"'callbackHost'"' ':' '"'CallbackHostValue'"'
CallbackBodyTypeItem := '"'callbackBodyType'"' : '"'CallbackBodyType'"'
CallbackBodyType := application/x-www-form-urlencoded | application/json

Example 1 of the CallbackJson value:

{
    "callbackUrl" : "http://abc.com/test.php",
    "callbackHost" : "oss-cn-hangzhou.aliyuncs.com",
    "callbackBody" : "{\"bucket\":${mimeType}, \"object\":${object},\"size\":${size},\"mimeType\":${mimeType},\"my_var\":${x:my_var}}",
    "callbackBodyType" : "application/json"
}

Example 2:

{
    "callbackUrl" : "http://abc.com/test.php",
    "callbackBody" : "bucket=${bucket}&object=${object}&etag=${etag}&size=${size}&mimeType=${mimeType}&my_var=${x:my_var}"
}

System variables and custom variables

Variables for callbackBody, such as ${bucket}, ${object}, and ${size}, in the CallbackJson example are OSS-defined system variables. During the callback, OSS replaces the system variables with actual values. The following table lists OSS-defined system variables.

Variable Meaning
${bucket} The name of the bucket.
${object} The name of the object.
${etag} The ETag of the object.
${size} The size of the object.
${mimeType} The Type of the object, such as IMAGE and JPEG.
${imageInfo.height} The height of the image.
${imageInfo.width} The width of the image.
${imageInfo.format} The format of the image, such as JPG and PNG.

Note:

  • The system variables are case-sensitive.
  • The system variables are in the ${bucket} format.
  • imageInfo is set for images. For the non-image format, the value of imageInfo is empty.

Variables for callbackBody, such as ${x:my_var}, in the CallbackJson example are the custom variables. During the callback, OSS replaces the custom variables with custom values. Custom variable values are defined and included in the upload request in one of the following two ways:

  • The custom variables are included by x-oss-callback-var in the message header. This is a common and recommended way.
  • The custom variables are included by callback-var in QueryString.

The following code is the rules for generating x-oss-callback-var or callback-var:

CallbackVar := Base64(CallbackVarJson)
CallbackVarJson := '{' CallbackVarItem [, CallbackVarItem]* '}'
CallbackVarItem := '"''x:'VarName'"' : '"'VarValue'"'

An example of the CallbackVarJson value:

{
    "x:my_var1" : "value1",
    "x:my_var2" : "value2"
}
Note:
  • The custom variables must start with x:. They are case-sensitive and in the format of ${x:my_var}.
  • The custom variable length is subject to the length of the message header and URL. We recommend that the number of the custom variables do not exceed 10 and the total length do not exceed 512 bytes.

Callback server

The callback server is an HTTP server that processes callback requests and POST messages sent by OSS. The callback server URL is the value specified by the callbackUrl parameter. You can implement your own processing logic on the callback server for recording, review, processing, and collecting statistics for the uploaded data.

  • Callback signature: The callback server needs to verify the signature of a POST request to make sure that the POST request is from OSS for upload callback. The callback server can also directly process the message without verifying the signature. To enhance the security of the callback server, we recommend that the callback server verifies the message signature. For more information about the callback signature rules, see Callback signature.
    Note: The example of implementing upload callback by using callback servers describes how to implement signature verification. We recommend that you directly use the code.
  • Message processing: The main logic of the callback server is to process the OSS callback request. Take note of the following items:
    • The callback server must process the POST request of OSS.
    • The OSS callback timeout time is 5 seconds. Therefore, the callback server must complete processing within 5 seconds and return the result.
    • The message body sent from the callback server to OSS must be in the JSON format.
    • The callback server uses its own logic, and OSS provides examples instead of the specific service logic.
  • Implementation example: The following table describes the implementation examples of the upload callback.
    Programming language Example Running method
    Java AppCallbackServer.zip Decompress the package and run java -jar oss-callback-server-demo.jar 9000.
    PHP callback-php-demo.zip Deploy and run the program in the Apache environment.
    Python callback_app_server.py.zip Decompress the package and run python callback_app_server.py.
    Ruby oss-callback-server Run ruby server.rb.

OSS SDK examples used for upload callback

The code used to perform the preceding steps is encapsulated in OSS SDK for Java and OSS SDK for JavaScript. The preceding rules are used to generate upload callback parameters and custom variables for OSS SDK for Python, OSS SDK for PHP, and OSS SDK for C. The following table describes OSS SDK examples for upload callback.

SDK Upload callback example Remarks
Java CallbackSample.java Take note of the escape characters in CallbackBody
Python object_callback.py -
PHP Callback.php OSS_CALLBACK and OSS_CALLBACK_VAR in $options do not need to be encoded by using Base64, which is implemented by the SDK.
C# UploadCallbackSample.cs Use using to read PutObjectResult.ResponseStream but make sure that PutObjectResult.ResponseStream is disabled.
JS object.test.js -
C oss_callback_sample.c -
Ruby callback.rb -
iOS

Callback notification after upload

Make sure that the format of <var1> is x:var1.
Andriod

Callback notification after upload

Take note of the escape characters in CallbackBody.

Note: OSS SDK for Go does not support upload callback.

Implement upload callback for PostObject

PostObject supports upload callback, whose callback parameters are included by the callback form field and custom variables are included by an independent form field. For more information, see PostObject.

SDK Upload callback example
Java PostObjectSample.java
Python object_post.py
JS JavaScript-based signatures on the client for object upload
C# PostPolicySample.cs

Debugging

To perform debugging on upload callback you must debug the client from which an object is uploaded and the callback server that processes the callback request We recommend that you debug the client first and then the callback server. After you debug the client and the callback server, perform the complete upload callback.

  • Debugging on the client: You can use the callback server address http://oss-demo.aliyuncs.com:23450 provided by OSS. The callback server address is specified by the callbackUrl parameter. The callback server only verifies the callback request signature, and does not process the callback request. For callback requests with successful signature verification, the callback server returns {"Status":"OK"}. For callback requests with failed signature verification, the callback server returns 400 Bad Request. For non-POST requests, the callback server returns 501 Unsupported method. For more information about the code of the callback server example, see callback_app_server.py.zip.
  • Debugging on the callback server: The callback server is an HTTP server that can process the POST request. You can modify the callback server based on the example provided by the OSS or implement it by yourself. The following table describes examples of the callback server provided by OSS.
    Programming language Example Running method
    Java AppCallbackServer.zip Decompress the package and run java -jar oss-callback-server-demo.jar 9000.
    PHP callback-php-demo.zip Deploy and run the program in the Apache environment.
    Python callback_app_server.py.zip Decompress the package and run python callback_app_server.py.
    C# callback-server-dotnet.zip Compile the program and run aliyun-oss-net-callback-server.exe 127.0.0.1 80.
    Go callback-server-go.zip Compile the program and run aliyun_oss_callback_server.
    Ruby oss-callback-server Run ruby aliyun_oss_callback_server.rb.

The callback server can be debugged by running the cURL command. The following commands may be used:

# Run the following command to send a `POST` request whose message body is `object=test_obj` to the callback server:
curl -d "object=test_obj" http://oss-demo.aliyuncs.com:23450 -v
# Run the following command to send a `POST` request whose message body is `post.txt` to the callback server:
curl -d @post.txt http://oss-demo.aliyuncs.com:23450 -v
# Run the following command to send a `POST` request whose message body is `post.txt` and that includes the specified message header `Content-Type` to the callback server:
curl -d @post.txt -H "Content-Type: application/json" http://oss-demo.aliyuncs.com:23450 -v

Note:

  • When you debug the callback server, ignore signature verification because it is difficult for cURL to simulate the signature feature.
  • The OSS example already provides the signature verification feature. We recommend that you directly use it.
  • We recommend that the callback server provide the logging feature to record all messages, which facilitates debugging and tracking.
  • After a callback request is correctly processed, the callback server must return 200 instead of other 20x HTTP status codes.
  • The message body sent from the callback server to OSS must be in the JSON format, and Content-Type is set to application/json.

Common errors and causes

InvalidArgument

The callback parameter settings are incorrect, or the parameter format is incorrect. The common error is that the callback parameters in ArgumentValue are not in the valid JSON format. In JSON, \" is an escape character. For example, "callbackBody":"{"bucket":${bucket},"object":${object}}" must be "callbackBody":"{\"bucket\":${bucket},\"object\":${object}}". For more information about the SDKs, see the upload callback examples in the SDK usage example section.

<Error>
  <Code>InvalidArgument</Code>
  <Message>The callback configuration is not json format.</Message>
  <RequestId>587C79A3DD373E2676F73ECE</RequestId>
  <HostId>bucket.oss-cn-hangzhou.aliyuncs.com</HostId>
  <ArgumentName>callback</ArgumentName>
  <ArgumentValue>{"callbackUrl":"8.8.8.8:9090","callbackBody":"{"bucket":${bucket},"object":${object}}","callbackBodyType":"application/json"}</ArgumentValue>
</Error>
Character after interpretation Character before interpretation
\\ \\\\
\" \\\"
\b \\b
\f \\f
\n \\n
\r \\r
\t \\t

CallbackFailed

Error message Cause Solution
<Error> <Code>CallbackFailed</Code> <Message>Response body is not valid json format.</Message> <RequestId>587C81A125F797621829923D</RequestId> <HostId>bucket.oss-cn-hangzhou.aliyuncs.com</HostId> </Error> The message body sent from the callback server to OSS is not in the JSON format.

You can confirm the content by running curl -d "<Content>" <CallbackServerURL> -v or capturing packets. We recommend that you use Wireshark to capture packets in Windows, and use tcpdump to capture packets in Linux. For more information, see How to capture packages when network exception occurs. Invalid returned messages include: OK and \357\273\277{"Status":"OK"} (the BOM header containing the ef bb bf bytes).

<Error> <Code>CallbackFailed</Code> <Message>Error status : -1.OSS can not connect to your callbackUrl, please check it.</Message> <RequestId>587C8735355BE8694A8E9100</RequestId> <HostId>bucket.oss-cn-hangzhou.aliyuncs.com</HostId> </Error> The processing time of the callback server exceeds 5 seconds. Therefore, OSS determines that a timeout occurs. We recommend that you modify the processing logic of the callback server to asynchronous processing to ensure that it can complete processing within five seconds and returns the result to OSS.
<Error> <Code>CallbackFailed</Code> <Message>Error status : -1 8.8.8.8:9090 reply timeout, cost:5000ms, timeout:5000ms (err -4, errno115)</Message> <RequestId>587C8D382AE0B92FA3EEF62C</RequestId> <HostId>bucket.oss-cn-hangzhou.aliyuncs.com</HostId> </Error> The processing time of the callback server exceeds 5 seconds. Therefore, OSS determines that a timeout occurs. -
<Error> <Code>CallbackFailed</Code> <Message>Error status : 400.</Message> <RequestId>587C89A02AE0B92FA3C7981D</RequestId> <HostId>bucket.oss-cn-hangzhou.aliyuncs.com</HostId> </Error> The HTTP status code of the message sent from the callback server to OSS is 400. Check the processing logic of the callback server.
<Error> <Code>CallbackFailed</Code> <Message>Error status : 502.</Message> <RequestId>587C8D382AE0B92FA3EEF62C</RequestId> <HostId>bucket.oss-cn-hangzhou.aliyuncs.com</HostId> </Error> The callback server is not started, CallbackUrl is missing in the callback parameters, or the network between OSS and the callback server is disconnected. We recommend that you deploy the callback server on ECS, which belongs to the same internal network as OSS, to save the traffic cost and guarantee the network quality.

The body of the response is not in the JSON format.

This error may be caused by the following reasons:

  • The body of the response returned by the callback server to OSS is not in the JSON format. OSS reports the error if
    Resp_body is not in valid JSON format. In addition, this error may be caused by other underlying factors, such as the callback server returning a stack trace instead of a normal response to OSS because of exceptions.

  • The body of the response returned by the callback server to OSS includes a BOM header.
    This problem occurs in callback servers coded in PHP, which includes a BOM header in the response returned to OSS. Therefore, OSS reports the error because three additional bytes are included in the response, which does not conform to the JSON format. The following figure shows the content included in the packet sent by the callback server.

    Note: In the preceding figure, the ef bb bf bytes are the three additional bytes of the BOM header. Remove the BOM header from the response returned by the callback server to OSS.

HTTP status code

HTTP status codes, such as 502 and 400, are returned. The following figure shows an example.

Note:
  • An HTTP status code, such as 400, 404, or 403, is returned by the callback server to OSS. If HTTP status code 200 is returned, the operation is successful.
  • HTTP status code 502 is returned when the web server is not enabled on the callback server. The server cannot receive the callback request sent by OSS.

Timeout

For security reasons, OSS waits a maximum of 5 seconds to receive the callback response. If the response is not returned after 5 seconds, OSS disconnects from the callback server and returns a timeout error to the client. The IP address included in the error message can be ignored.

References

Application scope

  • OSS