Object Storage Service:Multipart upload (PHP SDK V2)

Notes

• The sample code in this topic uses the China (Hangzhou) region as an example. The region ID is cn-hangzhou. By default, a public endpoint is used. If you want to access OSS from other Alibaba Cloud services in the same region, use an internal endpoint. For more information about the regions and endpoints supported by OSS, see OSS regions and endpoints.

• To perform a multipart upload, you must have the oss:PutObject permission. For more information, see Grant custom access policies to a RAM user.

• This topic provides an example of how to read access credentials from environment variables. For more examples of how to configure access credentials, see Configure access credentials for PHP.

Multipart upload flow

A multipart upload consists of the following three steps:

1. Initialize a multipart upload event.

   Call the InitiateMultipartUpload method to obtain a globally unique upload ID from OSS.

2. Upload parts.

   Call the UploadPart method to upload data for each part.

   Note

   • If you upload new data using the same part number, the existing data of the part in OSS is overwritten.

   • OSS includes the MD5 hash of the received part data in the ETag header and returns the header to you.

   • OSS calculates the MD5 hash of the uploaded data and compares it with the MD5 hash calculated by the SDK. If the two MD5 hashes are different, the InvalidDigest error code is returned.

3. Complete the multipart upload.

   After all parts are uploaded, call the CompleteMultipartUpload method to combine the parts into a complete file.

Sample code

The following sample code shows how to split a large local file into multiple parts, upload the parts to a bucket, and then combine the parts into a complete object.

<?php

// Import the autoloader file to ensure that dependency libraries can be loaded correctly.
require_once __DIR__ . '/../vendor/autoload.php';

use AlibabaCloud\Oss\V2 as Oss;

// Define the description of command-line arguments.
$optsdesc = [
    "region" => ['help' => 'The region in which the bucket is located.', 'required' => True], // The region where the bucket is located. (Required)
    "endpoint" => ['help' => 'The domain names that other services can use to access OSS.', 'required' => False], // The endpoint that can be used to access OSS. (Optional)
    "bucket" => ['help' => 'The name of the bucket', 'required' => True], // The name of the bucket. (Required)
    "key" => ['help' => 'The name of the object', 'required' => True], // The name of the object. (Required)
];

// Convert the argument descriptions to the long option format required by getopt.
// A colon (:) after each argument indicates that the argument requires a value.
$longopts = \array_map(function ($key) {
    return "$key:";
}, array_keys($optsdesc));

// Parse the command-line arguments.
$options = getopt("", $longopts);

// Check whether the required arguments exist.
foreach ($optsdesc as $key => $value) {
    if ($value['required'] === True && empty($options[$key])) {
        $help = $value['help']; // Obtain the help information of the argument.
        echo "Error: the following arguments are required: --$key, $help" . PHP_EOL;
        exit(1); // If a required argument is missing, exit the program.
    }
}

// Extract values from the parsed arguments.
$region = $options["region"]; // The region where the bucket is located.
$bucket = $options["bucket"]; // The name of the bucket.
$key = $options["key"];       // The name of the object.

// Load the credential information from environment variables.
// Use EnvironmentVariableCredentialsProvider to read the Access Key ID and Access Key Secret from environment variables.
$credentialsProvider = new Oss\Credentials\EnvironmentVariableCredentialsProvider();

// Use the default configurations of the SDK.
$cfg = Oss\Config::loadDefault();
$cfg->setCredentialsProvider($credentialsProvider); // Set the credential provider.
$cfg->setRegion($region); // Set the region where the bucket is located.
$cfg->setEndpoint('http://oss-cn-hangzhou.aliyuncs.com'); // Set the endpoint.

// Create an OSS client instance.
$client = new Oss\Client($cfg);

// Initialize the multipart upload task.
$initResult = $client->initiateMultipartUpload(
    new Oss\Models\InitiateMultipartUploadRequest(
        bucket: $bucket,
        key: $key
    )
);

// Define the path of the large file and the part size.
$bigFileName = "/Users/yourLocalPath/yourFileName"; // Specify the path of the large file.
$partSize = 5 * 1024 * 1024; // Set the part size in bytes. In this example, the part size is set to 5 MB.
$fileSize = filesize($bigFileName); // Obtain the file size.
$partsNum = intdiv($fileSize, $partSize) + intval(1); // Calculate the number of parts.
$parts = []; // Used to store the upload result of each part.

$i = 1; // The part number starts from 1.
$file = new \GuzzleHttp\Psr7\LazyOpenStream($bigFileName, 'rb'); // Open the file stream.
while ($i <= $partsNum) {
    // Upload a single part.
    $partResult = $client->uploadPart(
        new Oss\Models\UploadPartRequest(
            bucket: $bucket,
            key: $key,
            partNumber: $i, // The current part number.
            uploadId: $initResult->uploadId, // The upload ID returned by the initiated upload task.
            contentLength: null, // Optional: The content length of the part.
            contentMd5: null, // Optional: The MD5 hash of the part content for validation.
            trafficLimit: null, // Optional: The traffic limit.
            requestPayer: null, // Optional: The requester pays for the request.
            body: new \GuzzleHttp\Psr7\LimitStream($file, $partSize, ($i - 1) * $partSize) // Read the data of the current part.
        )
    );

    // Save the part upload result.
    $part = new Oss\Models\UploadPart(
        partNumber: $i, // The part number.
        etag: $partResult->etag // The ETag value returned after the part is uploaded.
    );

    array_push($parts, $part); // Save the upload result of the current part to the part list.
    $i++; // Increment the part number to process the next part.
}

// Complete the multipart upload task.
$comResult = $client->completeMultipartUpload(
    new Oss\Models\CompleteMultipartUploadRequest(
        bucket: $bucket,
        key: $key,
        uploadId: $initResult->uploadId, // The upload ID returned by the initiated upload task.
        acl: null, // Optional: Set the access control list (ACL) of the object.
        completeMultipartUpload: new Oss\Models\CompleteMultipartUpload(
            parts: $parts // Submit the upload results of all parts.
        )
    )
);

// Print the result of the completed multipart upload.
printf(
    'status code:' . $comResult->statusCode . PHP_EOL . // The HTTP status code. For example, 200 indicates that the request is successful.
    'request id:' . $comResult->requestId . PHP_EOL .   // The request ID, which is used to debug or track requests.
    'complete multipart upload result:' . var_export($comResult, true) . PHP_EOL // The detailed result after the multipart upload is complete.
);

Scenarios

<?php

// Import the autoloader file to ensure that dependency libraries can be loaded correctly.
require_once __DIR__ . '/../vendor/autoload.php';

use AlibabaCloud\Oss\V2 as Oss;

// Define the description of command-line arguments.
$optsdesc = [
    "region" => ['help' => 'The region in which the bucket is located.', 'required' => True], // The region where the bucket is located. (Required)
    "endpoint" => ['help' => 'The domain names that other services can use to access OSS.', 'required' => False], // The endpoint that can be used to access OSS. (Optional)
    "bucket" => ['help' => 'The name of the bucket', 'required' => True], // The name of the bucket. (Required)
    "key" => ['help' => 'The name of the object', 'required' => True], // The name of the object. (Required)
];

// Convert the argument descriptions to the long option format required by getopt.
// A colon (:) after each argument indicates that the argument requires a value.
$longopts = \array_map(function ($key) {
    return "$key:";
}, array_keys($optsdesc));

// Parse the command-line arguments.
$options = getopt("", $longopts);

// Check whether the required arguments exist.
foreach ($optsdesc as $key => $value) {
    if ($value['required'] === True && empty($options[$key])) {
        $help = $value['help']; // Obtain the help information of the argument.
        echo "Error: the following arguments are required: --$key, $help" . PHP_EOL;
        exit(1); // If a required argument is missing, exit the program.
    }
}

// Extract values from the parsed arguments.
$region = $options["region"]; // The region where the bucket is located.
$bucket = $options["bucket"]; // The name of the bucket.
$key = $options["key"];       // The name of the object.

// Load the credential information from environment variables.
// Use EnvironmentVariableCredentialsProvider to read the Access Key ID and Access Key Secret from environment variables.
$credentialsProvider = new Oss\Credentials\EnvironmentVariableCredentialsProvider();

// Use the default configurations of the SDK.
$cfg = Oss\Config::loadDefault();
$cfg->setCredentialsProvider($credentialsProvider); // Set the credential provider.
$cfg->setRegion($region); // Set the region where the bucket is located.
$cfg->setEndpoint('http://oss-cn-hangzhou.aliyuncs.com'); // Set the endpoint.

// Create an OSS client instance.
$client = new Oss\Client($cfg);

// Initialize the multipart upload task.
$initResult = $client->initiateMultipartUpload(
    new Oss\Models\InitiateMultipartUploadRequest(
        bucket: $bucket,
        key: $key
    )
);

// Define the path of the large file and the part size.
$bigFileName = "/Users/yourLocalPath/yourFileName"; // Specify the path of the large file.
$partSize = 5 * 1024 * 1024; // Set the part size in bytes. In this example, the part size is set to 5 MB.
$fileSize = filesize($bigFileName); // Obtain the file size.
$partsNum = intdiv($fileSize, $partSize) + intval(1); // Calculate the number of parts.
$parts = []; // Used to store the upload result of each part.

$i = 1; // The part number starts from 1.
$file = new \GuzzleHttp\Psr7\LazyOpenStream($bigFileName, 'rb'); // Open the file stream.
while ($i <= $partsNum) {
    // Upload a single part.
    $partResult = $client->uploadPart(
        new Oss\Models\UploadPartRequest(
            bucket: $bucket,
            key: $key,
            partNumber: $i, // The current part number.
            uploadId: $initResult->uploadId, // The upload ID returned by the initiated upload task.
            contentLength: null, // Optional: The content length of the part.
            contentMd5: null, // Optional: The MD5 hash of the part content for validation.
            trafficLimit: null, // Optional: The traffic limit.
            requestPayer: null, // Optional: The requester pays for the request.
            body: new \GuzzleHttp\Psr7\LimitStream($file, $partSize, ($i - 1) * $partSize) // Read the data of the current part.
        )
    );

    // Save the part upload result.
    $part = new Oss\Models\UploadPart(
        partNumber: $i, // The part number.
        etag: $partResult->etag // The ETag value returned after the part is uploaded.
    );

    array_push($parts, $part); // Save the upload result of the current part to the part list.
    $i++; // Increment the part number to process the next part.
}


// Add the x-oss-callback and x-oss-callback-var headers.
// Define the webhook address.
$call_back_url = "http://www.example.com/callback";

// Construct the callback parameter: Specify the webhook address and the request body for the callback. Use Base64 encoding.
// Use the placeholders {var1} and {var2} to replace ${x:var1} and ${x:var2}.
$callback_body_template = "bucket={bucket}&object={object}&my_var_1={var1}&my_var_2={var2}";
$callback_body_replaced = str_replace(
    ['{bucket}', '{object}', '{var1}', '{var2}'],
    [$bucket, $key, 'value1', 'value2'],
    $callback_body_template
);
$callback = base64_encode(json_encode([
    "callbackUrl" => $call_back_url,
    "callbackBody" => $callback_body_replaced
]));

// Construct the custom variable (callback-var). Use Base64 encoding.
$callback_var = base64_encode(json_encode([
    "x:var1" => "value1",
    "x:var2" => "value2"
]));

// Complete the multipart upload task.
$comResult = $client->completeMultipartUpload(
    new Oss\Models\CompleteMultipartUploadRequest(
        bucket: $bucket,
        key: $key,
        uploadId: $initResult->uploadId, // The upload ID returned by the initiated upload task.
        acl: null, // Optional: Set the access control list (ACL) of the object.
        completeMultipartUpload: new Oss\Models\CompleteMultipartUpload(
            parts: $parts // Submit the upload results of all parts.
        ),
        callback: $callback,
        callbackVar: $callback_var,
    )
);

// Print the result of the completed multipart upload.
printf(
    'status code:' . $comResult->statusCode . PHP_EOL . // The HTTP status code. For example, 200 indicates that the request is successful.
    'request id:' . $comResult->requestId . PHP_EOL .   // The request ID, which is used to debug or track requests.
    'complete multipart upload result:' . var_export($comResult, true) . PHP_EOL // The detailed result after the multipart upload is complete.
);

References

• For the complete sample code for multipart upload, see GitHub sample.