Object Storage Service:Download objects as files using OSS SDK for Java 1.0

Usage notes

• In this topic, the public endpoint of the China (Hangzhou) region is used. To access OSS from other Alibaba Cloud services in the same region, use an internal endpoint. For details about supported regions and endpoints, see Regions and endpoints.

• In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials.

• In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Configuration examples for common scenarios.

Permissions Description

By default, an Alibaba Cloud account has full permissions. RAM users or RAM roles under an Alibaba Cloud account initially have no permissions. The Alibaba Cloud account or an administrator must assign operation permissions through RAM Policy or Bucket Policy.

Download a single object

The following sample code provides an example on how to download exampleobject.txt in the testfolder directory of the examplebucket bucket to your local computer and save it to D:\localpath as examplefile.txt:

import com.aliyun.oss.*;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.common.comm.SignVersion;
import com.aliyun.oss.model.GetObjectRequest;
import java.io.File;

public class Demo {

    public static void main(String[] args) throws Exception {
        // In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint. 
        String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
        // Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
        EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
        // Specify the name of the bucket. Example: examplebucket. 
        String bucketName = "examplebucket";
        // Specify the full path of the object. Do not include the bucket name in the full path. Example: testfolder/exampleobject.txt. 
        String objectName = "testfolder/exampleobject.txt";
        // Specify the full path of the local file to which you want to download the object. 
        String pathName = "D:\\localpath\\examplefile.txt";
        // Specify the region in which the bucket is located. For example, if your bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou.
        String region = "cn-hangzhou";

        // Create an OSSClient instance. 
        // Call the shutdown method to release associated resources when the OSSClient is no longer in use.
        ClientBuilderConfiguration clientBuilderConfiguration = new ClientBuilderConfiguration();
        clientBuilderConfiguration.setSignatureVersion(SignVersion.V4);        
        OSS ossClient = OSSClientBuilder.create()
        .endpoint(endpoint)
        .credentialsProvider(credentialsProvider)
        .clientConfiguration(clientBuilderConfiguration)
        .region(region)               
        .build();

        try {
            // Download the object to the specified path. If a file that has the same name already exists, the downloaded object overwrites the file. Otherwise, the downloaded object is saved in the path. 
            // If you do not specify the local path for the downloaded object, the downloaded object is saved to the path of the project to which the sample program belongs. 
            ossClient.getObject(new GetObjectRequest(bucketName, objectName), new File(pathName));
        } catch (OSSException oe) {
            System.out.println("Caught an OSSException, which means your request made it to OSS, "
                    + "but was rejected with an error response for some reason.");
            System.out.println("Error Message:" + oe.getErrorMessage());
            System.out.println("Error Code:" + oe.getErrorCode());
            System.out.println("Request ID:" + oe.getRequestId());
            System.out.println("Host ID:" + oe.getHostId());
        } catch (ClientException ce) {
            System.out.println("Caught an ClientException, which means the client encountered "
                    + "a serious internal problem while trying to communicate with OSS, "
                    + "such as not being able to access the network.");
            System.out.println("Error Message:" + ce.getMessage());
        } finally {
            if (ossClient != null) {
                ossClient.shutdown();
            }
        }
    }
}   

Batch download objects

import com.aliyun.oss.*;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.common.comm.SignVersion;
import com.aliyun.oss.model.*;
import java.io.File;
import java.io.IOException;
import java.util.List;

public class BatchDownloadObject {
    public static void main(String[] args) throws Exception {
        // Set the Endpoint. Use the China (Hangzhou) region as an example. Set the Endpoint to the actual region.
        String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
        // Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
        EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
        // Specify the bucket name. Example: examplebucket.
        String bucketName = "examplebuckett";
        // Specify the prefix of the folder to download. Example: "images/". To download the entire bucket, leave this empty.
        String folderPrefix = "";
        // Specify the local download folder.
        String localDownloadPath = "./batch_downloads/";
        // Specify the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set Region to cn-hangzhou.
        String region = "cn-hangzhou";

        // Create an OSSClient instance.
        ClientBuilderConfiguration clientBuilderConfiguration = new ClientBuilderConfiguration();
        clientBuilderConfiguration.setSignatureVersion(SignVersion.V4);

        OSS ossClient = OSSClientBuilder.create()
                .endpoint(endpoint)
                .credentialsProvider(credentialsProvider)
                .clientConfiguration(clientBuilderConfiguration)
                .region(region)
                .build();

        try {
            // Make sure that the local download folder exists.
            createDirectoryIfNotExists(localDownloadPath);

            // Download objects in batches.
            batchDownloadObjects(ossClient, bucketName, folderPrefix, localDownloadPath);

        } catch (OSSException oe) {
            System.out.println("Caught an OSSException, which means your request made it to OSS, "
                    + "but was rejected with an error response for some reason.");
            System.out.println("Error Message:" + oe.getErrorMessage());
            System.out.println("Error Code:" + oe.getErrorCode());
            System.out.println("Request ID:" + oe.getRequestId());
            System.out.println("Host ID:" + oe.getHostId());
        } catch (ClientException ce) {
            System.out.println("Caught an ClientException, which means the client encountered "
                    + "a serious internal problem while trying to communicate with OSS, "
                    + "such as not being able to access the network.");
            System.out.println("Error Message:" + ce.getMessage());
        } finally {
            if (ossClient != null) {
                ossClient.shutdown();
            }
        }
    }

    /**
     * Downloads objects from OSS to a local path in batches.
     * @param ossClient The OSS client.
     * @param bucketName The bucket name.
     * @param folderPrefix The folder prefix.
     * @param localDownloadPath The local download path.
     */
    public static void batchDownloadObjects(OSS ossClient, String bucketName, String folderPrefix, String localDownloadPath) {
        String nextMarker = null;
        ObjectListing objectListing;
        int downloadCount = 0;
        long totalSize = 0;
        int failedCount = 0;

        System.out.println("Starting batch download...");
        System.out.println("Source path: " + bucketName + "/" + folderPrefix);
        System.out.println("Local path: " + localDownloadPath);
        System.out.println("----------------------------------------");

        long startTime = System.currentTimeMillis();

        do {
            // List objects.
            ListObjectsRequest listObjectsRequest = new ListObjectsRequest(bucketName)
                    .withPrefix(folderPrefix)
                    .withMarker(nextMarker)
                    .withMaxKeys(1000); // List a maximum of 1,000 objects at a time.

            objectListing = ossClient.listObjects(listObjectsRequest);
            List<OSSObjectSummary> sums = objectListing.getObjectSummaries();

            for (OSSObjectSummary s : sums) {
                // Skip folders (objects that end with /).
                if (s.getKey().endsWith("/")) {
                    continue;
                }

                try {
                    // Construct the local file path.
                    String localFilePath = constructLocalFilePath(localDownloadPath, s.getKey(), folderPrefix);

                    // Create the parent folder of the local file.
                    File localFile = new File(localFilePath);
                    createDirectoryIfNotExists(localFile.getParent());

                    // Check whether the file exists to avoid repeated downloads.
                    if (localFile.exists() && localFile.length() == s.getSize()) {
                        System.out.println("File already exists. Skip download: " + s.getKey());
                        downloadCount++;
                        totalSize += s.getSize();
                        continue;
                    }

                    // Download the object.
                    System.out.println("Downloading: " + s.getKey() + " -> " + localFilePath);

                    // Use a temporary file to prevent file corruption during download.
                    File tempFile = new File(localFilePath + ".tmp");
                    try {
                        ossClient.getObject(new GetObjectRequest(bucketName, s.getKey()), tempFile);

                        // After the download is complete, rename the temporary file to the final file name.
                        if (tempFile.renameTo(localFile)) {
                            downloadCount++;
                            totalSize += s.getSize();
                            System.out.println("Download complete: " + s.getKey() + " (Size: " + formatFileSize(s.getSize()) + ")");
                        } else {
                            throw new IOException("Failed to rename the temporary file: " + tempFile.getPath() + " -> " + localFile.getPath());
                        }
                    } finally {
                        // Clean up the temporary file.
                        if (tempFile.exists()) {
                            tempFile.delete();
                        }
                    }

                } catch (Exception e) {
                    failedCount++;
                    System.err.println("Failed to download the file: " + s.getKey() + ", Error: " + e.getMessage());
                    // Record detailed error messages for debugging.
                    if (e instanceof OSSException) {
                        OSSException oe = (OSSException) e;
                        System.err.println("OSS Error Code: " + oe.getErrorCode() + ", Request ID: " + oe.getRequestId());
                    }
                }
            }

            nextMarker = objectListing.getNextMarker();
        } while (objectListing.isTruncated());

        long endTime = System.currentTimeMillis();
        long duration = endTime - startTime;

        System.out.println("----------------------------------------");
        System.out.println("Batch download complete!");
        System.out.println("Total number of files downloaded: " + downloadCount);
        System.out.println("Number of failed downloads: " + failedCount);
        System.out.println("Total download size: " + formatFileSize(totalSize));
        System.out.println("Total time elapsed: " + formatDuration(duration));
        System.out.println("Average download speed: " + formatSpeed(totalSize, duration));
    }

    /**
     * Constructs the local file path.
     * @param localDownloadPath The root directory for local downloads.
     * @param objectKey The OSS object key.
     * @param folderPrefix The folder prefix.
     * @return The full path of the local file.
     */
    private static String constructLocalFilePath(String localDownloadPath, String objectKey, String folderPrefix) {
        // Remove the prefix to maintain the relative path structure.
        String relativePath = objectKey;
        if (folderPrefix != null && !folderPrefix.isEmpty() && objectKey.startsWith(folderPrefix)) {
            relativePath = objectKey.substring(folderPrefix.length());
        }

        // Make sure that the path separator is correct.
        String normalizedPath = localDownloadPath.endsWith(File.separator) ?
                localDownloadPath : localDownloadPath + File.separator;

        return normalizedPath + relativePath.replace("/", File.separator);
    }

    /**
     * Creates a folder if it does not exist.
     * @param dirPath The folder path.
     */
    private static void createDirectoryIfNotExists(String dirPath) {
        if (dirPath == null || dirPath.isEmpty()) {
            return;
        }

        try {
            File dir = new File(dirPath);
            if (!dir.exists()) {
                boolean created = dir.mkdirs();
                if (created) {
                    System.out.println("Creating folder: " + dirPath);
                } else if (!dir.exists()) {
                    throw new IOException("Failed to create folder: " + dirPath);
                }
            }
        } catch (Exception e) {
            System.err.println("Failed to create folder: " + dirPath + ", Error: " + e.getMessage());
            throw new RuntimeException("Failed to create folder", e);
        }
    }

    /**
     * Formats the file size.
     * @param size The file size in bytes.
     * @return The formatted file size string.
     */
    private static String formatFileSize(long size) {
        if (size < 1024) {
            return size + " B";
        } else if (size < 1024 * 1024) {
            return String.format("%.2f KB", size / 1024.0);
        } else if (size < 1024 * 1024 * 1024) {
            return String.format("%.2f MB", size / (1024.0 * 1024.0));
        } else {
            return String.format("%.2f GB", size / (1024.0 * 1024.0 * 1024.0));
        }
    }

    /**
     * Formats the duration.
     * @param duration The duration in milliseconds.
     * @return The formatted duration string.
     */
    private static String formatDuration(long duration) {
        long seconds = duration / 1000;
        long minutes = seconds / 60;
        long hours = minutes / 60;

        if (hours > 0) {
            return String.format("%d h %d min %d s", hours, minutes % 60, seconds % 60);
        } else if (minutes > 0) {
            return String.format("%d min %d s", minutes, seconds % 60);
        } else {
            return String.format("%d s", seconds);
        }
    }

    /**
     * Formats the download speed.
     * @param totalBytes The total number of bytes.
     * @param durationMs The duration in milliseconds.
     * @return The formatted speed string.
     */
    private static String formatSpeed(long totalBytes, long durationMs) {
        if (durationMs <= 0) {
            return "N/A";
        }

        double bytesPerSecond = (double) totalBytes / (durationMs / 1000.0);

        if (bytesPerSecond < 1024) {
            return String.format("%.2f B/s", bytesPerSecond);
        } else if (bytesPerSecond < 1024 * 1024) {
            return String.format("%.2f KB/s", bytesPerSecond / 1024.0);
        } else if (bytesPerSecond < 1024 * 1024 * 1024) {
            return String.format("%.2f MB/s", bytesPerSecond / (1024.0 * 1024.0));
        } else {
            return String.format("%.2f GB/s", bytesPerSecond / (1024.0 * 1024.0 * 1024.0));
        }
    }
}

Related API operation

The methods described above are fundamentally implemented based on the RESTful API, which you can directly call if your business requires a high level of customization. To directly call an API, you must include the signature calculation in your code. For more information, see GetObject.

Reference

• For the complete sample code that is used to download an object as a local file, visit GitHub.