All Products
Search
Document Center

Object Storage Service:Resumable download

Last Updated:Nov 08, 2023

Resumable download allows you to resume an interrupted object download from the position where the download was interrupted. When an interrupted download is resumed, downloaded parts are skipped and only the remaining parts are downloaded. This saves time and traffic.

Process

When you use an app to download a video on a mobile phone, if the network switches from Wi-Fi to Cellular during the download, the app automatically interrupts the download. After you enable resumable download, when the network switches from Cellular to Wi-Fi, the video is downloaded from the position at which the download task is interrupted.

The following figure provides a vivid description of resumable download.

The following figure shows the process of resumable download.

image

Header description

  • HTTP 1.1 supports the Range header. The range header allows you to specify the range of data that you want to download. The following table describes the formats of the value of the Range header:

    Data range

    Description

    Range: bytes=100-

    Specifies that the download starts from byte 101 and stops at the last byte.

    Range: bytes=100-200

    Specifies that the download starts from byte 101 and stops at byte 201. In most cases, this type of range specifying is used for multipart transmission of large objects, such as large videos.

    Range: bytes=-100

    Downloads the last 100 bytes.

    Range: bytes=0-100, 200-300

    Specifies multiple download ranges at the same time.

  • Resemble download uses the If-Match header to verify whether the object on the server changes based on the ETag header.

  • When the client initiates a request, the Range and If-Match headers are included in the request. The OSS server checks whether the ETag that is specified in the request matches the ETag of the object. If the Etags do not match, OSS returns the HTTP 412 Precondition Failed status code.

  • The OSS server supports the following headers for a GetObject request: Range, If-Match, If-None-Match, If-Modified-Since, and If-Unmodified-Since. You can use resumable download to download resources from OSS on your mobile device.

Sample code

Important

The following sample code is for reference only. We recommend that you do not use it in production projects. You can download the code by yourself or use third-party open source download software.

The following sample code provides an example on how to use resumable download in OSS SDK for iOS:

#import "DownloadService.h"
#import "OSSTestMacros.h"

@implementation DownloadRequest

@end

@implementation Checkpoint

- (instancetype)copyWithZone:(NSZone *)zone {
    Checkpoint *other = [[[self class] allocWithZone:zone] init];

    other.etag = self.etag;
    other.totalExpectedLength = self.totalExpectedLength;

    return other;
}

@end


@interface DownloadService()<NSURLSessionTaskDelegate, NSURLSessionDataDelegate>

@property (nonatomic, strong) NSURLSession *session;         // The network session. 
@property (nonatomic, strong) NSURLSessionDataTask *dataTask;   // The data request task. 
@property (nonatomic, copy) DownloadFailureBlock failure;    // The request failure. 
@property (nonatomic, copy) DownloadSuccessBlock success;    // The request success. 
@property (nonatomic, copy) DownloadProgressBlock progress;  // The download progress. 
@property (nonatomic, copy) Checkpoint *checkpoint;        // The checkpoint file. 
@property (nonatomic, copy) NSString *requestURLString;    // The object resource URL used in a download request. 
@property (nonatomic, copy) NSString *headURLString;       // The object resource URL used in a HEAD request. 
@property (nonatomic, copy) NSString *targetPath;     // The path to which the object is stored. 
@property (nonatomic, assign) unsigned long long totalReceivedContentLength; // The size of the downloaded content. 
@property (nonatomic, strong) dispatch_semaphore_t semaphore;

@end

@implementation DownloadService

- (instancetype)init
{
    self = [super init];
    if (self) {
        NSURLSessionConfiguration *conf = [NSURLSessionConfiguration defaultSessionConfiguration];
        conf.timeoutIntervalForRequest = 15;

        NSOperationQueue *processQueue = [NSOperationQueue new];
        _session = [NSURLSession sessionWithConfiguration:conf delegate:self delegateQueue:processQueue];
        _semaphore = dispatch_semaphore_create(0);
        _checkpoint = [[Checkpoint alloc] init];
    }
    return self;
}
// DownloadRequest is the core of the download logic. 
+ (instancetype)downloadServiceWithRequest:(DownloadRequest *)request {
    DownloadService *service = [[DownloadService alloc] init];
    if (service) {
        service.failure = request.failure;
        service.success = request.success;
        service.requestURLString = request.sourceURLString;
        service.headURLString = request.headURLString;
        service.targetPath = request.downloadFilePath;
        service.progress = request.downloadProgress;
        if (request.checkpoint) {
            service.checkpoint = request.checkpoint;
        }
    }
    return service;
}

/**
 * Obtain object information by calling the Head method. OSS compares the ETag of the object with the ETag stored in the local checkpoint file and returns the comparison result. 
 */
- (BOOL)getFileInfo {
    __block BOOL resumable = NO;
    NSURL *url = [NSURL URLWithString:self.headURLString];
    NSMutableURLRequest *request = [[NSMutableURLRequest alloc]initWithURL:url];
    [request setHTTPMethod:@"HEAD"];
    // Process the information about the object. For example, the ETag is used for precheck during resumable upload, and the Content-Length header is used to calculate the download progress. 
    NSURLSessionDataTask *task = [[NSURLSession sharedSession] dataTaskWithRequest:request completionHandler:^(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error) {
        if (error) {
            NSLog(@"Failed to obtain object metadata. error: %@", error);
        } else {
            NSHTTPURLResponse *httpResponse = (NSHTTPURLResponse *)response;
            NSString *etag = [httpResponse.allHeaderFields objectForKey:@"Etag"];
            if ([self.checkpoint.etag isEqualToString:etag]) {
                resumable = YES;
            } else {
                resumable = NO;
            }
        }
        dispatch_semaphore_signal(self.semaphore);
    }];
    [task resume];

    dispatch_semaphore_wait(self.semaphore, DISPATCH_TIME_FOREVER);
    return resumable;
}

/**
 * Query the size of the local file. 
 */
- (unsigned long long)fileSizeAtPath:(NSString *)filePath {
    unsigned long long fileSize = 0;
    NSFileManager *dfm = [NSFileManager defaultManager];
    if ([dfm fileExistsAtPath:filePath]) {
        NSError *error = nil;
        NSDictionary *attributes = [dfm attributesOfItemAtPath:filePath error:&error];
        if (!error && attributes) {
            fileSize = attributes.fileSize;
        } else if (error) {
            NSLog(@"error: %@", error);
        }
    }

    return fileSize;
}

- (void)resume {
    NSURL *url = [NSURL URLWithString:self.requestURLString];
    NSMutableURLRequest *request = [[NSMutableURLRequest alloc]initWithURL:url];
    [request setHTTPMethod:@"GET"];

    BOOL Resumable= [self getFileInfo]; // If the value of the resumable field is NO, the resumable download condition is not met. 
    if (resumable) {
        self.totalReceivedContentLength = [self fileSizeAtPath:self.targetPath];
        NSString *requestRange = [NSString stringWithFormat:@"bytes=%llu-", self.totalReceivedContentLength];
        [request setValue:requestRange forHTTPHeaderField:@"Range"];
    } else {
        self.totalReceivedContentLength = 0;
    }

    if (self.totalReceivedContentLength == 0) {
        [[NSFileManager defaultManager] createFileAtPath:self.targetPath contents:nil attributes:nil];
    }

    self.dataTask = [self.session dataTaskWithRequest:request];
    [self.dataTask resume];
}

- (void)pause {
    [self.dataTask cancel];
    self.dataTask = nil;
}

- (void)cancel {
    [self.dataTask cancel];
    self.dataTask = nil;
    [self removeFileAtPath: self.targetPath];
}

- (void)removeFileAtPath:(NSString *)filePath {
    NSError *error = nil;
    [[NSFileManager defaultManager] removeItemAtPath:self.targetPath error:&error];
    if (error) {
        NSLog(@"remove file with error : %@", error);
    }
}

#pragma mark - NSURLSessionDataDelegate

- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task
// Check whether the download task is complete and return the result to the upper layer. 
didCompleteWithError:(nullable NSError *)error {
    NSHTTPURLResponse *httpResponse = (NSHTTPURLResponse *)task.response;
    if ([httpResponse isKindOfClass:[NSHTTPURLResponse class]]) {
        if (httpResponse.statusCode == 200) {
            self.checkpoint.etag = [[httpResponse allHeaderFields] objectForKey:@"Etag"];
            self.checkpoint.totalExpectedLength = httpResponse.expectedContentLength;
        } else if (httpResponse.statusCode == 206) {
            self.checkpoint.etag = [[httpResponse allHeaderFields] objectForKey:@"Etag"];
            self.checkpoint.totalExpectedLength = self.totalReceivedContentLength + httpResponse.expectedContentLength;
        }
    }

    if (error) {
        if (self.failure) {
            NSMutableDictionary *userInfo = [NSMutableDictionary dictionaryWithDictionary:error.userInfo];
            [userInfo oss_setObject:self.checkpoint forKey:@"checkpoint"];

            NSError *tError = [NSError errorWithDomain:error.domain code:error.code userInfo:userInfo];
            self.failure(tError);
        }
    } else if (self.success) {
        self.success(@{@"status": @"success"});
    }
}

- (void)URLSession:(NSURLSession *)session dataTask:(NSURLSessionDataTask *)dataTask didReceiveResponse:(NSURLResponse *)response completionHandler:(void (^)(NSURLSessionResponseDisposition))completionHandler
{
    NSHTTPURLResponse *httpResponse = (NSHTTPURLResponse *)dataTask.response;
    if ([httpResponse isKindOfClass:[NSHTTPURLResponse class]]) {
        if (httpResponse.statusCode == 200) {
            self.checkpoint.totalExpectedLength = httpResponse.expectedContentLength;
        } else if (httpResponse.statusCode == 206) {
            self.checkpoint.totalExpectedLength = self.totalReceivedContentLength +  httpResponse.expectedContentLength;
        }
    }

    completionHandler(NSURLSessionResponseAllow);
}
// Write the received network data to the object by using append upload and update the download progress. 
- (void)URLSession:(NSURLSession *)session dataTask:(NSURLSessionDataTask *)dataTask didReceiveData:(NSData *)data {

    NSFileHandle *fileHandle = [NSFileHandle fileHandleForWritingAtPath:self.targetPath];
    [fileHandle seekToEndOfFile];
    [fileHandle writeData:data];
    [fileHandle closeFile];

    self.totalReceivedContentLength += data.length;
    if (self.progress) {
        self.progress(data.length, self.totalReceivedContentLength, self.checkpoint.totalExpectedLength);
    }
}

@end
  • The following sample code provides an example on how to define DownloadRequest:

    #import <Foundation/Foundation.h>
    
    typedef void(^DownloadProgressBlock)(int64_t bytesReceived, int64_t totalBytesReceived, int64_t totalBytesExpectToReceived);
    typedef void(^DownloadFailureBlock)(NSError *error);
    typedef void(^DownloadSuccessBlock)(NSDictionary *result);
    
    @interface Checkpoint : NSObject<NSCopying>
    
    @property (nonatomic, copy) NSString *etag;     // The ETag value of the resource. 
    @property (nonatomic, assign) unsigned long long totalExpectedLength;    // The total size of the object. 
    
    @end
    
    @interface DownloadRequest : NSObject
    
    @property (nonatomic, copy) NSString *sourceURLString;      // The URL for the download. 
    
    @property (nonatomic, copy) NSString *headURLString;        // The URL used to obtain the metadata of the object. 
    
    @property (nonatomic, copy) NSString *downloadFilePath;     // The local path to which the downloaded object is stored. 
    
    @property (nonatomic, copy) DownloadProgressBlock downloadProgress; // The download progress. 
    
    @property (nonatomic, copy) DownloadFailureBlock failure;   // The callback that is sent after the download fails. 
    
    @property (nonatomic, copy) DownloadSuccessBlock success;   // The callback that is sent after the download succeeds. 
    
    @property (nonatomic, copy) Checkpoint *checkpoint;        // The checkpoint file that stores the ETag value of the object. 
    
    @end
    
    
    @interface DownloadService : NSObject
    
    + (instancetype)downloadServiceWithRequest:(DownloadRequest *)request;
    
    /**
     * Start the download. 
     */
    - (void)resume;
    
    /**
     * Pause the download. 
     */
    - (void)pause;
    
    /**
     * Cancel the download. 
     */
    - (void)cancel;
    
    @end
  • The following sample code provides an example of the invocation code for the upper layer:

    - (void)initDownloadURLs {
        OSSPlainTextAKSKPairCredentialProvider *pCredential = [[OSSPlainTextAKSKPairCredentialProvider alloc] initWithPlainTextAccessKey:OSS_ACCESSKEY_ID secretKey:OSS_SECRETKEY_ID];
        _mClient = [[OSSClient alloc] initWithEndpoint:OSS_ENDPOINT credentialProvider:pCredential];
    
        // Generate a signed URL for GET requests. 
        OSSTask *downloadURLTask = [_mClient presignConstrainURLWithBucketName:@"aliyun-dhc-shanghai" withObjectKey:OSS_DOWNLOAD_FILE_NAME withExpirationInterval:1800];
        [downloadURLTask waitUntilFinished];
        _downloadURLString = downloadURLTask.result;
    
        // Generate a signed URL for HEAD requests. 
        OSSTask *headURLTask = [_mClient presignConstrainURLWithBucketName:@"aliyun-dhc-shanghai" withObjectKey:OSS_DOWNLOAD_FILE_NAME httpMethod:@"HEAD" withExpirationInterval:1800 withParameters:nil];
        [headURLTask waitUntilFinished];
    
        _headURLString = headURLTask.result;
    }
    
    - (IBAction)resumeDownloadClicked:(id)sender {
        _downloadRequest = [DownloadRequest new];
        _downloadRequest.sourceURLString = _downloadURLString;       // Specify the resource URL. 
        _downloadRequest.headURLString = _headURLString;
        NSString *documentPath = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES).firstObject;
        _downloadRequest.downloadFilePath = [documentPath stringByAppendingPathComponent:OSS_DOWNLOAD_FILE_NAME];   // Specify the local path to which you want to download. 
    
        __weak typeof(self) wSelf = self;
        _downloadRequest.downloadProgress = ^(int64_t bytesReceived, int64_t totalBytesReceived, int64_t totalBytesExpectToReceived) {
            // totalBytesReceived is the number of bytes cached by the client. totalBytesExpectToReceived is the total number of bytes that need to be downloaded. 
            dispatch_async(dispatch_get_main_queue(), ^{
                __strong typeof(self) sSelf = wSelf;
                CGFloat fProgress = totalBytesReceived * 1.f / totalBytesExpectToReceived;
                sSelf.progressLab.text = [NSString stringWithFormat:@"%.2f%%", fProgress * 100];
                sSelf.progressBar.progress = fProgress;
            });
        };
        _downloadRequest.failure = ^(NSError *error) {
            __strong typeof(self) sSelf = wSelf;
            sSelf.checkpoint = error.userInfo[@"checkpoint"];
        };
        _downloadRequest.success = ^(NSDictionary *result) {
            NSLog(@"The download succeeds.");
        };
        _downloadRequest.checkpoint = self.checkpoint;
    
        NSString *titleText = [[_downloadButton titleLabel] text];
        if ([titleText isEqualToString:@"download"]) {
            [_downloadButton setTitle:@"pause" forState: UIControlStateNormal];
            _downloadService = [DownloadService downloadServiceWithRequest:_downloadRequest];
            [_downloadService resume];
        } else {
            [_downloadButton setTitle:@"download" forState: UIControlStateNormal];
            [_downloadService pause];
        }
    }
    
    - (IBAction)cancelDownloadClicked:(id)sender {
        [_downloadButton setTitle:@"download" forState: UIControlStateNormal];
        [_downloadService cancel];
    }
Note

The checkpoint file can be obtained from the failure callback when the download is paused or canceled. When you restart the download, you can import the checkpoint file to the DownloadRequest, and then DownloadService uses the checkpoint file to perform consistency verification.