Set up direct data transfer for mobile apps

Last Updated: Aug 10, 2017

Background

In the era of mobile Internet, more and more data are uploaded using mobile apps. By transferring data storage to OSS, developers can focus more on their app logic.

This tutorial describes how to set up OSS-based direct data transfer for mobile apps within 30 minutes. Direct data transfer is a service that allows mobile apps to directly connect to OSS for uploading and downloading data, with only the control flow sent to an app server.

Advantages

OSS-based direct data transfer has the following advantages:

  • More secure upload/download method (temporary and flexible permission assignment and authentication)
  • Low cost. Fewer app servers. The mobile app is directly connected to the cloud storage and only the control traffic is sent to the app server.
  • High concurrency and support of massive users (OSS provides massive upload and download bandwidths.)
  • Elasticity (OSS allows unlimited storage space resizing.)
  • Convenience. You can easily connect to Media Transcoding-video multiport adapters, Image Service, CDN download acceleration, and other features.

The architecture diagram is as follows:

For details, see here.

Roles:

  • Android/iOS mobile app, which is the app installed on the end user’s mobile phone
  • OSS, short for Alibaba Cloud Object Storage Service, which stores app-uploaded data
  • RAM/STS, which generates temporary access credentials
  • App server, which is the background service developed for the Android/iOS mobile app and used to manage the tokens used for data uploading/downloading by the app and the metadata of the app-uploaded data

Steps

  1. The app applies for a temporary upload credential from the app server.

    To avoid the risk of information leakage, the Android/iOS app does not store the AccessKeyID and AccessKeySecret. Therefore, the app must request for a temporary upload credential (a token) from the app server. The token is only valid for a certain period. For example, if a token is set to be valid for 30 minutes (the time is configurable by the app server), the Android/iOS app can use this token to upload/download data to/from OSS within the 30 minutes. After the validity period, the app must request for a new token.

  2. The app server checks the validity of the above request and then returns a token to the app.

  3. After receiving the token, the mobile phone can upload/download data to/from OSS.

This tutorial describes the steps circled in red and blue in the following figure:

  • The blue circle shows how the app server generates a token.
  • The red circle shows how the Android/iOS app receives the token.

Preconditions for Setting Up Direct Data Transfer

Do the following to prepare for app data upload/download setup:

  1. Activate OSS and create a bucket.

  2. Activate STS.

    1. Log on to the OSS Console.

    2. Click Security Token for quick configuration.

    3. The Security Token Quick Configuration page is displayed.

      Note: If RAM is not activated, a pop-up dialog box appears for RAM activation. Click Activate and perform verification. Then click Start Authorization.

    4. The system performs authorization. Then save the three parameters: AccessKeyID, AccessKeySecret, and RoleAM. So far, STS activation is complete.

Set up an app server

App server sample code configuration

Note: The app in this tutorial is written in PHP. You may write your app in the language you prefer, for example, Java, Python, Go, Ruby, Node.js, and C#.

For easy development, this tutorial provides sample programs in the following languages:

The downloaded package in each language contains a configuration file named config.json. For example:

  1. {
  2. "AccessKeyID" : "",
  3. "AccessKeySecret" : "",
  4. "RoleArn" : "",
  5. "TokenExpireTime" : "900",
  6. "PolicyFile": "policy/all_policy.txt"
  7. }

Configuration items:

  • AccessKeyID: Set it to the parameter value marked as 1 in red.
  • AccessKeySecret: Set it to the parameter value marked as 2 in red.
  • RoleArn: Set it to the parameter value marked as 3 in red.
  • TokenExpireTime: indicates the expiration time of the token obtained by the Android/iOS app. The minimum value is 900s. The default value can be retained.
  • PolicyFile: indicates the file that lists the permissions the token grants. The default value can be retained.

This tutorial provides three token files defining the most common permissions in the policy folder. The token files are as follows:

  • all_policy.txt: indicates the token that grants this account the permissions of creating a bucket, deleting a bucket, uploading a file, downloading a file, and deleting a file.
  • bucket_read_policy.txt: indicates the token that grants this account the permission of reading the specified bucket.
  • bucket_read_write_policy.txt: indicates the token that grants this account the permission of reading/writing the specified bucket.

If you want to create a token to grant the read and write permissions on the specified bucket, replace $BUCKET_NAME in the bucket_read_policy.txt and bucket_read_write_policy.txt files with the name of the desired bucket.

Returned content:

  1. {
  2. "status":200,
  3. "AccessKeyId":"STS.3pYjsdgdgagdasdg",
  4. "AccessKeySecret":"rpnwO9kvEgetGdrddgsR2YrTtI",
  5. "Security":"CAES+wMIARKAAZhjH0EUOIhJMQBMjRywXq7MQ/cjLYg80Aho1ek0Jm63XMhr9Oc5s˙∂˙∂3qaPer8p1YaX1NTDiCFZWFkvlHf1pQhuxfKBc+mRR9KAbHUefqH+rdjZqjTF7p2m1wJXP8S6k+G2MpHrUe6TYBkJ43GhhTVFMuM3BZajY3VjZWOXBIODRIR1FKZjIiEjMzMzE0MjY0NzM5MTE4NjkxMSoLY2xpZGSSDgSDGAGESGTETqOio6c2RrLWRlbW8vKgoUYWNzOm9zczoqOio6c2RrLWRlbW9KEDExNDg5MzAxMDcyNDY4MThSBTI2ODQyWg9Bc3N1bWVkUm9sZVVzZXJgAGoSMzMzMTQyNjQ3MzkxMTg2OTExcglzZGstZGVtbzI=",
  6. "Expiration":"2015-12-12T07:49:09Z",
  7. }
  • status indicates the result that the app retrieves the token. The app returns a 200 status code for successful retrieval of the token.
  • AccessKeyId indicates the AccessKeyId the Android/iOS app obtains when initializing the OSS client.
  • AccessKeySecret indicates the AccessKeySecret the Android/iOS app obtains when initializing the OSS client.
  • SecurityToken indicates the token the Android/iOS app initializes.
  • Expiration indicates the time when the token expires. The Android SDK automatically determines the validity of the token and then retrieves a new one as needed.

Note: The four variables shown above are used to create a token.

Running the sample code

  • For PHP, after you download and decompress the package, modify the config.json file and run php sts.php directly to generate a token. Then set up the app server at the specified address.
  • For Java 1.7, after you download and decompress the package, run java -jar oss-token-server.jar (port). If you run java –jar oss-token-server.jar without specifying a port, the program listens to Port 7080. To change the listening port to Port 9000, run java –jar oss-token-server.jar 9000. Specify the port number as needed for another port.

Uploading files from your app to OSS

  1. After setting up the app server, write down the server address, for example, http://abc.com:8080. Then replace the app server address in the sample code with the above address.
  2. Specify the desired bucket and region for data uploading in the sample code.
  3. Click Set to load the configuration.
  4. Select an image file, set the file name saved on OSS, and upload the file. Now, you can use the OSS service on your Android app. The data of the Android app can be directly uploaded to OSS.
  5. After uploading is completed, check that the data is on OSS.

Core code explanation: OSS initialization

Initialization

The following describes how to use the Android/iOS SDK to request for a token from your app server.

  • App for Android
  1. //Initialize an OssService for uploading
  2. public OssService initOSS(String endpoint, String bucket, UIDisplayer displayer) {
  3. OSSCredentialProvider credentialProvider;
  4. //Use your own class to retrieve an STSToken
  5. //Read the server address from app server controls
  6. String stsServer = ((EditText) findViewById(R.id.stsserver)).getText().toString();
  7. //STSGetter class, which encapsulates the way of retrieving data from the app server and must be inherited from the class OSSFederationCredentialProvider. The way your app retrieves tokens depends on the protocol between the app and the app server.
  8. if (stsServer .equals("")) {
  9. credentialProvider = new STSGetter();
  10. }else {
  11. credentialProvider = new STSGetter(stsServer);
  12. }
  13. //Retrieve the bucket name from the controls
  14. bucket = ((EditText) findViewById(R.id.bucketname)).getText().toString();
  15. //Initialize an OSSClient
  16. ClientConfiguration conf = new ClientConfiguration();
  17. conf.setConnectionTimeout(15 * 1000); // Connection timeout, 15 seconds by default
  18. conf.setSocketTimeout(15 * 1000); // Socket timeout, 15 seconds by default
  19. conf.setMaxConcurrentRequest(5); // Maximum concurrent requests, 5 by default
  20. conf.setMaxErrorRetry(2); // Maximum error retries, 2 by default
  21. OSS oss = new OSSClient(getApplicationContext(), endpoint, credentialProvider, conf);
  22. return new OssService(oss, bucket, displayer);
  23. }
  • App for iOS
  1. // Initialize an OSSClient instance
  2. - (void)ossInit {
  3. //Construct a credential provider for retrieving STSTokens
  4. id<OSSCredentialProvider> credential = [[OSSFederationCredentialProvider alloc] initWithFederationTokenGetter:^OSSFederationToken * {
  5. // Implement a function to synchronize the STSToken retrieved from the server
  6. return [self getFederationToken];
  7. }];
  8. // Use the endpoint and credential provider to initialize an OSSClient
  9. client = [[OSSClient alloc] initWithEndpoint:endPoint credentialProvider:credential];
  10. }

Core code explanation: Retrieving tokens from the app server

The specific method by which the app obtains tokens from the app server must be written into the function public OSSFederationToken getFederationToken() { }. You can define the logic of this function as desired; however, the returned message must contain the variable return new OSSFederationToken(ak, sk, token, expiration). Here, ak, sk, token, and expiration must be taken from the body of the message returned by the server.

The following is an example. Note that you can specify the protocol linking the app and the app server.

  • App for Android
  1. public OSSFederationToken getFederationToken() {
  2. String stsJson;
  3. OkHttpClient client = new OkHttpClient();
  4. Request request = new Request.Builder().url(stsServer).build();
  5. try {
  6. Response response = client.newCall(request).execute();
  7. if (response.isSuccessful()) {
  8. stsJson = response.body().string();
  9. } else {
  10. throw new IOException("Unexpected code " + response);
  11. }
  12. }
  13. catch (IOException e) {
  14. e.printStackTrace();
  15. Log.e("GetSTSTokenFail", e.toString());
  16. return null;
  17. }
  18. try {
  19. JSONObject jsonObjs = new JSONObject(stsJson);
  20. String ak = jsonObjs.getString("AccessKeyId");
  21. String sk = jsonObjs.getString("AccessKeySecret");
  22. String token = jsonObjs.getString("SecurityToken");
  23. String expiration = jsonObjs.getString("Expiration");
  24. return new OSSFederationToken(ak, sk, token, expiration);
  25. }
  26. catch (JSONException e) {
  27. Log.e("GetSTSTokenFail", e.toString());
  28. e.printStackTrace();
  29. return null;
  30. }}
  • App for iOS
  1. NSURL * url = [NSURL URLWithString:STSServer];
  2. NSURLRequest * request = [NSURLRequest requestWithURL:url];
  3. OSSTaskCompletionSource * tcs = [OSSTaskCompletionSource taskCompletionSource];
  4. NSURLSession * session = [NSURLSession sharedSession];
  5. NSURLSessionTask * sessionTask = [session dataTaskWithRequest:request
  6. completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
  7. if (error) {
  8. [tcs setError:error];
  9. return;
  10. }
  11. [tcs setResult:data];
  12. }];
  13. [sessionTask resume];
  14. // Implementation of this callback must be synchronized with the returned token, so the following task is required: waitUntilFinished
  15. [tcs.task waitUntilFinished];
  16. if (tcs.task.error) {
  17. // If the network request has an error, nil is returned, indicating the token cannot be retrieved. In this case, the OSS request fails.
  18. return nil;
  19. } else {
  20. // Parse the JSON string returned for the network request, in order to get each token field and return an STSToken
  21. NSDictionary * object = [NSJSONSerialization JSONObjectWithData:tcs.task.result
  22. options:kNilOptions
  23. error:nil];
  24. OSSFederationToken * token = [OSSFederationToken new];ni
  25. token.tAccessKey = [object objectForKey:@"AccessKeyId"];
  26. token.tSecretKey = [object objectForKey:@"AccessKeySecret"];
  27. token.tToken = [object objectForKey:@"SecurityToken"];
  28. token.expirationTimeInGMTFormat = [object objectForKey:@"Expiration"];
  29. return token;
  30. }

Implementing the sample app

App server sample code for downloading

Thank you! We've received your feedback.