Set up direct data transfer for mobile apps

Last Updated: Nov 27, 2017

Background

In the era of mobile Internet, mobile apps upload more and more data every day. By handing off their data storage issues to OSS, developers can focus more on their app logic.

This article describes how to set up an OSS-based direct data transfer service for a mobile app in 30 minutes. Direct data transfer is a service that allows a mobile app to directly connect to OSS for data upload and download, while only sending the control traffic to the app server.

Advantages

Setting up an OSS-based direct data transfer service for a mobile app offers the following advantages:

  • More secure upload/download method (temporary and flexible permission assignment and authentication).
  • Low cost. Fewer app servers. The mobile app is directly connected to the cloud storage and only the control traffic is sent to the app server.
  • High concurrency and support for a massive amount of users (OSS has massive bandwidth for uploading and downloading use).
  • Elasticity (OSS’s storage space can be expanded unlimitedly).
  • Convenience. You can easily connect to the MTS -video multiport adapter, Image Service, CDN download acceleration, and other services.

The architecture diagram is as follows:

For more information, see here.

Details:

  • Android/iOS mobile app, which is the app installed on the end user’s mobile phone
  • OSS, short for Alibaba Cloud Object Storage Service, which stores app-uploaded data (For more information, see the OSS description on Alibaba Cloud website.
  • RAM/STS, which generates temporary access credentials
  • App server, which is the background service developed for the Android/iOS mobile app and used to manage the tokens used for data uploading/downloading by the app and the metadata of the app-uploaded data.

Steps:

  1. Request for a temporary upload credential from the app server.

    The Android/iOS app cannot store AccessKeyID/AccessKeySecret directly, which may cause the risk of information leakage. Therefore, the app must request a temporary upload credential (a token) from the app server. The token is only valid for a certain period. For example, if a token is set to be valid for 30 minutes (editable by the app server), then the Android/iOS app can use this token to upload/download data to/from the OSS within the next 30 minutes. 30 minutes later, the app must request a new token to upload/download data.

  2. The app server checks the validity of the preceding request and then returns a token to the app.

  3. After the cell phone receives this token, it can upload or download data from the OSS.

This article mainly describes the content in the red circle and blue circle of the following figure.

  • The blue circle shows how the app server generates a token.
  • The red circle shows how the Android/iOS app receives the token.

Prerequisites for setting up direct data transfer service

Preparations for setting up direct data transfer service:

  1. Activate the OSS service and create a bucket.

  2. Activate the STS service.

    1. Log on to the OSS console.

    2. On the OSS Overview page, find the Basic Settings area, and click Security Token to open the Quick Security Token Configuration page.

      Note: If RAM has not yet been activated, a dialog box to activate RAM appears. Click Activate and perform real-name registration. After the verification is finished, click Start Authorization.

    3. The system performs authorization automatically. Make sure that you save the three parameters: AccessKeyID, AccessKeySecret, and RoleArn. Meanwhile, STS activation is completed.

Set up an app server

Configuration of sample app server

Note: The app in this example is written in PHP. You may write your app in your preferred language, e.g. Java, Python, Go, Ruby, Node.js, or C#.

This tutorial provides development sample programs available for download in multiple languages. The download addresses are shown at the end of this article.

The downloaded package in each language contains a configuration file named config.json.

  1. {
  2. "AccessKeyID" : "",
  3. "AccessKeySecret" : "",
  4. "RoleArn" : "",
  5. "TokenExpireTime" : "900",
  6. "PolicyFile": "policy/all_policy.txt"
  7. }

Note:

  • TokenExpireTime indicates the expiration time of the token obtained by the Android/iOS app. The minimum value is 900s. The default value can be retained.
  • PolicyFile indicates the file that lists the permissions the token grants. The default value can be retained.

This document has provided three token files defining the most common permissions in the policy directory. They are:

  • all_policy.txt: specifying a token that grants permissions to create or delete a bucket, or upload, download, or delete a file for this account.
  • bucket_read_policy.txt: specifying a token that grants permission to read the specified bucket for this account.
  • bucket_read_write_policy.txt: specifying a token that grants permission to read and write the specified bucket for this account.

If you want to create a token to grant read and write permissions for the specified bucket, replace $BUCKET_NAME in the bucket_read_policy.txt and bucket_read_write_policy.txt files with the name of the specified bucket.

Explanation of the formats of returned data:

  1. //Correct result returned
  2. {
  3. "StatusCode":200,
  4. "AccessKeyId":"STS.3p***dgagdasdg",
  5. "AccessKeySecret":"rpnwO9***tGdrddgsR2YrTtI",
  6. "SecurityToken":"CAES+wMIARKAAZhjH0EUOIhJMQBMjRywXq7MQ/cjLYg80Aho1ek0Jm63XMhr9Oc5s˙?˙?3qaPer8p1YaX1NTDiCFZWFkvlHf1pQhuxfKBc+mRR9KAbHUefqH+rdjZqjTF7p2m1wJXP8S6k+G2MpHrUe6TYBkJ43GhhTVFMuM3BZajY3VjZWOXBIODRIR1FKZjIiEjMzMzE0MjY0NzM5MTE4NjkxMSoLY2xpZGSSDgSDGAGESGTETqOio6c2RrLWRlbW8vKgoUYWNzOm9zczoqOio6c2RrLWRlbW9KEDExNDg5MzAxMDcyNDY4MThSBTI2ODQyWg9Bc3N1bWVkUm9sZVVzZXJgAGoSMzMzMTQyNjQ3MzkxMTg2OTExcglzZGstZGVtbzI=",
  7. "Expiration":"2015-12-12T07:49:09Z",
  8. }
  9. //Wrong result returned
  10. {
  11. "StatusCode":500,
  12. "ErrorCode":"InvalidAccessKeyId.NotFound",
  13. "ErrorMessage":"Specified access key is not found."
  14. }

Explanation of correct result returned: (The following five variables comprise a token)

  • StatusCode: The status indicates the result that the app retrieves the token. The app returns 200 for successful retrieval of the token.
  • AccessKeyId: indicates the AccessKeyId the Android/iOS app obtains when initializing the OSS client.
  • AccessKeySecret: indicates the AccessKeySecret the Android/iOS app obtains when initializing the OSS client.
  • SecurityToken: indicates the token the Android/iOS app initializes.
  • Expiration: indicates the time when the token expires. The Android SDK automatically determines the validity of the token and retrieves a new one as needed.

Explanation of wrong result returned:

  • StatusCode: The status indicates the result that the app retrieves the token. The app returns 500 for unsuccessful retrieval of the token.
  • ErrorCode: indicates the error causes.
  • ErrorMessage: indicates the detailed information about the error.

Method for running sample code:

For PHP, download and unzip a pack, modify the config.json file, run php sts.php to generate a token, and deploy the program to the specified address.

  • For Java (based on Java 1.7), after downloading and unzipping a pack,

    Run this command: java -jar oss-token-server.jar (port). If you run java –jar oss-token-server.jar without specifying a port, the program listens to Port 7080. To change the listening port to 9000, run java –jar app-token-server.jar 9000. Specify the port number as needed.

How to upload files from your app to oss

  1. After setting up the app server, write down the server address, which is http://abc.com:8080. Then, replace the app server address in the sample project with this address.
  2. Specify the bucket and region for the upload in the sample apps.
  3. Click Set to load the configuration.
  4. Select an image file, set the object name to upload to OSS, and select Upload. Now you can experience the OSS service on Android. Data from the Android app can be uploaded directly to OSS.
  5. After the upload is complete, check that the data is on OSS.

Explanation of core code

OSS initialization

The following explains how to use the Android/iOS SDK to request a token from your app server.

  • Android version
  1. //Initialize an OssService for upload and download.
  2. public OssService initOSS(String endpoint, String bucket, UIDisplayer displayer) {
  3. OSSCredentialProvider credentialProvider;
  4. //Use your own class to retrieve an STSToken.
  5. //Read the server address from app server controls.
  6. String stsServer = ((EditText) findViewById(R.id.stsserver)).getText().toString();
  7. //STSGetter class, encapsulating the way of retrieving data from the app server, must be inherited from the class OSSFederationCredentialProvider. The way that your app retrieves tokens depends on the protocol between the app and the app server.
  8. if (stsServer .equals("")) {
  9. credentialProvider = new STSGetter();
  10. }else {
  11. credentialProvider = new STSGetter(stsServer);
  12. }
  13. //Retrieve the bucket name from the controls.
  14. bucket = ((EditText) findViewById(R.id.bucketname)).getText().toString();
  15. //Initialize an OSSClient.
  16. ClientConfiguration conf = new ClientConfiguration();
  17. conf.setConnectionTimeout(15 * 1000); // Connection timeout. The default value is 15 seconds.
  18. conf.setSocketTimeout(15 * 1000); // Socket timeout. The default value is 15 seconds.
  19. conf.setMaxConcurrentRequest(5); // The maximum number of concurrent requests. The default value is 5.
  20. conf.setMaxErrorRetry(2); // The maximum number of retry attempts after each failed attempt. The default value is 2.
  21. OSS oss = new OSSClient(getApplicationContext(), endpoint, credentialProvider, conf);
  22. return new OssService(oss, bucket, displayer);
  23. }
  • iOS version
  1. //Initialize an OSSClient instance.
  2. - (void)ossInit {
  3. //Construct a credential provider for retrieving STSTokens.
  4. id<OSSCredentialProvider> credential = [[OSSFederationCredentialProvider alloc] initWithFederationTokenGetter:^OSSFederationToken * {
  5. //Implement a function to synchronize the STSToken retrieved from the server.
  6. return [self getFederationToken];
  7. }];
  8. //Use endpoint and the credential provider to initialize an OSSClient.
  9. client = [[OSSClient alloc] initWithEndpoint:endPoint credentialProvider:credential];
  10. }

Retrieve tokens from app server for mobile app

The specific method by which the app gets tokens from the app server must be written into the function public OSSFederationToken getFederationToken() { }.

Note: you can define the logic for this function; however, the return message must contain this variable: return new OSSFederationToken(ak, sk, token, expiration). Here, ak, sk, token, and expiration must be taken from the body of the message returned by the server.

In this example, you can specify the protocol linking the app and app server.

  • Android version
  1. public OSSFederationToken getFederationToken() {
  2. String stsJson;
  3. OkHttpClient client = new OkHttpClient();
  4. Request request = new Request.Builder().url(stsServer).build();
  5. try {
  6. Response response = client.newCall(request).execute();
  7. if (response.isSuccessful()) {
  8. stsJson = response.body().string();
  9. } else {
  10. throw new IOException("Unexpected code " + response);
  11. }
  12. }
  13. catch (IOException e) {
  14. e.printStackTrace();
  15. Log.e("GetSTSTokenFail", e.toString());
  16. return null;
  17. }
  18. try {
  19. JSONObject jsonObjs = new JSONObject(stsJson);
  20. String ak = jsonObjs.getString("AccessKeyId");
  21. String sk = jsonObjs.getString("AccessKeySecret");
  22. String token = jsonObjs.getString("SecurityToken");
  23. String expiration = jsonObjs.getString("Expiration");
  24. return new OSSFederationToken(ak, sk, token, expiration);
  25. }
  26. catch (JSONException e) {
  27. Log.e("GetSTSTokenFail", e.toString());
  28. e.printStackTrace();
  29. return null;
  30. }}
  • iOS version
  1. NSURL * url = [NSURL URLWithString:STSServer];
  2. NSURLRequest * request = [NSURLRequest requestWithURL:url];
  3. OSSTaskCompletionSource * tcs = [OSSTaskCompletionSource taskCompletionSource];
  4. NSURLSession * session = [NSURLSession sharedSession];
  5. NSURLSessionTask * sessionTask = [session dataTaskWithRequest:request
  6. completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
  7. if (error) {
  8. [tcs setError:error];
  9. return;
  10. }
  11. [tcs setResult:data];
  12. }];
  13. [sessionTask resume];
  14. // Implementation of this callback must be synchronized with the returned token, so the task waitUntilFinished is necessary.
  15. [tcs.task waitUntilFinished];
  16. if (tcs.task.error) {
  17. // If the network request fails, the return of nil indicates the token cannot be retrieved. In this case, this OSS request fails.
  18. return nil;
  19. } else {
  20. // Parse the JSON string returned to the network request to get each token field and return an STSToken.
  21. NSDictionary * object = [NSJSONSerialization JSONObjectWithData:tcs.task.result
  22. options:kNilOptions
  23. error:nil];
  24. OSSFederationToken * token = [OSSFederationToken new];ni
  25. token.tAccessKey = [object objectForKey:@"AccessKeyId"];
  26. token.tSecretKey = [object objectForKey:@"AccessKeySecret"];
  27. token.tToken = [object objectForKey:@"SecurityToken"];
  28. token.expirationTimeInGMTFormat = [object objectForKey:@"Expiration"];
  29. return token;
  30. }

Download source code

Example program

Download sample code of app server

Thank you! We've received your feedback.