Set up direct data transfer for mobile apps

Last Updated: Oct 20, 2017

Background

Ever since the world of mobile Internet is revolutionized, volumes of data is being uploaded using mobile apps. By transferring data storage to OSS, developers can focus more on their app logic.

This document elaborates how to set up OSS-based direct data transfer for mobile apps in 30 minutes. Direct data transfer is a service that allows mobile apps to connect directly to OSS to upload and download data, with only the control flow sent to the app server.

Benefits

OSS-based direct data transfer has the following advantages:

  • More secure upload/download method (temporary and flexible permission assignment and authentication).

  • Low cost. Fewer app servers. The mobile app is directly connected to the cloud storage and only the control traffic is sent to the app server.

  • High concurrency and support of massive users (OSS provides massive upload and download bandwidths).

  • Elasticity (OSS allows unlimited storage space resizing).

  • Convenience. You can easily connect to Media Transcoding-video multiport adapters, Image Service, CDN download acceleration, and other features.

The architecture diagram is as follows:

For more information, see here.

Role:

  • Android/iOS mobile app: It is the app installed on the end user’s mobile phone.

  • OSS (Object Storage Service): Stores app-uploaded data.

  • RAM/STS: Generates temporary access credentials.

  • App server: It is the background service developed for the Android/iOS mobile app. It manages the tokens used for data upload/download by the app and the metadata of the app-uploaded data.

Procedure

  1. The app applies for a temporary upload credential from the app server.

    To avoid the risk of information leakage, the Android/iOS app does not store the AccessKeyID and AccessKeySecret. Therefore, the app must request for a temporary upload credential (a token) from the app server.

    The token is only valid for a certain period. For example, if a token is set to be valid for 30 minutes (the time is configurable by the app server), the Android/iOS app can use this token to upload/download data to/from OSS within 30 minutes. Once the validity period expires, the app must request for a new token.

  2. The app server checks the validity of the preceding request and then returns a token to the app.

  3. After receiving the token, the mobile phone can upload/download data to/from OSS.

  • The blue circle indicates how the app server generates a token.

  • The red circle indicates how the Android/iOS app receives the token.

Preconditions for Setting Up Direct Data Transfer

Perform following actions to prepare app data upload/download setup:

  1. Activate OSS and create a bucket.

  2. Activate STS.

    1. Log on to the OSS console.

    2. Click Security Token for quick configuration.

    3. The Security Token Quick Configuration page is displayed.

      Note: If RAM is not activated, a dialog box appears for RAM activation. Click Activate and perform verification. Then click Start Authorization.

    4. The system performs authorization and then you are required to save the three parameters: AccessKeyID, AccessKeySecret, and RoleAM. Meanwhile, STS activation is completed.

Set up an app server

App server sample code configuration

Note: The app in this tutorial is written in PHP. You may write the app in your preferred language. For example, Java, Python, Go, Ruby, Node.js, and C#.

This tutorial provides sample programs in the following languages, to help easy development of the app:

The downloaded package in each language contains a configuration file named config.json. For example:

  1. {
  2. "AccessKeyID" : "",
  3. "AccessKeySecret" : "",
  4. "RoleArn" : "",
  5. "TokenExpireTime" : "900",
  6. "PolicyFile": "policy/all_policy.txt"
  7. }

The following are the configuration items:

  • AccessKeyID: Set it to the parameter value marked as 1 in red.

  • AccessKeySecret: Set it to the parameter value marked as 2 in red.

  • RoleArn: Set it to the parameter value marked as 3 in red.

  • TokenExpireTime: It is the expiration time of the token obtained by the Android/iOS app. The minimum value is 900s. The default value can be retained.

  • PolicyFile: It is the file that enlists the permissions the token grants. The default value can be retained.

The following are the three token files that define the most common permissions in the policy folder:

  • all_policy.txt: It is a token that grants an account the permissions to create a bucket, delete a bucket, upload a file, download a file, and delete a file.

  • bucket_read_policy.txt: It is a token that grants this account the permission to read the specified bucket.

  • bucket_read_write_policy.txt: It is a token that grants this account the permission to read/write the specified bucket.

If you want to create a token to grant the read and write permissions on the specified bucket, replace $BUCKET_NAME in the bucket_read_policy.txt and bucket_read_write_policy.txt files with the name of the desired bucket.

Returned content:

  1. {
  2. "status":200,
  3. "AccessKeyId":"STS.3p***dgagdasdg",
  4. "AccessKeySecret":"rpnwO9***tGdrddgsR2YrTtI",
  5. "Security":"CAES+wMIARKAAZhjH0EUOIhJMQBMjRywXq7MQ/cjLYg80Aho1ek0Jm63XMhr9Oc5s˙∂˙∂3qaPer8p1YaX1NTDiCFZWFkvlHf1pQhuxfKBc+mRR9KAbHUefqH+rdjZqjTF7p2m1wJXP8S6k+G2MpHrUe6TYBkJ43GhhTVFMuM3BZajY3VjZWOXBIODRIR1FKZjIiEjMzMzE0MjY0NzM5MTE4NjkxMSoLY2xpZGSSDgSDGAGESGTETqOio6c2RrLWRlbW8vKgoUYWNzOm9zczoqOio6c2RrLWRlbW9KEDExNDg5MzAxMDcyNDY4MThSBTI2ODQyWg9Bc3N1bWVkUm9sZVVzZXJgAGoSMzMzMTQyNjQ3MzkxMTg2OTExcglzZGstZGVtbzI=",
  6. "Expiration":"2015-12-12T07:49:09Z",
  7. }

The following variables are used to create a token.

  • Status: indicates the result that the app retrieves the token. The app returns a 200 status code for successful retrieval of the token.

  • AccessKeyId: indicates the AccessKeyId the Android/iOS app obtains when initializing the OSS client.

  • AccessKeySecret: indicates the AccessKeySecret the Android/iOS app obtains while initializing the OSS client.

  • SecurityToken: indicates the token that Android/iOS app initializes.

  • Expiration: indicates the time when the token expires. The Android SDK automatically determines the validity of the token and then retrieves a new one as required.

Run the sample code

  • For PHP, once you download and decompress the package, modify the config.json file and run php sts.php directly to generate a token. Then set up the app server at the specified address.

  • For Java 1.7, once you download and decompress the package, run java -jar oss-token-server.jar (port). If you run java –jar oss-token-server.jar without specifying a port, the program listens to Port 7080. To change the listening port to Port 9000, run java –jar oss-token-server.jar 9000. Specify the port number as required for another port.

Upload files from your app to OSS

  1. After setting up the app server, write down the server address, for example, http://abc.com:8080. Then replace the app server address in the sample code with the preceding address.

  2. Specify the desired bucket and region for data uploading in the sample code.

  3. Click Set to load the configuration.

  4. Select an image file, set the file name saved on OSS, and upload the file. Now, you can use the OSS service on your Android app. The data of the Android app can be uploaded directly to OSS.

  5. Once uploaded, check whether the data is on OSS.

Core code explanation: OSS initialization

Initialization

The following process describes how to use the Android/iOS SDK to request for a token from your app server.

  • App for Android
  1. //Initialize an OssService for uploading
  2. public OssService initOSS(String endpoint, String bucket, UIDisplayer displayer) {
  3. OSSCredentialProvider credentialProvider;
  4. //Use your own class to retrieve an STSToken
  5. //Read the server address from app server controls
  6. String stsServer = ((EditText) findViewById(R.id.stsserver)).getText().toString();
  7. //STSGetter class, which encapsulates the way of retrieving data from the app server and must be inherited from the class OSSFederationCredentialProvider. The way your app retrieves tokens depends on the protocol between the app and the app server.
  8. if (stsServer .equals("")) {
  9. credentialProvider = new STSGetter();
  10. }else {
  11. credentialProvider = new STSGetter(stsServer);
  12. }
  13. //Retrieve the bucket name from the controls
  14. bucket = ((EditText) findViewById(R.id.bucketname)).getText().toString();
  15. //Initialize an OSSClient
  16. ClientConfiguration conf = new ClientConfiguration();
  17. conf.setConnectionTimeout(15 * 1000); // Connection timeout, 15 seconds by default
  18. conf.setSocketTimeout(15 * 1000); // Socket timeout, 15 seconds by default
  19. conf.setMaxConcurrentRequest(5); // Maximum concurrent requests, 5 by default
  20. conf.setMaxErrorRetry(2); // Maximum error retries, 2 by default
  21. OSS oss = new OSSClient(getApplicationContext(), endpoint, credentialProvider, conf);
  22. return new OssService(oss, bucket, displayer);
  23. }
  • App for iOS
  1. // Initialize an OSSClient instance
  2. - (void)ossInit {
  3. //Construct a credential provider for retrieving STSTokens
  4. id<OSSCredentialProvider> credential = [[OSSFederationCredentialProvider alloc] initWithFederationTokenGetter:^OSSFederationToken * {
  5. // Implement a function to synchronize the STSToken retrieved from the server
  6. return [self getFederationToken];
  7. }];
  8. // Use the endpoint and credential provider to initialize an OSSClient
  9. client = [[OSSClient alloc] initWithEndpoint:endPoint credentialProvider:credential];
  10. }

Core code explanation: Retrieving tokens from the app server

The specific method by which the app obtains tokens from the app server must be written into the function public OSSFederationToken getFederationToken() { }.

Note: You can define the logic of this function as desired. However, the returned message must contain the variable return new OSSFederationToken(ak, sk, token, expiration). Here, ak, sk, token, and expiration must be taken from the body of the message returned by the server.

You can specify the protocol that links the app and the app server.

  • App for Android
  1. public OSSFederationToken getFederationToken() {
  2. String stsJson;
  3. OkHttpClient client = new OkHttpClient();
  4. Request request = new Request.Builder().url(stsServer).build();
  5. try {
  6. Response response = client.newCall(request).execute();
  7. if (response.isSuccessful()) {
  8. stsJson = response.body().string();
  9. } else {
  10. throw new IOException("Unexpected code " + response);
  11. }
  12. }
  13. catch (IOException e) {
  14. e.printStackTrace();
  15. Log.e("GetSTSTokenFail", e.toString());
  16. return null;
  17. }
  18. try {
  19. JSONObject jsonObjs = new JSONObject(stsJson);
  20. String ak = jsonObjs.getString("AccessKeyId");
  21. String sk = jsonObjs.getString("AccessKeySecret");
  22. String token = jsonObjs.getString("SecurityToken");
  23. String expiration = jsonObjs.getString("Expiration");
  24. return new OSSFederationToken(ak, sk, token, expiration);
  25. }
  26. catch (JSONException e) {
  27. Log.e("GetSTSTokenFail", e.toString());
  28. e.printStackTrace();
  29. return null;
  30. }}
  • App for iOS
  1. NSURL * url = [NSURL URLWithString:STSServer];
  2. NSURLRequest * request = [NSURLRequest requestWithURL:url];
  3. OSSTaskCompletionSource * tcs = [OSSTaskCompletionSource taskCompletionSource];
  4. NSURLSession * session = [NSURLSession sharedSession];
  5. NSURLSessionTask * sessionTask = [session dataTaskWithRequest:request
  6. completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
  7. if (error) {
  8. [tcs setError:error];
  9. return;
  10. }
  11. [tcs setResult:data];
  12. }];
  13. [sessionTask resume];
  14. // Implementation of this callback must be synchronized with the returned token, so the following task is required: waitUntilFinished
  15. [tcs.task waitUntilFinished];
  16. if (tcs.task.error) {
  17. // If the network request has an error, nil is returned, indicating the token cannot be retrieved. In this case, the OSS request fails.
  18. return nil;
  19. } else {
  20. // Parse the JSON string returned for the network request, to get each token field and return an STSToken
  21. NSDictionary * object = [NSJSONSerialization JSONObjectWithData:tcs.task.result
  22. options:kNilOptions
  23. error:nil];
  24. OSSFederationToken * token = [OSSFederationToken new];ni
  25. token.tAccessKey = [object objectForKey:@"AccessKeyId"];
  26. token.tSecretKey = [object objectForKey:@"AccessKeySecret"];
  27. token.tToken = [object objectForKey:@"SecurityToken"];
  28. token.expirationTimeInGMTFormat = [object objectForKey:@"Expiration"];
  29. return token;
  30. }

Implementing the sample app

App server sample code for downloading

Thank you! We've received your feedback.