All Products
Search
Document Center

Simple Log Service:Build a service to upload logs from mobile apps to Simple Log Service

Last Updated:Jan 23, 2024

Mobile apps are used to upload data based on the rapid development of mobile Internet technologies. You can upload the logs of mobile apps to Simple Log Service without the need of log transfer over the servers of the apps. In the following sections, this type of upload service is referred to as direct data transfer. This topic describes how to build a direct data transfer service to upload the logs of mobile apps to Simple Log Service.

Background information

Proxy-based data transfer

image

In the traditional proxy-based data transfer solution, an app server is used as a transit station to process logs that are uploaded from mobile devices. If the number of logs from mobile apps is large or the number of mobile devices rapidly increases, the use of a single app server may become a bottleneck for data upload.

Direct data transfer

You can write your AccessKey pair to the code of a mobile app and upload the logs of the mobile app to Simple Log Service based on the code. This may cause your AccessKey pair to be leaked. If your AccessKey pair is leaked, the security of cloud services within your Alibaba Cloud account is threatened. You can also use a Security Token Service (STS) token to directly access and upload the logs of a mobile app to Simple Log Service. An STS token is valid for a custom period of time and defines the required access permissions.

Benefits

  • Higher access security: An STS token can be used. Permissions are granted based on the principle of least privilege.

  • Lower costs: The server costs are reduced because a smaller number of app servers are required.

  • Higher concurrency: A large number of users can concurrently use the service. More upload bandwidth and download bandwidth are provided by Simple Log Service.

  • Auto scaling: Simple Log Service allows you to increase your storage capacity without imposing limits.

Process

image

Terms

Node

Description

Android app or iOS app

The mobile app of the user from which logs are collected.

Simple Log Service

Simple Log Service, which stores the logs uploaded from the mobile app.

RAM/STS

Resource Access Management (RAM) or STS, which generates temporary credentials. You can use RAM and STS to manage user identities and access to resources.

App server

The service that is used to obtain a temporary access credential and provides the metadata of data that is uploaded from the mobile app.

Process description

  1. A mobile app sends a request to obtain an STS token from your app server.

  2. Your app server checks the validity of the request based on actual conditions such as the logon status and symmetric encryption result. If the request passes the check, your app server uses the AccessKey pair of a created RAM user as the access credential to call the AssumeRole operation of STS and authorize the RAM user to assume a specified RAM role.

  3. STS checks the validity of the AccessKey pair and verifies whether the RAM user has permissions to assume the specified RAM role. If the AccessKey pair passes the check and the RAM user passes the verification, STS returns a token to your app server.

  4. After your app server obtains the STS token, it returns the STS token to the mobile app.

  5. After the mobile app obtains the STS token, it uses the STS token as an access credential to directly upload logs to Simple Log Service.

  6. Simple Log Service checks the validity of the STS token by using RAM or STS and verifies whether the specified RAM role has permissions to access the Logstore that is specified. If the STS token passes the check and the RAM role passes the verification, Simple Log Service processes the upload request from the mobile app and returns the upload result to the mobile app.

Procedure

Step 1: Create a project and a Logstore in Simple Log Service

  1. Log on to the Simple Log Service console.

  2. Create a project.

    1. In the Projects section, click Create Project.

    2. In the Create Project panel, select China (Hangzhou) for Region, specify slstestproject for Project Name, and then retain the default settings for other parameters.

    3. Click Create.

  3. Create a Logstore.

    After the project is created, you are prompted to create a Logstore.

    1. In the Create Project dialog box, click Create Logstore.

    2. In the Create Logstore panel, specify slstestlogstore for Logstore Name. Then, configure Billing Mode and Shards and determine whether to enable Automatic Sharding based on your business requirements. For more information, see Manage shards.

    3. Click OK.

Step 2: Create a RAM user and authorize the RAM user to assume a RAM role

  1. Log on to the RAM console.

  2. Create a RAM user.

    1. In the left-side navigation pane, choose Identities > Users.

    2. On the Users page, click Create User.

    3. On the Create User page, specify ramslsuser for Logon Name and select OpenAPI Access in the Access Mode section.

      image.png

    4. Click OK.

      After the RAM user is created, the system automatically generates an AccessKey pair for the RAM user. We recommend that you record the AccessKey pair and keep the AccessKey pair secure.

      Note

      You can also manually create an AccessKey pair for the RAM user. For more information, see Create an AccessKey pair.

  3. Authorize the RAM user to call the AssumeRole operation of STS.

    1. On the Users page, find the ramslsuser RAM user and click Add Permissions in the Actions column.

    2. In the Add Permissions panel, click System Policy in the Select Policy section. Then, find the AliyunSTSAssumeRoleAccess policy, click the policy name, and then retain the default settings for other parameters.

      image.png

    3. Click OK.

    4. Click Complete.

Step 3: Create a RAM role and authorize the RAM role to access resources

  1. Log on to the RAM console.

  2. Create a RAM role.

    1. In the left-side navigation pane, choose Identities > Roles.

    2. On the Roles page, click Create Role.

    3. In the Create Role panel, select Alibaba Cloud Account for Select Trusted Entity and click Next.

    4. Specify ramslsrole for RAM Role Name and select Current Alibaba Cloud Account for Select Trusted Alibaba Cloud Account.

    5. Click OK.

    6. Click Close.

      After the RAM role is created, find the ramslsrole RAM role on the Roles page and click the name of the RAM role. Then, view and record the Alibaba Cloud Resource Name (ARN) of the RAM role.

      image.png

  3. Create a custom policy on the JSON tab.

    1. In the left-side navigation pane, choose Permissions > Policies.

    2. On the Policies page, click Create Policy.

    3. On the Create Policy page, click the JSON tab.

    4. Enter the following policy document and click Next to edit policy information.

      {
        "Version": "1",
        "Statement": [
          {
            "Action": [
              "log:PostLogStoreLogs",
              "log:PutLogs"
            ],
            "Resource": "acs:log:*:*:project/slstestproject/*",
            "Effect": "Allow"
          }
        ]
      }
    5. Specify putlogspolicy for Name.

    6. Click OK.

  4. Attach the custom policy to the RAM role.

    1. In the left-side navigation pane, choose Identities > Roles.

    2. On the Roles page, find the ramslsrole RAM role and click Add Permissions in the Actions column.

    3. In the Add Permissions panel, click Custom Policy in the Select Policy section. Then, find the putlogspolicy policy, click the policy name, and then retain the default settings for other parameters.

      image.png

    4. Click OK.

    5. Click Complete.

Step 3: Build an app server

Sample programs in multiple programming languages are provided. You can download the programs at PHP, Java, Ruby, and Node.js.

Configuration file

In this example, each program package contains a file named config.json. The file contains configuration information of the app server. You can modify the config.json file to modify configurations.

{
    "AccessKeyID" : "",
    "AccessKeySecret" : "",
    "RoleArn" : "",
    "TokenExpireTime" : "900",
    "PolicyFile": "policy/write_policy.txt"
}
                            

Parameter

Description

AccessKeyID

The AccessKey ID of the RAM user.

AccessKeySecret

The AccessKey secret of the RAM user.

RoleArn

The ARN of the RAM role.

TokenExpireTime

The validity period of the token that is obtained by the mobile app.

Minimum value: 900. Unit: seconds. You can retain the default value.

PolicyFile

The file that describes the permissions of the token. You can retain the default value.

The following files are most commonly used to describe the permissions of a token. The files are stored in the policy directory. You can also configure a file based on your business requirements.

  • write_policy.txt: A token is granted the write permissions on the projects within an Alibaba Cloud account. When you use the write_policy.txt file, replace the project names in the file based on your business requirements.

  • readonly_policy.txt: A token is granted the read permissions on the projects within an Alibaba Cloud account.

The permissions of a token are the intersection of the permissions of the RAM role and the permissions described in the files. If you do not configure a file that describes the permissions of a token, the permissions of the RAM role are used as the permissions of the token.

Sample code

For Java 1.7 or later, create a Java project after you download and decompress a program package. Copy the required dependency, code, and configurations to the project. Then, run the main function. By default, the program listens on port 7080 and waits for HTTP requests. You can perform the preceding operations for programs in a different language.

Step 5: Use direct data transfer to upload logs from a mobile app to Simple Log Service

  1. Obtain an STS token

    Initiate an HTTP request to access port 7080 of the app server and obtain an STS token. Sample response:

    {
      "StatusCode":"200",
      "AccessKeyId":"STS.3pdgagd****",
      "AccessKeySecret":"rpnwO9wr34tGdrddgsR2Y****",
      "SecurityToken":"CAES+wMIARKAAZhjH0EUOIhJMQBMjR****tZGVtbzI=",
      "Expiration":"2021-11-20T08:23:15Z"
    }                          
  1. Directly upload logs from a mobile app to Simple Log Service

    The following code provides an example on how to use an STS token to directly upload logs from a mobile app to Simple Log Service:

    const ALY = require("aliyun-sdk");
    
    const sls = new ALY.SLS({
      accessKeyId: "Your temporary accesKeyId",
      secretAccessKey: "Your temporary accessKeySecret",
      securityToken: "Your sts token",
      endpoint: "http://cn-hangzhou.log.aliyuncs.com",
    });
    
    // -------------------------------
    // put logs
    // -------------------------------
    const projectName = "your_project_name";
    const logStoreName = "your_logstore";
    
    const logGroup = {
      logs: [
        {
          time: Math.floor(new Date().getTime() / 1000),
          contents: [
            {
              key: "a",
              value: "1",
            },
            {
              key: "a",
              value: "2",
            },
            {
              key: "a",
              value: "3",
            },
          ],
        },
      ],
      topic: "vv",
      source: "127.0.0.1",
    };
    
    sls.putLogs(
      {
        // The following fields are required.
        projectName: projectName,
        logStoreName: logStoreName,
        logGroup: logGroup,
      },
      function (err, data) {
        if (err) {
          console.log("error:", err);
          return;
        }
    
        console.log("success:", data);
      }
    );