This topic describes how to collect logs from a server across Alibaba Cloud accounts.

Background information

If you want to use Logtail to collect logs from a server, you must install Logtail on the server, and then configure the ID of the Alibaba Cloud account for which Log Service is activated as a user identifier on the server. This way, the Alibaba Cloud account can use Logtail to collect logs from the server. If you do not configure a user identifier on a server, Log Service cannot receive the heartbeat status of the server and Logtail cannot collect logs from the server.
For example, an e-commerce enterprise has two e-commerce applications that are deployed on Elastic Compute Service (ECS) clusters in the China (Hangzhou) region. The enterprise uses two Log Service projects that reside in the China (Hangzhou) region to manage logs.
  • Application A is deployed on a Linux ECS cluster that belongs to Alibaba Cloud Account A (12****456) and Log Service is activated for the account to manage logs.
  • Application B is deployed on a Linux ECS cluster that belongs to Alibaba Cloud Account B (17****397) and Log Service is activated for the account to manage logs.

The enterprise wants to use Log Service that is activated for Alibaba Cloud Account A (12****456) to collect the logs of the two applications and store the logs in two Logstores of the same project. In this case, you must create a Logtail configuration, a machine group, and a Logstore to collect and store the logs of Application B. The Logtail configuration, machine group, and Logstore that are configured for Application A remain unchanged.

Architecture

Step 1: Create a user identifier file

  1. Log on to an ECS instance that belongs to Alibaba Cloud Account B.
    Important You must create a user identifier file on each ECS instance of ECS Cluster B.
  2. Run the following command to create a user identifier file.

    In this example, a file named the ID of Alibaba Cloud Account A is created. For more information, see Configure a user identifier. 

    touch /etc/ilogtail/users/12****456

Step 2: Create a custom identifier-based machine group

  1. Create a custom identifier file for the machine group on an ECS instance.
    Important You must create a custom identifier file for the machine group on each ECS instance of ECS Cluster B.
    1. Log on to an ECS instance that belongs to Alibaba Cloud Account B.
    2. Create a file named user_defined_id in the /etc/ilogtail/ directory and specify a custom identifier in the /etc/ilogtail/user_defined_id file.
      For example, if you want to set the custom identifier to application_b, specify application_b in the file, and then save the file. For information about file paths, see Create a custom identifier-based machine group.
  2. Create a machine group in the Log Service console.
    1. Log on to the Log Service console with Alibaba Cloud Account A.
    2. In the Projects section, click the project that you want to manage.
    3. In the left-side navigation pane, choose Resources > Machine Groups.
    4. On the Machine Groups tab, choose Machine groups > Create Machine Group.
    5. In the Create Machine Group panel, configure the parameters and click OK, as shown in the following figure.
      In the Custom Identifier field, enter the custom identifier that you specified in Step 1. For information about other parameters, see Create a custom identifier-based machine group. Machine groups
  3. Check whether the heartbeat status of each server in the machine group is OK.
    1. In the Machine Groups list, click the machine group that you created.
    2. On the Machine Group Settings page, view the status of the machine group. You can view the list of servers that use the same custom identifier. You can also view the heartbeat status of each server.
      If the Heartbeat status is OK, the ECS instance is connected to Log Service. If the status is FAIL, see What do I do if a Logtail machine group has no heartbeats? Machine group status

Step 3: Collect logs

  1. Log on to the Log Service console with Alibaba Cloud Account A.
  2. In the Import Data section, select RegEx - Text Log.
  3. In the Specify Logstore step, select the project and the Logstore, and then click Next.
  4. In the Create Machine Group step, click Use Existing Machine Groups.
  5. In the Machine Group Settings step, select the machine group that you created in Step 2, move the machine group from the Source Server Groups section to the Applied Server Groups, and then click Next.
  6. Create a Logtail configuration and click Next.

    For information about the parameters, see Collect logs in full regex mode.

    Important
    • By default, you can use only one Logtail configuration to collect each log file. The collection process of Logtail in Alibaba Cloud Account B is not stopped. In this case, the Logtail configuration of Alibaba Cloud Account A cannot take effect. To make sure that the Logtail configuration of Alibaba Cloud Account A takes effect, you can use one of the following methods:
    • After you create the Logtail configuration, delete the original Logtail configuration of Alibaba Cloud Account B to prevent repeated collection of logs. For more information, see Delete Logtail configurations.
    Logtail configuration
  7. Preview data, configure indexes, and then click Next.
    By default, Log Service enables full-text indexing. You can configure field indexes based on the logs that are collected in manual mode or automatic mode. For more information, see Configure indexes.

Related operations

If you want to migrate historical data from Alibaba Cloud Account B to the current Logstore, you can create a data transformation job in the original Logstore, and then replicate the data to the current Logstore. For more information, see Replicate data from a Logstore.
Important
If you create a data transformation job to transform data across Alibaba Cloud accounts, you must use a custom role or an AccessKey pair to grant the required permissions for the job. In this example, a custom role is used.