Use cases

Common reasons for centralizing logs across accounts:

• Unified monitoring: Centralize application and system logs from multiple accounts to simplify alerting and troubleshooting.

• Security and compliance: Aggregate operation and security logs into a dedicated audit account for compliance.

• Business migration: Move log collection to a new account during business migration.

The following example migrates log collection from a source account to a destination account.

How it works

This solution uses file-based authorization. You create authorization and identifier files on the source ECS instances, which grant the destination account permission to collect logs through Logtail.

Workflow:

1. On each source ECS instance (Account B), create an authorization file named after the destination account UID (Account A).

2. On the same instances, create a custom identifier file to group these servers in the destination account.

3. After Logtail recognizes the authorization file, it sends heartbeats to SLS in the destination account (Account A) and reports the custom identifier.

4. In the SLS console of the destination account (Account A), create a machine group using this custom identifier. Instances with normal heartbeats are automatically added to the group.

5. Apply a Logtail collection configuration to the machine group. Logtail collects logs based on the configuration and sends them to the logstore in the destination account (Account A).

Procedure

#!/bin/bash

# Replace with the 16-digit Alibaba Cloud account ID of the destination account (Account A).
TARGET_ACCOUNT_ID="{TARGET_ACCOUNT_ID}"

# Define a unique identifier for this group of machines.
# Use a combination of the business and environment names, such as "my-app-prod-servers".
MACHINE_GROUP_ID="{YOUR_MACHINE_GROUP_ID}"

# 1. Create the authorization file directory.
sudo mkdir -p /etc/ilogtail/users

# 2. Create the authorization file. The file name is the destination account ID. This grants collection permissions to that account.
sudo touch "/etc/ilogtail/users/${TARGET_ACCOUNT_ID}"

# 3. Create the custom identifier file. This file is used to identify and group the machines in the destination account.
echo "${MACHINE_GROUP_ID}" | sudo tee /etc/ilogtail/user_defined_id > /dev/null

# 4. Restart the Logtail service to apply the configuration.
# If your system does not use systemd, run 'sudo /etc/init.d/ilogtaild restart'.
sudo systemctl restart ilogtail

echo "Configuration complete. Check the heartbeat status of the machine group in the SLS console of the destination account."

Step 1: Create a user identity file

1. Log on to the ECS server that belongs to Alibaba Cloud account B.

   null

   You must create the user identity file on each ECS server in ECS cluster B.

2. Run the following command to create the user identity file.

   Set the user identity to Alibaba Cloud account A. This creates a file named after the account ID. Configure a user identity.

   touch /etc/ilogtail/users/12****456

Step 2: Create a custom identifier-based machine group

1. Create a custom identifier file for the machine group on the ECS server.

   null

   You must create the custom identifier file for the machine group on each ECS server in ECS cluster B.

   1. Log on to the ECS server that belongs to Alibaba Cloud account B.

   2. In the specified directory, create the /etc/ilogtail/user_defined_id file and add a custom identifier.

      For example, to set the custom identifier to application_b, enter application_b in the file and save it. For the file path, see Create a custom identifier-based machine group.

2. Create a machine group in the SLS console.

   1. Log on to the SLS console with Alibaba Cloud account A.

   2. In the Project List area, click the destination project.

   3. In the left-side navigation pane, choose Resources > Machine Groups.

   4. To the right of Machine Groups, click > Create Machine Group.

   5. In the Create Machine Group dialog box, configure the parameters and click OK.

      Set Custom Identifier to the custom identifier that you configured in step 1. For other parameters, see Create a machine group that uses a custom identifier.

3. Verify that the heartbeat status of the servers in the machine group is OK.

   1. In the machine group list, click the destination machine group.

   2. On the Machine Group Configurations page, view the ECS instances that use the same custom identifier and their heartbeat status.

      Heartbeat status of OK indicates that the ECS instance is connected to SLS. If the status is FAIL, see A Logtail machine group has no heartbeat.

Step 3: Collect logs

1. Log on to the SLS console with Alibaba Cloud account A.

2. Click Quick Start.

   

3. In the Quick Data Import dialog box, select Regex - Text Log.

4. In the Import Data wizard, select the destination project and logstore, and then click Next.

5. Configure Machine Group Configurations.

   1. Set Scenario to Servers.

   2. Set Installation Environment to ECS.

   3. Select the machine group that you created in Step 2. Move the machine group from Source Machine Group to Applied Machine Group, and then click Next.

6. Create a Logtail collection configuration and click Next.

   For parameter details, see Collect logs in full regex mode.

   null

   • By default, only one Logtail collection configuration can match a file. Because the collection configuration for account B is still active, the new configuration for account A cannot take effect. You can use one of the following methods to activate the configuration for account A:

     • Stop the log collection for account B. To do this, log on to the SLS console with account B and remove the Logtail collection configuration from the destination machine group. Apply a Logtail configuration to a specified machine group.

     • Add a forced collection configuration for account A. How to collect logs from a file multiple times.

   • After you create the Logtail collection configuration, delete the original configuration for Alibaba Cloud account B to prevent duplicate log collection. Delete a Logtail configuration.

   

7. Preview the data, configure indexes, and then click Next.

   SLS enables full-text indexing by default. You can also configure field indexes based on collected logs. Create indexes.

Related operations