Overview of machine groups

A machine group is a virtual collection of servers. Simple Log Service uses Logtail to manage and collect logs from these servers. You can gather logs from many servers into a logstore using machine groups.

Prerequisites

A project is created.
Logtail is installed on your servers.

A Simple Log Service project lets you apply a Logtail configuration to multiple machine groups, or bind one group to multiple Logtail configurations. To collect data from multiple servers, categorize them into machine groups based on their deployment regions or categories, then apply Logtail configurations to the specified groups. Simple Log Service delivers the configurations to all servers in the groups to collect data from them.

Create a machine group

Step 1: Configure a user identifier (optional)

Configure a user identifier on a server if one of the following conditions is met. If your Elastic Compute Service (ECS) instance and Simple Log Service belong to the same Alibaba Cloud account, skip this step.

The server is not an ECS instance.
The server is an ECS instance, but the ECS instance and Simple Log Service belong to different Alibaba Cloud accounts.

Note

A user identifier allows a server to be accessed by an Alibaba Cloud account and enables a Simple Log Service project to use Logtail for log collection from the server.

Use your Alibaba Cloud account to log on to the Simple Log Service console. Move your cursor over the profile picture in the upper-right corner, then view and copy the Account ID from the tab that appears. If you use a RAM user to log on, copy the Main Account.
Log on to the server from which you want to collect data. Then configure a user identifier by using one of the following methods to configure an Alibaba Cloud account ID file.
Linux
- Create an Alibaba Cloud account ID file in the /etc/ilogtail/users directory.
```
touch /etc/ilogtail/users/{Alibaba Cloud account ID}
```
  Important
  If the /etc/ilogtail/users directory does not exist, create it.
  After you configure or delete a user identifier, the change takes effect within 1 minute.
- To use multiple Simple Log Service projects belonging to different Alibaba Cloud accounts to collect logs from the same server, create multiple files whose names are the IDs of the Alibaba Cloud accounts on the server. For example:
```
touch /etc/ilogtail/users/{Alibaba Cloud account ID 1}
touch /etc/ilogtail/users/{Alibaba Cloud account ID 2}
```
Windows
- Create an Alibaba Cloud account ID file in the C:\LogtailData\users directory.
  - Use Windows PowerShell.
    ni C:\LogtailData\users\{Alibaba Cloud account ID}
  - Run the following command in the Command Prompt to create the file:
    type nul > C:\LogtailData\users\{Alibaba Cloud account ID}
- To use multiple Simple Log Service projects belonging to different Alibaba Cloud accounts to collect logs from the same server, create multiple files whose names are the IDs of the Alibaba Cloud accounts on the server.
Container environment
If Logtail is deployed in an Alibaba Cloud Kubernetes cluster and the version of Logtail-ds is 1.7.3 or later, you can set a custom identifier in the Container Service for Kubernetes console. On the Components page, modify the logtail-ds component and set the LogtailDSExternalUserDefinelDs parameter. For more information, see Manage components.
Note
- To set up a user identifier, configure only the file name, not the file extension.
- You can have multiple user identifiers for a server, but only one for a Logtail container.
- If you no longer need a user identifier, remove the Alibaba Cloud account ID file from the server.

Step 2: Create a machine group

Note

A Simple Log Service project can create machine groups using custom identifiers (recommended) or IP addresses. Creating groups with IP addresses is easier, but custom identifiers offer the following benefits:

They prevent IP conflicts that can occur in custom networks like VPCs, ensuring Logtail collection doesn't fail.
They support elastic scaling. New servers with the same identifier are automatically added to the machine group. To stop log collection from a server, remove the custom identifier file. It will be automatically removed from the group.

Create a custom identifier-based machine group

Typically, a system has multiple modules, and you can expand each by adding servers. To efficiently collect and organize logs, create a machine group for each module. Make sure all servers in a module share the same user identifier to ensure they belong to the correct machine group.

For example, a website has modules for HTTP requests, caching, logic processing, and storage, with identifiers such as http_module, cache_module, logic_module, and store_module.

Important

Linux and Windows servers cannot be in the same machine group. Avoid using the same Custom Identifier for both Linux and Windows servers.
A server can have multiple Custom Identifiers, separated by line breaks.

Configure a custom identifier.
Linux
1. Log on to a Linux server on which Logtail is installed, then run the following command to configure a custom identifier.
  Note
  Create the /etc/ilogtail/ directory if it does not exist.
```
echo "user-defined-1" > /etc/ilogtail/user_defined_id
```
2. (Optional) Run the following command to check the configuration. If user-defined-1 is returned, the custom identifier is configured.
```
cat /etc/ilogtail/user_defined_id
```
3. By default, changes to the user_defined_id file take effect within 1 minute after creation, deletion, or modification. To apply changes immediately, restart Logtail with the following commands:
```
/etc/init.d/ilogtaild stop
/etc/init.d/ilogtaild start
```
Windows
1. Log on to a Windows server on which Logtail is installed, create a user_defined_id file in the C:\LogtailData directory, then write user-defined-1 to the file. Save the file.
  Note
  Create the C:\LogtailData directory if it does not exist.
2. By default, changes to the user_defined_id file take effect within 1 minute after creation, deletion, or modification. To apply changes immediately, restart Logtail with the following steps:
  1. Choose Start > Control Panel > Administrative Tools > Services.
  2. In the Services dialog box, select the corresponding service.
  3. For Logtail V0.x.x.x, select LogtailWorker.
  4. For Logtail V1.0.0.0 or later, select LogtailDaemon.
  5. Right-click Restart to apply the configuration.
Container
The custom identifier is configured in the ALIYUN_LOGTAIL_USER_DEFINED_ID environment variable of the Logtail container. Run the docker inspect ${logtail_container_name} | grep ALIYUN_LOGTAIL_USER_DEFINED_ID command to view it.
Log on to the Simple Log Service console. In the Projects section, click the target project. In the left-side navigation pane, choose Resources > Machine Groups. On the Machine Groups page, click the icon and choose Create Machine Group.

In the Create Machine Group panel, configure the following parameters, then click OK.

Parameter	Description
Name	The name of the machine group. The name must meet the following requirements: Contains only lowercase letters, digits, hyphens (-), and underscores (_). Starts and ends with a lowercase letter or a digit. Is 2 to 128 characters in length. Important After you create a machine group, you cannot change its name. Proceed with caution.
Machine Group Identifier	Select Custom Identifier.
Machine Group Topic (optional)	The topic of the machine group. Used to differentiate log data generated on different servers.
Custom Identifier	Enter the custom identifier configured in Step 1, for example, `user-defined-1`.

Create an IP address-based machine group

Log on to the Simple Log Service console. In the Projects section, click the target project.
In the left-side navigation pane, choose Resources > Machine Groups. On the Machine Groups page, choose Machine Groups > > Create Machine Group.

In the Create Machine Group panel, configure the following parameters and click OK.

Parameter	Description
Name	The name of the Machine Group. The name must meet the following requirements: The name can contain only lowercase letters, digits, hyphens (-), and underscores (_). The name must start and end with a lowercase letter or a digit. The name must be 2 to 128 characters in length. Important After the machine group is created, you cannot change the name of the machine group. Proceed with caution.
Identifier	Select IP Address.
Topic	(Optional) The Topic of the machine group. The topic is used to differentiate log data that is generated on different servers. For more information, see Log topics.
IP Address	Enter the IP address that is automatically obtained by Logtail: On a server on which Logtail is installed, open the `app_info.json` file and view the value of the `ip` field. The path of the `app_info.json` file is as follows. The IP address that is automatically obtained by Logtail is recorded in the `ip` field of the `app_info.json` file, as shown in the following figure. Important If you want to add multiple servers to the machine group, manually enter the IP addresses of the servers and separate the IP addresses with line feeds. A machine group cannot include both Linux and Windows servers. Do not add the IP addresses of Windows and Linux servers to the same Machine Group.

View machine group configurations

In the Machine Groups list, click one to go to the Machine Group Configurations page. Here you can view Machine Group Information, Machine Group Status, and configuration management.

If the Heartbeat status is OK, the Logtail instance on the server is connected to the Simple Log Service project.
If the Heartbeat status is FAIL, the configuration may not take effect. Wait about 2 minutes. If the status is still FAIL, other configuration issues may exist. See How do I troubleshoot an error that is related to a Logtail machine group in a host environment?

Apply a Logtail configuration to a machine group

In the Machine Groups list, click one to go to the Machine Group Configurations page. Then, click Modify in the upper-right corner of the page.
In the Manage Configuration section, select a Logtail configuration from the left side, then click the > icon to move the configuration to the Applied Logtail Configurations section on the right side.
Important
Adding a Logtail configuration to a machine group applies it to each server's Logtail. Removing it takes it away from each server's Logtail in the group.
Click Save.

Modify machine group information

In the Machine Groups list, click one to go to the Machine Group Configurations page. Then, click Modify in the upper-right corner of the page.
Modify the following information:
- Machine Group Identifier: Machine group type, which can be IP Address or Custom Identifier.
- Machine Group Topic: See Log topics.
- Custom Identifier or IP Address: You can add or remove machines from a machine group, with each machine separated by a line break. Ensure consistency with the type selected in the machine group identifier.
  Important
  Newly added machines must have Logtail installed. If not, adding them to the machine group will not automatically install Logtail, potentially resulting in a heartbeat failure.
Click Save.

Delete a machine group

In the Machine Groups list, select the one you want to delete, then choose > Delete.
In the dialog box that appears, enter the name of the machine group and click Confirm Deletion.

References

Manually install Logtail to collect text logs from servers.
The following APIs are available to manage machine groups:

Prerequisites