LoongCollector is a data collector provided by Simple Log Service. To collect logs from a Linux server, you must first install the LoongCollector client on the server. Select an installation method based on the relationship between your server and the Simple Log Service project.
Limits
Prerequisites
Select an installation method
Installation method | Scenario |
This method applies only when the server is an Alibaba Cloud ECS instance, and both the ECS instance and the project belong to the same Alibaba Cloud account and the same region. | |
This method applies when the server is an Alibaba Cloud ECS instance, and both the ECS instance and the project belong to the same Alibaba Cloud account but are in different regions. | |
This method applies when the server is an Alibaba Cloud ECS instance, and the ECS instance and the project are in the same region but belong to different Alibaba Cloud accounts. | |
|
Same account and region
This method applies only when the server is an Alibaba Cloud ECS instance, and both the ECS instance and the project belong to the same Alibaba Cloud account and the same region. Simple Log Service provides a one-click method to install LoongCollector on the ECS instance. This method uses the orchestration capabilities of OOS and does not require you to log on to the ECS instance to manually perform the installation.
The one-click installation feature is integrated into the data ingestion templates of Simple Log Service. Simple Log Service provides various text log ingestion templates, such as single-line and multi-line templates. These templates differ only in their processing plugins. You can add or remove processing plugins within a template. Select a template based on the characteristics of your logs, or select any text log template and then configure the processing plugins as needed.
Perform the following operations:
Log on to the Simple Log Service console and click the target project in the project list.
Click
Log Storage. In the Logstores list, click the
icon before the target Logstore to expand the Logstore.Click the
icon after Data Collection, select the Single Line - Text Log ingestion template in the dialog box, and click Integrate Now.Click Create Machine Group, select one or more ECS instances that are in the same region as the Project, and then click Install and Create Machine Group.
Wait for the installation to complete, enter a name, and then click OK. If the installation fails, click Recreate Installation Task, re-select an ECS instance, and ensure that the ECS region and the Project region are the same.
Click Next. If the heartbeat is FAIL, click Automatic Retry and wait for about two minutes until the heartbeat becomes OK.
The one-click installation is now complete. Click Next to proceed with the collection configuration.
Same account but different regions
This method applies when the server is an Alibaba Cloud ECS instance, and both the ECS instance and the project belong to the same Alibaba Cloud account but are in different regions. In this case, you must manually download the installation package and use the public network or transfer acceleration method in the installation command.
Perform the following operations:
Download the installation package. On the server, run the download command. In the example code, you can replace
${region_id}withcn-hangzhou. To speed up the download, replace${region_id}with the Region ID of the region where the ECS instance is located.wget https://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/loongcollector.sh -O loongcollector.sh;Select a transmission method and run the installation command. Replace
${region_id}with the Region ID of the region where the project is located.Public network: This method is suitable for most scenarios, such as cross-region collection or collection from other cloud or on-premises servers. However, it is limited by bandwidth and may be unstable.
chmod +x loongcollector.sh; ./loongcollector.sh install ${region_id}-internetTransfer acceleration: This method is used for cross-region data transfer, such as from the Chinese mainland to regions outside China. It uses CDN to accelerate performance and avoid high latency and instability over the public network. However, you are charged for the traffic.
You must first enable the cross-region data transfer acceleration feature for the project before you run the installation command.
chmod +x loongcollector.sh; ./loongcollector.sh install ${region_id}-acceleration
Check the startup status. Run the command. If
loongcollector is runningis returned, the startup is successful.sudo /etc/init.d/loongcollectord statusConfigure a machine group. Simple Log Service uses machine groups to discover custom identifiers and establish heartbeat connections with LoongCollector on hosts.
On the server, write the custom string
user-defined-test-1to the custom identifier file. This string will be used in a later step.# Write a custom string to the specified file. If the folder does not exist, create it manually. The file path and name are fixed by Simple Log Service and cannot be customized. echo "user-defined-test-1" > /etc/ilogtail/user_defined_idLog on to the Simple Log Service console. In the project list, click the target project.
Click
Resources, and then click Machine Groups.Click
to the right of Machine Groups, and click Create Machine Group.
After you configure the following settings, click OK.
Set a name for the machine group. The name must be unique within the project. It must start and end with a lowercase letter or a digit, and can contain only lowercase letters, digits, hyphens (-), and underscores (_). The name must be 3 to 128 characters in length.
Machine Group Identifier: Select Custom Identifier.
Custom Identifier: Enter the configured custom identifier. It must be the same as the custom string in the custom identifier file on the server. In this example, it is
user-defined-test-1.
After the machine group is created, click the target machine group in the machine group list and check the Heartbeat status. If the status is FAIL, wait for about two minutes and manually refresh. If the heartbeat is OK, the machine group is created successfully.
After the installation is complete, you also need to perform the collection configuration to collect logs.
Different accounts but same region
This method applies when the server is an Alibaba Cloud ECS instance, and the ECS instance and the project are in the same region but belong to different Alibaba Cloud accounts. You must manually download the installation package, use the private network method in the installation command, and configure a user ID.
Perform the following operations:
Download the installation package. On the server, run the download command. In the example code, you can replace
${region_id}withcn-hangzhou. To speed up the download, replace${region_id}with the Region ID of the region where the ECS instance is located.wget https://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/loongcollector.sh -O loongcollector.sh;Run the installation command. Replace
${region_id}with the Region ID of the region where the project is located.If a connection timeout occurs, it may be because
${region_id}was replaced with the wrong value. A private network connection cannot be established between different regions, which causes a timeout. Correct the value and run the installation command again.chmod +x loongcollector.sh; ./loongcollector.sh install ${region_id}Check the startup status. Run the command. If
loongcollector is runningis returned, the startup is successful.sudo /etc/init.d/loongcollectord statusConfigure a user ID. The user ID file contains the ID of the Alibaba Cloud account to which the project belongs. This ID indicates that the account has the permission to access and collect logs from this server.
You need to configure a user ID only when collecting logs from an ECS instance that is not under your account, an on-premises server, or a server from another cloud provider. If multiple accounts need to collect logs from the same server, you can create multiple user ID files on that server.
Log on to the Simple Log Service console, hover your mouse over the profile picture in the upper-right corner, and then view and copy the account ID from the tab that appears. Make sure to copy the ID of the Alibaba Cloud account.
On the server where LoongCollector is installed, create a user ID file with the Alibaba Cloud account ID as the filename.
touch /etc/ilogtail/users/{AlibabaCloudAccountID} # If the /etc/ilogtail/users folder does not exist, create it manually. The user ID file requires only a filename and no file extension.
Configure a machine group. Simple Log Service uses machine groups to discover custom identifiers and establish heartbeat connections with LoongCollector on hosts.
On the server, write the custom string
user-defined-test-1to the custom identifier file. This string will be used in a later step.# Write a custom string to the specified file. If the folder does not exist, create it manually. The file path and name are fixed by Simple Log Service and cannot be customized. echo "user-defined-test-1" > /etc/ilogtail/user_defined_idLog on to the Simple Log Service console. In the project list, click the target project.
Click
Resources, and click Machine Groups.Click the
icon to the right of Machine Groups, and click Create Machine Group.
After you complete the configurations, click OK.
Set a name for the machine group. The name must be unique within the project. It must start and end with a lowercase letter or a digit, and can contain only lowercase letters, digits, hyphens (-), and underscores (_). The name must be 3 to 128 characters in length.
Machine Group Identifier: Select Custom Identifier.
Custom Identifier: Enter the configured custom identifier. It must be the same as the custom string in the custom identifier file on the server. In this example, it is
user-defined-test-1.
After the machine group is created, click the target machine group in the machine group list and check its Heartbeat status. If the status is FAIL, wait for about two minutes and then manually refresh. A status of OK indicates that the machine group is created successfully.
After the installation is complete, you also need to perform the collection configuration to collect logs.
Other cloud or on-premises servers
This method applies when the server is from another cloud provider or is an on-premises server. If the server is an Alibaba Cloud ECS instance, but the ECS instance and the project belong to different Alibaba Cloud accounts and are in different regions, it can also be treated as an on-premises server. You must manually download the installation package, use the public network or transfer acceleration method in the installation command, and configure a user ID.
For server limits, see Supported Linux operating systems.
Perform the following operations:
Download the installation package. On the server, run the download command. In the example code, you can replace
${region_id}withcn-hangzhou. To speed up the download, replace${region_id}with the Region ID of the region where the ECS instance is located.wget https://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/loongcollector.sh -O loongcollector.sh;Select a transmission method and run the installation command. Replace
${region_id}with the Region ID of the region where the project is located.Public network: This method is suitable for most scenarios, such as cross-region collection or collection from other cloud or on-premises servers. However, it is limited by bandwidth and may be unstable.
chmod +x loongcollector.sh; ./loongcollector.sh install ${region_id}-internetTransfer acceleration: This method is used for cross-region data transfer, such as from the Chinese mainland to regions outside China. It uses CDN to accelerate performance and avoid high latency and instability over the public network. However, you are charged for the traffic.
You must first enable the cross-region data transfer acceleration feature for the project before you run the installation command.
chmod +x loongcollector.sh; ./loongcollector.sh install ${region_id}-acceleration
Check the startup status. Run the command. If
loongcollector is runningis returned, the startup is successful.sudo /etc/init.d/loongcollectord statusConfigure a user ID. The user ID file contains the ID of the Alibaba Cloud account to which the project belongs. This ID indicates that the account has the permission to access and collect logs from this server.
You need to configure a user ID only when collecting logs from an ECS instance that is not under your account, an on-premises server, or a server from another cloud provider. If multiple accounts need to collect logs from the same server, you can create multiple user ID files on that server.
Log on to the Simple Log Service console, hover your mouse over the profile picture in the upper-right corner, and then view and copy the account ID from the tab that appears. Make sure to copy the ID of the Alibaba Cloud account.
On the server where LoongCollector is installed, create a user ID file with the Alibaba Cloud account ID as the filename.
touch /etc/ilogtail/users/{AlibabaCloudAccountID} # If the /etc/ilogtail/users folder does not exist, create it manually. The user ID file requires only a filename and no file extension.
Configure a machine group. Simple Log Service uses machine groups to discover custom identifiers and establish heartbeat connections with LoongCollector on hosts.
On the server, write the custom string
user-defined-test-1to the custom identifier file. This string will be used in a later step.# Write a custom string to the specified file. If the folder does not exist, create it manually. The file path and name are fixed by Simple Log Service and cannot be customized. echo "user-defined-test-1" > /etc/ilogtail/user_defined_idLog on to the Simple Log Service console. In the project list, click the target project.
Click
Resources, and click Machine Groups.Click
to the right of Machine Groups, and click Create Machine Group.
After you configure the settings, click OK.
Set a name for the machine group. The name must be unique within the project. It must start and end with a lowercase letter or a digit, and can contain only lowercase letters, digits, hyphens (-), and underscores (_). The name must be 3 to 128 characters in length.
Machine Group Identifier: Select Custom Identifier.
Custom Identifier: Enter the configured custom identifier. It must be the same as the custom string in the custom identifier file on the server. In this example, it is
user-defined-test-1.
After the machine group is created, click the target machine group in the machine group list and check the Heartbeat status. If the status is FAIL, wait for about two minutes and then manually refresh. If the heartbeat status is OK, the machine group is created successfully.
After the installation is complete, you also need to perform the collection configuration to collect logs.
Batch install LoongCollector
For the scenario where the ECS instances and the project belong to the same account and region, you can select multiple ECS instances during the automatic installation.
For other scenarios, use Cloud Assistant to run temporary tasks. This lets you run LoongCollector installation commands in batches on ECS instances.
Select the download and installation commands based on your installation scenario. For more information about how to use Cloud Assistant, see Create and run a command.
Upgrade and rollback
LoongCollector stops running during an upgrade and automatically starts after the upgrade is complete. The upgrade overwrites only the necessary files. Configuration files and checkpoint files are retained to ensure that no logs are lost during the upgrade.
Upgrade LoongCollector or upgrade from Logtail to LoongCollector
On the server, run the download command to get the latest installation package. In the example code, you can replace
${region_id}withcn-hangzhou. To speed up the download, replace${region_id}with the region ID of the region where the ECS instance is located.wget https://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/loongcollector.sh -O loongcollector.sh;Run the upgrade command. Use the
upgradecommand for upgrades. If you use theinstallcommand, it will perform an overwrite installation and you will lose your original configuration.chmod +x loongcollector.sh; sudo ./loongcollector.sh upgrade;If the following information is displayed, the upgrade is successful.
Upgrade loongcollector files successfully. Starting loongcollector ... Upgrade loongcollector successfully.
Roll back from LoongCollector to Logtail
You must download the logtail.sh script again. Do not use the old logtail.sh script.
On the server, run the download command to get the installation package. In the example code, you can replace
${region_id}withcn-hangzhou. To speed up the download, replace${region_id}with the region ID of the region where the ECS instance is located.wget https://logtail-release-${region_id}.oss-${region_id}.aliyuncs.com/linux64/logtail.sh -O logtail.sh;Execute the rollback command. To specify a version, such as 1.8.7, refer to the comment and replace the version number in the command.
chmod +x logtail.sh; sudo ./logtail.sh upgrade; #chmod +x logtail.sh; sudo ./logtail.sh upgrade -v 1.8.7;
Install a specific version
To install a specific version, use the -v parameter. See the following commands and replace the version number as needed.
LoongCollector
chmod +x loongcollector.sh; sudo ./loongcollector.sh install ${region_id} -v 3.2.6;Logtail
chmod +x logtail.sh; sudo ./logtail.sh install ${region_id} -v 1.8.7;
Uninstall LoongCollector
In the example code, you can replace
${region_id}withcn-hangzhou. To speed up the execution, replace${region_id}with the region ID of the region where the ECS instance is located.wget https://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/loongcollector.sh -O loongcollector.sh;Run the uninstall command.
chmod +x loongcollector.sh; sudo ./loongcollector.sh uninstall;
FAQ
How do I install LoongCollector if the server cannot connect to an external network?
On the server where you want to install
LoongCollector, rununame -mto check the system architecture. Then, on a server that can access the public network, select and run the corresponding download command. Replace${region_id}with the Region ID of the region where the project is located.ARM architecture:
wget https://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/loongcollector.sh;wget http://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/aarch64/main/loongcollector-linux64.tar.gz;x86-64 architecture:
wget https://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/loongcollector.sh;wget http://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/x86_64/main/loongcollector-linux64.tar.gz;
Copy the downloaded installation script and package to the server where you want to install
LoongCollector, and then run the following command. Replace${region_id}with the Region ID of the region where the project is located.chmod +x loongcollector.sh; ./loongcollector.sh install-local ${region_id}-internetRun the check command. If
loongcollector is runningis returned, the startup is successful.sudo /etc/init.d/loongcollectord statusBecause the server cannot access the public network, you also need to establish a connection to the public network by configuring a proxy.
How to Upgrade When a Server Cannot Connect to an External Network
On the server where you want to upgrade
LoongCollector, rununame -mto check the system architecture. Then, on a server that can access the public network, select and run the corresponding command. Replace${region_id}with the region ID of the region where the project is located.ARM architecture:
wget https://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/loongcollector.sh;wget http://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/aarch64/main/loongcollector-linux64.tar.gz;x86-64 architecture:
wget https://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/loongcollector.sh;wget http://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/x86_64/main/loongcollector-linux64.tar.gz;
Copy the installation script and package to the server where you want to upgrade
LoongCollector, and then run the command:chmod +x loongcollector.sh;./loongcollector.sh upgrade-local;
One-click installation fails
Automatic installation is supported only when the ECS instance and the project belong to the same account and region. If this condition is not met, select another installation method. If the condition is met, check whether the permission and network requirements in the Prerequisites section are met.
How do I resolve a "No Heartbeat" or "Heartbeat FAIL" status?
If the heartbeat status is FAIL, it may be because the initial heartbeat connection takes some time. Wait about two minutes and then refresh the heartbeat status. If the status is still FAIL, perform the following checks:
Confirm the relationship between the server and the Simple Log Service project. Different relationships correspond to different installation procedures.
If you find that you selected the wrong procedure, run the
chmod +x loongcollector.sh; sudo ./loongcollector.sh uninstall;command on the server to uninstall LoongCollector. Then, select the correct procedure and run it.
If the procedure is correct but the heartbeat status is FAIL, check if the
regioninformation in the/usr/local/ilogtail/ilogtail_config.jsonfile on the server where LoongCollector is installed matches the Region ID of the Simple Log Service project.If they do not match, replace ${region_id} in the installation command and run the command again. LoongCollector will update the content of the file.
This method performs an overwrite installation and will cause you to lose your original configuration. Use it with caution on servers that already have collection configurations.
If the information matches or the heartbeat status is still FAIL after reinstallation, continue with the following checks.
If the procedure requires you to set a user ID file (for a cross-account scenario), check the following:
The user ID value must be the ID of the Alibaba Cloud account. If not, modify it.
This Alibaba Cloud account ID should be the ID of the account to which the Simple Log Service project belongs, not the ID of the account to which the ECS server belongs.
Check if the content of the custom identifier configured in the machine group in the Simple Log Service console is consistent with the content in the custom identifier file on the server. If they are not consistent, modify one of them to make them consistent.
If the heartbeat status is still FAIL, check if the network requirements in the Prerequisites section are met.
References
Regions
Log on to the Simple Log Service console and click the target project in the project list.
Click the
icon to the right of the project name to go to the project overview page.In the basic information, you can view the region name of the current project. For the Region ID corresponding to the region name, see the table below.
A region represents the geographical location of the physical data center where the Alibaba Cloud service resources are located. A Region ID is the unique identifier for an Alibaba Cloud service region.
LoongCollector network transmission types
An endpoint is the access domain name for Simple Log Service. It is the URL used to access a project and its log data, and it is related to the region where the project is located. Simple Log Service provides private endpoints, public endpoints, and acceleration endpoints. You can view the endpoints as follows:
Log on to the Simple Log Service console and click the target project in the project list.
Click the
icon to the right of the project name to go to the project overview page.In the Endpoints section, you can view the endpoint information for the current project. Different network transmission methods correspond to different endpoints. A suitable network transmission method helps ensure faster and more stable log data transmission.
Network type | Applicable domain names | Description | Scenario |
Alibaba Cloud private network | Private endpoint | The Alibaba Cloud private network is a gigabit shared network. Transmitting log data over the Alibaba Cloud private network is faster and more stable than over the public network. The private network includes VPCs and the classic network. | The ECS instance and the Simple Log Service project are in the same region, or the on-premises server is connected to the private network. Note Create a Simple Log Service project in the same region as your ECS instance. This lets you collect logs from the ECS instance over the Alibaba Cloud private network without consuming public bandwidth. |
Public network | Public endpoint | When you use the public network to transmit log data, the speed and stability of data collection can be affected by network bandwidth limitations, network jitter, latency, and packet loss. | You can choose to transmit data over the public network in the following two cases:
|
Transfer acceleration | Acceleration endpoint | This method uses Alibaba Cloud CDN points of presence (POPs) to accelerate log collection. It offers significant advantages in network latency and stability compared to public network collection. However, you are charged for the traffic. | If your business server and Simple Log Service project are in different regions, such as one in the Chinese mainland and the other outside China, you may experience high network latency and unstable transmission when using the public network. In this case, you can choose to use transfer acceleration. For more information, see Transfer acceleration. |
