You can integrate the application performance monitoring (APM) service provided by open source Elasticsearch with Alibaba Cloud Elasticsearch to build an APM system, which helps achieve observability of services and applications. This topic describes how to use a self-managed APM server to collect data to Alibaba Cloud Elasticsearch.
Background information
Observability is the capability to measure the performance of your infrastructures, platforms, and applications, helping you understand how they work. Compared with traditional monitoring services, the mainstream monitoring services focus on issue identification and alerting. Observability can provide explanations for all events that occur on a complex distributed system. The monitoring capability focuses on the service status of software during and after delivery, whereas the observability capability focuses on the entire lifecycle of the R&D and O&M process.Logs, metrics for infrastructures, and APM are the key elements of observability. APM bridges the gap between metrics and logs. Logs and metrics provide information about infrastructures and components and are partly overlapped. APM focuses on applications and allows IT and development engineers to monitor the application layer of a stack and user experience. You can integrate APM into a system monitoring service to obtain the following benefits:
- Helps you understand what takes the longest period of time when services are running, and the reasons why services stop responding.
- Helps you understand how services interact with each other and view the bottlenecks of the services.
- Helps you understand how services interact with each other and view the bottlenecks of the services.
- Improves the productivity of the development team.
- Tracks user experience in browsers.
APM is widely used in the following scenarios:
- User experience monitoring: APM can improve user experience by monitoring user behaviors. For example, APM can monitor the interactions between users and web interfaces or clients and record the time when the interactions occur.
- Runtime application architectures: APM can help you understand the dependencies between services, and the network typologies of interactions between applications.
- Business transactions: APM can generate in-depth SLA reports and provide trend information about application performance from the business perspective.
- Deep dive component monitoring: If you want to use APM in this scenario, you must install an APM agent. APM is mainly used to monitor the middle layer of a service, including web servers, applications, and message servers. The robust monitoring capability provided by APM can display the clear path of code execution. Deep dive component monitoring is closely correlated with runtime application architectures. In most cases, the scenarios overlap with each other, and APM is concurrently used in the scenarios.
- Analytics and reporting: APM can perform analytics on metric data that is collected from applications and display the analysis results in common views in a standard manner.
Prerequisites
- An Alibaba Cloud Elasticsearch V7.10 cluster is created. For more information, see Create an Alibaba Cloud Elasticsearch cluster.
- An Elastic Compute Service (ECS) instance whose operating system is Linux is created. For more information, see Connect to a Linux instance by using a password or key.
- The Go programming language is installed in the ECS instance. Note In this example, an APM agent that uses the Go programming language is used. You must install the Go programming language in advance.
Procedure
Step 1: Build an APM server
- Connect to the ECS instance. For more information, see Connect to a Linux instance by using a password or key.Note In this example, a regular user is used to connect to the ECS instance.
- Install an APM server.
- Modify the configurations of the APM server.
- Start the APM server.
nohup ./apm-server -e > apmserver.log 2>&1 &
Step 2: Configure an APM agent
In this example, the Go programming language is used.
- Install an APM agent.
- Configure the APM agent.
- Use the APM agent to detect an application.
Step 3: View and analyze data that is collected by the APM server in the Kibana console
- Log on to the Kibana console of the Elasticsearch cluster. For information about how to log on to the Kibana console, see Log on to the Kibana console.Note In this example, an Elasticsearch V7.10 cluster is used. Different operations may be required to log on to the Kibana console of clusters of other versions.
- Go to the homepage of the Kibana console and click Dev tools in the upper-right corner.
- On the Console tab of the page that appears, run the following command to enable automatic creation of an APM onboarding index. Note
- After the APM server is started, the system automatically creates APM-related indexes in the Elasticsearch cluster. You can view the indexes in the Kibana console. Alibaba Cloud Elasticsearch has requirements on the naming convention of indexes that are automatically created. The name of the APM onboarding index that is automatically created does not conform to the naming convention. Therefore, you must manually enable automatic creation of an APM onboarding index.
- 7.10.2 in the following command indicates the version number of the APM server that you installed.
PUT _cluster/settings { "persistent": { "action.auto_create_index":"+.*,+apm-7.10.2-onboarding-*,-*" } }
- View services that are monitored by APM.
- In the upper-left corner, click the icon to expand the left-side navigation pane.
- In the left-side navigation pane, click APM below Observability.
- On the Services tab of the page that appears, view all services that are monitored by APM.
- Click the name of a service and view and analyze data related to the service.