Application monitoring FAQ

Overview

• FAQ about manually installing the security center Agent for Java applications
  • Is the ARMS Agent compatible with agents of other APM products, such as Pinpoint?
  • What can I do if a OutOfMemoryError error occurs when an application starts after the EDAs Agent is installed?
  • How do I test the network connectivity?
  • How can I check whether ARMS Agent is successfully installed?
  • Why is there no monitoring data in the console after ARMS Agent is installed?
  • What do I do if the ARMS Agent is successfully installed but the IP address is incorrectly displayed?
  • What are the common troubleshooting methods for ARMS Agent log (path: ArmsAgent/log) errors?
  • How do I support single-instance multi-instances?
• FAQ about installing the arms Agent for Java applications
  • How can I resolve getcwd errors when running the import script?
  • Where can I view the logs after I install the security center Agent with one click?
• FAQ about installing the security center Agent for Java applications on ECS instances
  • How do I handle an Agent installation failure?
  • What do I do if the ECS instance process information is incorrect after the Agent is installed?
  • What can I do if the process in an ECS instance fails to start application monitoring?
• FAQ about installing the security center Agent for Alibaba Cloud Container Service for Kubernetes Java applications
  • Why is there no data on the Java application in a Kubernetes cluster application monitoring container service is installed with the Agent?
• FAQ about installing the EDAs Agent for Java applications in open-source Kubernetes environments
  • What do I do if an application fails to start?
  • How do I view the Agent log?
• FAQs about changing the name of a Java application without reinstalling the Agent
  • How do I change the application name for a normal Java application that installs Agent in a common way?
  • How do I change the name of a Java application deployed in a Kubernetes cluster of container service?
  • How can I change the name of a Java application deployed in EDAS?
• FAQ about uninstalling the security center Agent
  • How do I uninstall the Agent that is installed in a common way?
  • How do I uninstall the Agent installed in quick mode?
  • How do I uninstall the arms Agent from the Java application on an ECS instance?
  • How do I uninstall the Agent of the Java application in a container service Kubernetes cluster?
  • How do I uninstall the Agent from the Java application in an open-source Kubernetes environment?
  • How do I uninstall the Agent of the Java application in Docker?
  • How do I uninstall the Agent?
  • How do I uninstall the Agent of the PHP application in a container service Kubernetes cluster?
• Other problems
  • What should I do if the data of my application with the OpenFeign component is incomplete in ARMS?

Is the ARMS Agent compatible with agents of other APM products, such as Pinpoint?

The ARMS Agent is incompatible with other APM agents. APM mostly implements bytecode instrumentation based on ASM framework. Installing two agents at the same time is equivalent to instrumentation your code twice, while the instrumentation code implementation for different merchants is different. Code conflicts may cause serious performance problems. We strongly recommend that you do not install multiple APM agents at the same time.

[Back to the top]

What can I do if a OutOfMemoryError error occurs when an application starts after the EDAs Agent is installed?

To properly increase the JVM value, append the heap memory configuration items to the Java startup command. Configuration items following example indicates that the initial heap memory size (Xms) is 512 MB and the maximum heap memory size (Xmx) is 2 GB.

   -Xms512M
   -Xmx2048M

If OutOfMemoryError: PermGen space error, please add the following configuration.

   -XX:PermSize=256M 
   -XX:MaxPermSize=512M

[Back to the top]

How do I test the network connectivity?

Before the ARMS Agent runs, make sure that ports 8883, 8443, and 8442 are properly connected. Can be used Telnet command to check whether the target host can connect to the ARMS server. For example, to test the connectivity to the China (Shenzhen) region, log on to the ECS instance where the application is deployed, and run the following command:

telnet arms-dc-sz.aliyuncs.com 8883
telnet arms-dc-sz.aliyuncs.com 8443
telnet arms-dc-sz.aliyuncs.com 8442

[Back to the top]

How can I check whether ARMS Agent is successfully installed?

Use PS command to check whether ARMS Agent is successfully installed in the command line startup parameter.

ps -ef | grep 'arms-bootstrap'

Upon successful installation, the result is shown in the following figure.

At the command line, Darms.licenseKey the value of must be the same as that of ARMS application Access the License Key displayed on the page is the same.

[Back to the top]

Why is there no monitoring data in the console after ARMS Agent is installed?

1. If the send agent metrics. no metrics., check whether your application has continuous external access requests, including HTTP requests, HSF requests, and Dubbo requests, and whether the development framework is supported by ARMS Agent. For more information about third-party components and frameworks supported by ARMS application monitoring, see Overview.
2. Check whether the selected time range for query is correct. Select the last 15 minutes as the query time range and check whether monitoring data is available.
3. If it is through -jar the script is started through the command line. Check the command line settings to make sure -javaagent the parameter is in the -jar before. The following is a correct example.

   java -javaagent:/{user.workspace}/ArmsAgent/arms-bootstrap-1.7.0-SNAPSHOT.jar -Darms.licenseKey=xxx -Darms.appName=xxx -jar demoApp.jar

4. If ArmsAgent/log/ appears in the log under "LicenseKey is invalid", Exception, check if the application and the Agent are in the same region.
5. If the application has been started up ArmsAgent if the log sub-directory does not exist in the directory, the arms-bootstrap-1.7.0-SNAPSHOT.jar has not been successfully loaded. Check whether the permissions of the ArmsAgent installation directory are correct.

6. If the following error is reported when your application is started, check arms-bootstrap-1.7.0-SNAPSHOT.jar software package and the corresponding permissions.

   Error opening zip file or JAR manifest missing: /root/ArmsAgent/arms-bootstrap-1.7.0-SNAPSHOT.jar
   Error occurred during initialization of VM
   agent library failed to init: instrument

7. If there is still no monitoring data, copy the Java Agent log (path: ArmsAgent/log) as a compressed file and contact the DingTalk service account. arms160804 solve the problem.

8. Check the JDK Version. If the JDK Version is 1.8.0 _25 or 1.8.0 _31, you may fail to install the Agent. We recommend that you upgrade the JDK or contact DingTalk arms160804.

[Back to the top]

What do I do if the ARMS Agent is successfully installed but the IP address is incorrectly displayed?

1. First through ifconfig -a check the network configuration of the current machine. If the machine uses network interface controller, the IP addresses collected by the arms Agent may not be as expected, which is related to the network configuration.
2. Choose one of the following ways to solve the problem.
   • Configure the JVM -DEAGLEEYE.LOCAL.IP=10.XX.XX.XX parameter.
     Note Will 10.XX.XX.XX replace it with the actual IP address.
   • Configure the Agent to obtain the JVM -DNETWORK.INTERFACE=eth0 parameter, where eth0 is network interface controller name.

[Back to the top]

What are the common troubleshooting methods for ARMS Agent log (path: ArmsAgent/log) errors?

LicenseKey is invalid error:

1. Ensure that you have successfully created the application in ARMS and that you entered the correct LicenseKey when installing the Agent.
2. ARMS is a multi-Region environment. Therefore, you also need to check whether the download link of the ARMS Agent is in the same Region as your application.

[Back to the top]

How do I support single-instance multi-instances?

To deploy multiple instances of the same application on the same server, you can use -Darms.agentId(Logical number, such as 001 or 002) parameter to distinguish the Accessed JVM process, for example:

java -javaagent:/{user.workspace}/ArmsAgent/arms-bootstrap-1.7.0-SNAPSHOT.jar -Darms.licenseKey=<LicenseKey> -Darms.appName=<AppName> -Darms.agentId=001 -jar demoApp.jar

[Back to the top]

How can I resolve getcwd errors when running the import script?

If the following error message is returned when you run the script for one-click Java application access:

shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory Error occurred during initialization of VM java.lang.Error: Properties init: Could not determine current working directory. at java.lang.System.initProperties(Native Method) at java.lang.System.initializeSystemClass(System.java:1119)

This may be because the current directory was deleted by mistake when the script was executed. To solve this problem, run the CD and then run the script again.

[Back to the top]

Where can I view the logs after I install the security center Agent with one click?

The default directory for logs is /root/.arms/supervisor/logs/ if there are no logs in this directory, run ps -ef |grep arms view the directory where the logs are located.

[Back to the top]

How do I handle an Agent installation failure?

1. Make sure that your ECS instance can access the Internet and the Agent download link in the region where your instance is located can be accessed.

   # China (Hangzhou)
   http://arms-apm-hangzhou.oss-cn-hangzhou.aliyuncs.com/install.sh
   # China (Shanghai)
   http://arms-apm-shanghai.oss-cn-shanghai.aliyuncs.com/install.sh
   # China (Qingdao)
   http://arms-apm-qingdao.oss-cn-qingdao.aliyuncs.com/install.sh
   # China (Beijing)
   http://arms-apm-beijing.oss-cn-beijing.aliyuncs.com/install.sh
   # China (Shenzhen)
   http://arms-apm-shenzhen.oss-cn-shenzhen.aliyuncs.com/install.sh
   # Singapore
   http://arms-apm-ap-southeast.oss-ap-southeast-1.aliyuncs.com/cloud_ap-southeast-1/install.sh

2. Make sure that your ECS instance can access the console.

   # China
   https://arms.console.aliyun.com/
   #Singapore
   https://arms-ap-southeast-1.console.aliyun.com

3. Login ECS console, and complete the following checks.
   1. In the left-side navigation pane, choose maintenance and monitoring > send remote commands (Cloud Assistant).
   2. In cloud Assistant select command name, input InstallJavaAgent command and press Enter.

      

      Note If the search results do not exist, contact the ARMS DingTalk service account arms160804.
   3. In cloud Assistant page's execution Records click the tab and enter InstallJavaAgent the ID of the corresponding command. Click the record ID operation column's view Results, you can view InstallJavaAgent indicates whether the command was executed successfully. If the execution fails, troubleshoot based on the detailed execution results. If the ECS disk is full or Agent is not installed, you can clear the disk or install Agent to solve the problem. If the problem persists, submit the detailed execution results to ARMS DingTalk account. arms160804.

      

[Back to the top]

What do I do if the ECS instance process information is incorrect after the Agent is installed?

If no ECS instance process information is displayed after you successfully install the Agent on the ECS or the ECS instance process information is inaccurate, click on the left of the ECS instance -and click +refresh data. If the problem cannot be solved, contact the ARMS DingTalk service account arms160804.

[Back to the top]

What can I do if the process in an ECS instance fails to start application monitoring?

Check the IP addresses of the /root/.arms/supervisor/logs/arms-supervisor.log whether the log contains Error log level information, and according to this information, troubleshoot. If the problem cannot be solved, contact the ARMS DingTalk service account arms160804.

[Back to the top]

Why is there no data on the Java application in a Kubernetes cluster application monitoring container service is installed with the Agent?

1. In the left-side navigation pane, choose application > container Group, in container Group (Pod) page, from cluster list and namespace in the left-side navigation pane, select the cluster and namespace of your application.
2. In The Pod list, locate the Pod of the target application. Edit, in edit YAML dialog box to check whether initContainers exists in the YAML file.

   

   • If it does not exist, the arms-init-container has not been injected. Run the following command: step3.
   • If yes, arms-init-container has been injected. Run the following command: step6.
3. In container Group (Pod) page, from cluster list and namespace in the list, select the cluster and arms-pilot namespace and check whether any Pod list with the prefix arms-pilot pod.
   • If yes, perform the step4.
   • Otherwise, install arms-pilot in the application market. For more information, see install ARMS application monitoring.
4. In the left-side navigation pane, choose application > stateless, in Deployment page from cluster list and namespace select the cluster and namespace where your application is located from the drop-down list. Operation column's more > view Yaml files, in edit YAML dialog box to check whether the following Annotations exist in the YAML file:

   annotations:
     armsPilotAutoEnable: 'on'
     armsPilotCreateAppName: <your-deployment-name>

   • If yes, perform the step5.
   • If it does not exist, it is in the edit YAML in the dialog box spec > template > metadata add the Annotations above in the hierarchy, and add <your-deployment-name> with the name of your application and then click update.
5. In the left-side navigation pane, choose application > container Group, in container Group (Pod) page, from cluster list and namespace in the list, select the cluster and arms-pilot namespace. Click log check whether the STS error message is displayed in the Pod log of arms-pilot. "Message":"STS error".
   • If an STS error is reported, you must authorize the cluster where the application is located and restart the Pod where the application is located. For more information, see authorize Alibaba Cloud Container Service for Kubernetes.
   • If no STS error is reported, contact ARMS DingTalk service account: arms160804.
6. In edit YAML dialog box to check whether the following exists in the YAML file: javaagent parameter.

   -javaagent:/home/admin/.opt/ArmsAgent/arms-bootstrap-1.7.0-SNAPSHOT.jar

   • If it exists, container Group (Pod) find the Pod where the application is located. Terminal enter command line page and run the following command to check whether there is a log file suffixed with. log, and then contact ARMS DingTalk service account: arms160804.

     cd /home/admin/.opt/ArmsAgent/logs

   • If it does not exist, contact ARMS DingTalk service account: arms160804.

[Back to the top]

What do I do if an application fails to start?

Run the following command to view the arms-pilot-system of the log for troubleshooting.

kubectl logs -f {arms-pilot-arms-pilot-XXX} -n arms-pilot-system

[Back to the top]

How do I view the Agent log?

View logs on the Worker of the Kubernetes cluster. /home/admin/.opt/ArmsAgent/logs/xxxx.log.

[Back to the top]

How do I change the application name for a normal Java application that installs Agent in a common way?

Common Java applications are Java applications other than those deployed on ECS instances of Alibaba Cloud. If you installed the Agent in a common way, the Agent directory is the location you customized.

Check the Agent Version in the Version file in the Agent directory. For example, 2.5.8_cf020486_20190816150025 the Agent version is 2.5.8, and the release date of the Agent is August 16, 2019.

• If your Agent version is earlier than 2.5.8.1, uninstall the Agent and then reinstall it. Use a new application name when you reinstall the Agent.
  • To uninstall the Agent that is manually installed, see how do I uninstall the Agent installed in a common way.
  • To uninstall the EDAs Agent that is automatically installed by using the script, see how to uninstall the Agent installed in quick mode.
  • For more information about how to uninstall the ECS Agent, see how to uninstall the Agent from the Java application in an ECS instance.
• If your Agent version is 2.5.8.1 or later, you can change the name of the Java application without reinstalling the Agent. For more information, see the following procedure:
  Note Agents downloaded after August 20, 2019 support this function by default.

1. Download and extract the Supervisor using one of the following methods.
   Note Replace the region in the download URL with the region where your application is located.
   • Public network:

     wget http://arms-apm-hangzhou.oss-cn-hangzhou.aliyuncs.com/ArmsSupervisor.zip -O ArmsSupervisor.zip
     unzip ArmsSupervisor.zip

   • VPC address (used when the public endpoint cannot be downloaded)

     wget http://arms-apm-hangzhou.oss-cn-hangzhou-internal.aliyuncs.com/ArmsSupervisor.zip -O ArmsSupervisor.zip
     unzip ArmsSupervisor.zip

2. Run the following command to change the application name:

   cd ArmsSupervisor
   ./attach.sh </path/to/ArmsAgent/arms-bootstrap-1.7.0-SNAPSHOT.jar> <PID> <NewAppName> <LicenseKey>

   • </path/to/ArmsAgent/arms-bootstrap-1.7.0-SNAPSHOT.jar>: indicates the path of a file with the same name under the Agent.
   • <PID>: Target process ID, which can be used jps/ps command.
   • <NewAppName>: new application name.
   • <LicenseKey>:ARMS LicenseKey of the application in the application monitoring, which can be obtained from the console.

If the standard output log of the application displays the following information, the application name is successfully changed:

[Back to the top]

How do I change the application name for a normal Java application that installs Agent in a quick way?

If you installed the Agent quickly, the Agent directory is ~/.arms/supervisor/agent. Note that the account you use must be the same as the application account.

Change your application name as follows:

1. Log on to the machine where the application is located and run the following command:

   cd ~/.arms/supervisor
   ./cli.sh <LicenseKey> <NewAppName>

   • <LicenseKey>:ARMS LicenseKey of the application in the application monitoring, which can be obtained from the console.
   • <NewAppName>: new application name.
2. Select the correct process from all Java processes listed in the program.
   Note If only one process is available, this process is selected by default.
3. Open ~/.arms/attach.info file, modify the old application name to the new application name and save the file.
   Warning When modifying a file, do not add extra spaces. Otherwise, the modification will fail due to incompatibility with the new application name modified in the preceding steps.

   Wait a moment after your application name is changed. No monitoring data will be reported using the old application name, and monitoring data will be reported using the new application name.

How do I change the name of a Java application deployed on an ECS instance?

If your Java application is deployed on an ECS instance, the Agent directory is /.arms/agent.

Change your application name as follows:

1. Log on to the ECS instance where your application is located and run the following command with the root account:

   su <USER> -c "./attach.sh /.arms/agent/arms-bootstrap-1.7.0-SNAPSHOT.jar <PID> <NewAppName> <LicenseKey>"

   • <PID>: Target process ID, which can be used jps/ps command.
   • <NewAppName>: new application name.
   • <LicenseKey>:ARMS LicenseKey of the application in the application monitoring, which can be obtained from the console.
2. Open ~/.arms/attach.info file, modify the old application name to the new application name and save the file.
   Warning When modifying a file, do not add extra spaces. Otherwise, the modification will fail due to incompatibility with the new application name modified in the preceding steps.

If the standard output log of the application displays the following information, the application name is successfully changed:

How do I change the name of a Java application deployed in a Kubernetes cluster of container service?

Check the Agent Version in the Version file in the Agent directory. For example, 2.5.8_cf020486_20190816150025 the Agent version is 2.5.8, and the release date of the Agent is August 16, 2019.

• If your Agent version is earlier than 2.5.8.1, uninstall the Agent and then reinstall it. Use a new application name when you reinstall the Agent.
  • To uninstall the Agent that is manually installed, see how do I uninstall the Agent installed in a common way.
  • To uninstall the EDAs Agent that is automatically installed by using the script, see how to uninstall the Agent installed in quick mode.
  • For more information about how to uninstall the ECS Agent, see how to uninstall the Agent from the Java application in an ECS instance.
• If your Agent version is 2.5.8.1 or later, you can change the name of the Java application without reinstalling the Agent. For more information, see the following procedure:
  Note Agents downloaded after August 20, 2019 support this function by default.

Modify the values of the armsPilotCreateAppName parameter and restart the Pod.

Wait a moment after your application name is changed. No monitoring data will be reported using the old application name, and monitoring data will be reported using the new application name.

[Back to the top]

How can I change the name of a Java application deployed in EDAS?

Currently, you cannot change the name of a Java application deployed in Enterprise Distributed Application Service (EDAS).

[Back to the top]

How do I uninstall the Agent that is installed in a common way?

In the common way, you need to manually install the security center Agent for your Java application. For more information, see Manually install the ARMS agent for Java applications. Perform the following steps to uninstall the Agent that is installed in the common mode:

1. When you no longer need ARMS to monitor your Java applications, delete the preceding installation document. step 8 the added AppName, LicenseKey all relevant parameters.
2. Restart the Java application.

[Back to the top]

How do I uninstall the Agent installed in quick mode?

In quick mode, you need to install the security center Agent for your Java application by using a script with one click. For more information, see Install the ARMS agent for Java applications by using scripts in one click. Perform the following steps to uninstall the quick mode Agent:

1. When you no longer need ARMS to monitor your Java application, run the jps -l command to view all processes, and in the execution results, find com.alibaba.mw.arms.apm.supervisor.daemon.Daemon the process ID.
   In this example, the com.alibaba.mw.arms.apm.supervisor.daemon.Daemon the process number is 62857.
2. Run commands kill -9 <process id>.
   For example: kill -9 62857.
3. Execution rm -rf /.arms /root/.arms.
4. Restart your application.

[Back to the top]

How do I uninstall the arms Agent from the Java application on an ECS instance?

1. When you no longer need ARMS to monitor your Java application, run the jps -l command to view all processes, and in the execution results, find com.alibaba.mw.arms.apm.supervisor.daemon.Daemon the process ID.
   In this example, the com.alibaba.mw.arms.apm.supervisor.daemon.Daemon the process number is 62857.
2. Run commands kill -9 <process id>.
   For example: kill -9 62857.
3. Execution rm -rf /.arms /root/.arms.
4. Restart your application.

[Back to the top]

How do I uninstall the Agent of the Java application in a container service Kubernetes cluster?

When you no longer need ARMS to monitor the Java application in a Alibaba Cloud Container Service for Kubernetes cluster, you can uninstall the ARMS Agent as follows:

1. In the left-side navigation pane, choose Applications > Releases.
2. In publish page's Helm on the page, from cluster select the cluster from which you want to uninstall ARMS Agent.
3. Select the release name corresponding to the ARMS Agent arms-pilot, click operation column's delete.
4. In delete an application dialog box, click confirm.
5. Restart your business pod.

[Back to the top]

How do I uninstall the Agent from the Java application in an open-source Kubernetes environment?

1. When you no longer need to monitor the Java application in the open-source Kubernetes environment, you can run the following command to uninstall arms-pilot:

   helm del --purge arms-pilot

2. Restart your business pod.

[Back to the top]

How do I uninstall the Agent of the Java application in Docker?

1. When you no longer need to monitor the Java application in the Docker cluster, you can delete the step 1 edited Dockerfile the content.
2. Run docker build command to build the image.
3. Run docker run command to start the image.

[Back to the top]

How do I uninstall the Agent?

When you no longer need ARMS to monitor PHP applications in a Kubernetes cluster, you can uninstall the ARMS Agent as follows:

1. Delete php.ini the following four lines in the file:

   [arms] 
   extension=<php_extension_dir>/arms.so
   arms.trace_exception=true
   arms.config_full_name=/<php-agent-dir>/arms-agent.conf                  

2. Restart the PHP application.

[Back to the top]

How do I uninstall the Agent of the PHP application in a container service Kubernetes cluster?

If you no longer need ARMS to monitor the PHP application in Alibaba Cloud Container Service for Kubernetes cluster, you can uninstall the ARMS Agent as follows:

1. In the left-side navigation pane, choose Applications > Releases.
2. In publish page's Helm on the page, from cluster select the cluster from which you want to uninstall ARMS Agent.
3. Select the release name corresponding to the ARMS Agent arms-pilot, click operation column's delete.
4. In delete an application dialog box, click confirm.
5. Restart your business pod.

[Back to the top]

What should I do if the data of my application with the OpenFeign component is incomplete in ARMS?

After your OpenFeign application is connected to ARMS, if the data is incomplete or the data of downstream applications cannot be seen, a possible cause is that the OpenFeign component enables Hystrix using the RxJava asynchronous framework by default. ARMS does not support asynchronous frameworks.

You can disable Hystrix and enable the OkHttp request class to solve the following problems:

1. In pom.xml add the following dependencies to the file.

   <! -- OKHttp supports Feign -->
   <dependency>
       <groupId>io.github.openfeign
       <artifactId>feign-okhttp</artifactId>
   </dependency>                

2. Add the following content to the spring cloud configuration file:

   feign.okhttp.enabled: true
   feign.hystrix.enabled: false                 

3. Configure the OkHttp request class.

   @Configuration
   @ConditionalOnClass(Feign.class)
   @AutoConfigureBefore(FeignAutoConfiguration.class)
   public class FeignClientOkHttpConfiguration {
   
       @Bean
       public OkHttpClient okHttpClient() {
           return new OkHttpClient.Builder()
                   // The connection times out.
                   .connectTimeout(20, TimeUnit.SECONDS)
                   // The response times out.
                   .readTimeout(20, TimeUnit.SECONDS)
                   // The write request times out.
                   .writeTimeout(20, TimeUnit.SECONDS)
                   // Indicates whether to enable automatic reconnection.
                   .retryOnConnectionFailure(true)
                   // The connection tool.
                   .connectionPool(new ConnectionPool())
                   .build();
       }                

[Back to the top]