This topic provides answers to frequently asked questions (FAQ) about application monitoring in Application Real-time Management Service (ARMS).

Overview

Is the ARMS agent compatible with the agents of other APM services, such as Pinpoint?

The ARMS agent is incompatible with the agents of other application performance management (APM) services. APM is implemented by using bytecode instrumentation based on the ASM framework. If you install two agents, bytecode instrumentation is performed twice on the code. Agents developed by different vendors use different code to implement bytecode instrumentation. Therefore, if you install multiple agents, performance issues may occur due to code conflicts. We recommend that you do not install the agents of other APM services.

[Back to the top]

What can I do if the "OutOfMemoryError" error is reported when I start an application after the ARMS agent is installed?

Append the corresponding heap memory parameters to the start command to increase the memory of the Java virtual machine (JVM). In the following example, the initial size (Xms) of the heap memory is 512 MB and the maximum size (Xmx) of the heap memory is 2 GB.

Note Adjust the sizes based on your actual requirement. In other environments such as Tomcat, add this parameter to JAVA_OPTS in the configuration file.
   -Xms512M
   -Xmx2048M

If the OutOfMemoryError: PermGen space error is reported, append the following parameters to the start command:

   -XX:PermSize=256M 
   -XX:MaxPermSize=512M

If the OutOfMemoryError: metaspace error is reported, append the following parameters to the start command:

   -XX:MetaspaceSize=256M 
   -XX:MaxMetaspaceSize=512M

[Back to the top]

How do I test network connectivity?

Before you install the ARMS agent, make sure that the 8883, 8443, and 8442 ports can be accessed. You can run the Telnet command to check whether the host on which your application is deployed is connected to the ARMS server. For example, to test the connectivity in the China (Shenzhen) region, log on to the host on which the application is deployed and run the following commands:

telnet arms-dc-sz.aliyuncs.com 8883
telnet arms-dc-sz.aliyuncs.com 8443
telnet arms-dc-sz.aliyuncs.com 8442
Note Replace the domain name in the preceding commands with the actual endpoint of ARMS application monitoring. The endpoint of ARMS application monitoring varies in the classic network, the Internet, and virtual private clouds (VPCs).
Table 1. Endpoints of ARMS application monitoring
Region Endpoint for access from the classic network and the Internet Endpoint for access from a VPC
China (Hangzhou) arms-dc-hz.aliyuncs.com arms-dc-hz-internal.aliyuncs.com
China (Beijing) arms-dc-bj.aliyuncs.com arms-dc-bj-internal.aliyuncs.com
China (Shanghai) arms-dc-sh.aliyuncs.com arms-dc-sh-internal.aliyuncs.com
China (Qingdao) arms-dc-qd.aliyuncs.com arms-dc-qd-internal.aliyuncs.com
China (Shenzhen) arms-dc-sz.aliyuncs.com arms-dc-sz-internal.aliyuncs.com
China (Zhangjiakou) arms-dc-zb.aliyuncs.com arms-dc-zb-internal.aliyuncs.com
China (Hong Kong) arms-dc-hk.aliyuncs.com arms-dc-hk-internal.aliyuncs.com
Singapore arms-dc-sg.aliyuncs.com arms-dc-sg-internal.aliyuncs.com
Regions for Alibaba Gov Cloud arms-dc-gov.aliyuncs.com arms-dc-gov-internal.aliyuncs.com
China East 1 Finance arms-dc-hz-finance.aliyuncs.com arms-dc-hz-finance-internal.aliyuncs.com

[Back to the top]

How do I check whether the ARMS agent is installed?

Run the following ps command to check whether the ARMS agent is installed based on the parameters in the start command.

ps -ef | grep 'arms-bootstrap'

The results shown in the following figure indicates that the ARMS agent is installed.

ARMS Agent

The value of the Darms.licenseKey parameter in the command must be the same as the license key displayed on the Add Application page in the ARMS console.

ARMS Access Application

[Back to the top]

Why is no monitoring data displayed in the ARMS console after I install the ARMS agent?

  1. If the log of the ARMS agent contains send agent metrics. no metrics., check whether your application is continuously accessed by external requests and whether the development framework is supported by the ARMS agent. The external requests include HTTP requests, High-Speed Service Framework (HSF) requests, and Dubbo requests. For more information about third-party components and frameworks supported by the ARMS agent, see Overview.
  2. Check whether the specified time range for query is valid. Set the query time range to the last 15 minutes and check whether monitoring data is displayed.
  3. If you start the ARMS agent by running the -jar command, check the setting of the command and make sure that the -javaagent parameter is before -jar. The following command provides an example on how to add parameters to the start command:
    java -javaagent:/{user.workspace}/ArmsAgent/arms-bootstrap-1.7.0-SNAPSHOT.jar -Darms.licenseKey=xxx -Darms.appName=xxx -jar demoApp.jar
  4. If the logs stored in ArmsAgent/log/ contain the "LicenseKey is invalid." error,check whether the region of your application is the same as that of the ARMS agent.
  5. After your application is started, if the log folder does not exist in the ArmsAgent folder, the arms-bootstrap-1.7.0-SNAPSHOT.jar package fails to be loaded. Check whether the permissions on the ArmsAgent folder are obtained.
  6. If the following error is reported when your application is started, check whether the arms-bootstrap-1.7.0-SNAPSHOT.jar package is valid and the required permissions are obtained:

    Error opening zip file or JAR manifest missing: /root/ArmsAgent/arms-bootstrap-1.7.0-SNAPSHOT.jar
    Error occurred during initialization of VM
    agent library failed to init: instrument
  7. If no monitoring data is displayed, compress the logs of the ARMS agent for Java applications in the ArmsAgent/log directory into a compressed file, and contact the DingTalk account arms160804 for support.
  8. Check the Java Development Kit (JDK) version. If the JDK version is 1.8.0_25 or 1.8.0_31, you may fail to install the agent. We recommend that you update the JDK or contact the DingTalk account arms160804.

[Back to the top]

What can I do if no IP address or an invalid IP address is displayed after the ARMS agent is installed?

  1. Run the ifconfig -a command to check the network configuration of the current host. If the host uses multiple network interface controllers (NICs), the IP address obtained by the ARMS agent may be inconsistent with the actual IP address due to network configurations.
  2. You can resolve the issue by using one of the following methods:
    • Set the -DEAGLEEYE.LOCAL.IP=10.XX.XX.XX parameter for the JVM.
      Note Replace 10.XX.XX.XX with the actual IP address.
    • Configure the ARMS agent to obtain the value of the -DNETWORK.INTERFACE=eth0 parameter, where eth0 indicates the NIC name.

[Back to the top]

How do I troubleshoot errors contained in the logs of the ARMS agent that are stored in the ArmsAgent/log directory?

If the "LicenseKey is invalid" error is contained in the logs, perform the following operations to troubleshoot the error:

  1. Make sure that the application is created in ARMS and the License key that you specified when the ARMS agent is installed is valid.
  2. ARMS supports multiple regions. Therefore, check whether the download URL of the ARMS agent is in the same region as your application.

[Back to the top]

How do I deploy multiple instances on a single host?

To deploy multiple instances of an application on a single host, set the -Darms.agentId parameter to specify the JVM process to connect to. This parameter indicates a logical number such as 001 and 002. The following command provides an example on how to deploy multiple instances on a single host:

java -javaagent:/{user.workspace}/ArmsAgent/arms-bootstrap-1.7.0-SNAPSHOT.jar -Darms.licenseKey=<LicenseKey> -Darms.appName=<AppName> -Darms.agentId=001 -jar demoApp.jar

[Back to the top]

What can I do if getcwd errors are reported when I run the script to access a Java application?

The following error message may be returned after you run the script to access the Java application:

shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory Error occurred during initialization of VM java.lang.Error: Properties init: Could not determine current working directory. at java.lang.System.initProperties(Native Method) at java.lang.System.initializeSystemClass(System.java:1119)

This may be because that the current directory is accidentally deleted when the script is running. To resolve this issue, run the cd command and then run the script again.

[Back to the top]

Where can I view logs after I install the ARMS agent with one click?

By default, the logs are stored in the/root/.arms/supervisor/logs/ directory. If no logs are stored in this directory, run the ps -ef |grep arms command to view the directory where the logs are stored.

[Back to the top]

What can I do if the ARMS agent cannot be installed?

  1. Make sure that your ECS instance can access the download URL of the ARMS agent over the Internet in the region in which the ECS instance is located.
    Region Download URL for the Internet
    China (Hangzhou)
    http://arms-apm-hangzhou.oss-cn-hangzhou.aliyuncs.com/install.sh
    China (Shanghai)
    http://arms-apm-shanghai.oss-cn-shanghai.aliyuncs.com/install.sh
    China (Qingdao)
    http://arms-apm-qingdao.oss-cn-qingdao.aliyuncs.com/install.sh
    China (Beijing)
    http://arms-apm-beijing.oss-cn-beijing.aliyuncs.com/install.sh
    China (Shenzhen)
    http://arms-apm-shenzhen.oss-cn-shenzhen.aliyuncs.com/install.sh
    Singapore
    http://arms-apm-ap-southeast.oss-ap-southeast-1.aliyuncs.com/cloud_ap-southeast-1/install.sh
  2. Make sure that your ECS instance can access the ARMS console.
  3. Log on to the ECS console and perform the following operations:
    1. In the left-side navigation pane, choose Maintenance & Monitoring > ECS Cloud Assistant.
    2. On the Cloud Assistant page, click the Commands tab, click the search box and select Command Name, enter InstallJavaAgent, and then press the Enter key.
      Commands on the Cloud Assistant page
      Note If no result is returned for the search, contact the DingTalk account arms160804.
    3. On the Cloud Assistant page, click the Command Execution Result tab and enter the task ID of the InstallJavaAgent command in the search box. In the search results, click View in the Actions column of this command and check whether this command is run. If the command fails to be run, troubleshoot the errors based on the execution results. For example, if this issue is caused because the disk of the ECS instance is full or the ARMS agent is not installed, clear the disk or install the ARMS agent. If this issue cannot be resolved, send the execution results to the DingTalk account arms160804 for support.

[Back to the top]

What can I do if the information about the processes of the ECS instance is inaccurate after the ARMS agent is installed?

If the information about the processes of the ECS instance is not displayed or is inaccurate after the ARMS agent is installed, click the - icon on the left side of the ECS instance and click the + icon to refresh the data. If the issue cannot be resolved, contact the DingTalk account arms160804 for support.

[Back to the top]

What can I do if I cannot enable ARMS application monitoring for a process on an ECS instance?

Check whether the /root/.arms/supervisor/logs/arms-supervisor.log file of the ECS instance contains errors. If the log file contains errors, troubleshoot the error based on the error message. If the error cannot be resolved, contact the DingTalk account arms160804 for support.

[Back to the top]

Why is no application monitoring data displayed after the ARMS agent is installed on a Java application in an ACK cluster?

  1. Log on to the Alibaba Cloud Container Service for Kubernetes console.
  2. In the left-side navigation pane, click Clusters. On the Clusters page, find the ACK cluster in which the Java application is deployed and click Applications in the Actions column.
  3. At the top of the Pods page, select the namespace in which your application resides. Find the application and click Edit in the Actions column.
  4. In the Edit YAML dialog box, check whether the YAML file contains initContainers.
    db_am_ack_apppod_yaml
    • If the YAML file does not contain initContainers, the pod has not been injected to arms-init-container. Perform Step 5.
    • If the YAML file contains initContainers, the pod has been injected to arms-init-container. Perform Step 8.
  5. At the top of the Pods tab, select the arms-pilot namespace. Check whether a pod whose name is prefixed with arms-pilot exists in the pod list.
  6. On the Deployments or StatefulSets tab, click More in the Actions column and select View in YAML. In the Edit YAML dialog box, check whether the YAML file contains the following annotations:
    annotations:
      armsPilotAutoEnable: 'on'
      armsPilotCreateAppName: <your-deployment-name>
    • If the YAML file contains the annotations, perform Step 7.
    • If the YAML file does not contain the annotations, add the preceding annotations to the spec > template > metadata section of the YAML file in the Edit YAML dialog box and replace <your-deployment-name> with your application name. Then, click Update.
  7. On the Pods tab, find the pod where your application resides and click Logs in the Actions column to check whether the pod logs of arms-pilot reports a Security Token Service (STS) error in the "Message":"STS error" format.
  8. On the Pods tab, find the pod where your application resides and click Edit in the Actions column. In the Edit YAML dialog box, check whether the YAML file contains the following javaagent parameter:
    -javaagent:/home/admin/.opt/ArmsAgent/arms-bootstrap-1.7.0-SNAPSHOT.jar
    • If the YAML file contains the parameter, find the pod of the application on the Pods page, and click Terminal in the Actions column to go to the command line page. Run the following command to check whether a log file name is suffixed with .log. Then, contact the DingTalk account arms160804.
      cd /home/admin/.opt/ArmsAgent/logs
    • If the YAML file does not contain the parameter, contact the DingTalk account arms160804 for support.

[Back to the top]

What can I do if the ARMS agent cannot be installed for an application in an ACK cluster?

This may be because the version of arms-pilot is earlier than V1.30.

To resolve this issue, perform the following steps to grant ACK the permissions to access ARMS.

  1. Log on to the ACK console.
  2. In the left-side navigation pane, click Clusters. On the Clusters page, find the cluster in which your application is deployed and click Details in the Actions column.
    Manage clusters
  3. On the details page of the cluster, click the Cluster Resources tab. On the Cluster Resources tab, click the hyperlink next to Worker RAM Role.
    Worker RAM Role_Hyperlink
  4. On the details page of the worker RAM role in the RAM console, click the policy name on the Permissions tab.
  5. On the Policy Document tab, click Modify Policy Document, add the following code in the code editor below Policy Document, and then click OK.
    {
       "Action": "arms:*",
       "Resource": "*",
       "Effect": "Allow"
    }
    Modify Policy Document

[Back to the top]

What can I do if an application cannot be started?

Run the following command to view the arms-pilot-system logs and troubleshoot the error based on the logs:

kubectl logs -f {arms-pilot-arms-pilot-XXX} -n arms-pilot-system

[Back to the top]

How do I view the logs of the ARMS agent?

On the worker of the ACK cluster, view the /home/admin/.opt/ArmsAgent/logs/xxxx.log file.

[Back to the top]

How do I modify the name of a common Java application on which the ARMS agent is manually installed?

Common Java applications indicate the Java applications that are not deployed on ECS instances. If you manually install the ARMS agent on the application, the directory of the agent is specified by you during the installation.

You can check the version of the ARMS agent by viewing the Version file in the directory of the agent. For example, 2.5.8_cf020486_20190816150025 indicates that the version of the ARMS agent is 2.5.8 and was released on August 16, 2019.

  1. Run one of the following commands to download and decompress Supervisor from the Internet or a VPC.
    Note Replace the download URL in the commands with the URL for the region in which the Java application resides.
    • Download Supervisor from the Internet
      wget http://arms-apm-hangzhou.oss-cn-hangzhou.aliyuncs.com/ArmsSupervisor.zip -O ArmsSupervisor.zip
      unzip ArmsSupervisor.zip
    • Download Supervisor from a VPC when the download URL for the Internet is unavailable
      wget http://arms-apm-hangzhou.oss-cn-hangzhou-internal.aliyuncs.com/ArmsSupervisor.zip -O ArmsSupervisor.zip
      unzip ArmsSupervisor.zip
  2. Run the following command to modify the name of the Java application:
    cd ArmsSupervisor
    ./attach.sh </path/to/ArmsAgent/arms-bootstrap-1.7.0-SNAPSHOT.jar> <PID> <NewAppName> <LicenseKey>
    • </path/to/ArmsAgent/arms-bootstrap-1.7.0-SNAPSHOT.jar>: the path of the arms-bootstrap-1.7.0-SNAPSHOT.jar file.
    • <PID>: the process identifier (PID) of the application. You can run the jps/ps command to obtain the PID.
    • <NewAppName>: the new application name.
    • <LicenseKey>: the license key of the application monitored by ARMS. You can obtain the license key in the ARMS console.

If the standard output shown in the following figure is displayed, the application name is modified.

Name modified

[Back to the top]

How do I modify the name of a common Java application on which the ARMS agent is installed with one click?

If you install the ARMS agent with one click, the agent is installed in the ~/.arms/supervisor/agent directory. The account you use must be the same as the application account.

Perform the following steps to modify your application name:

  1. Log on to the host where the application resides and run the following command:
    cd ~/.arms/supervisor
    ./cli.sh <LicenseKey> <NewAppName>
    • <LicenseKey>: the license key of the application monitored by ARMS. You can obtain the license key in the ARMS console.
    • <NewAppName>: the new application name.
  2. Select the proper process from all Java processes.
    Note If only one process is available, this process is selected by default.
  3. Open the ~/.arms/attach.info file, change the application name to the new name, and then save the file.
    Warning Do not add content such as spaces when you modify the file. Otherwise, the modification may fail because it does not match the new application name modified in the preceding steps.

    Wait a moment after your application name is modified. Monitoring data of the application is reported by using the new name rather than the previous name.

How do I modify the name of a Java application deployed on an ECS instance?

If your Java application is deployed on an ECS instance, the ARMS agent directory is /.arms/agent.

Perform the following steps to modify your application name:

  1. Log on to the ECS instance where your application is deployed and run the following command by using the root account:
    su <USER> -c "./attach.sh /.arms/agent/arms-bootstrap-1.7.0-SNAPSHOT.jar <PID> <NewAppName> <LicenseKey>"
    • <PID>: the PID of your application. You can run the jps/ps command to obtain the PID.
    • <NewAppName>: the new application name.
    • <LicenseKey>: the license key of the application monitored by ARMS. You can obtain the license key in the ARMS console.
  2. Open the ~/.arms/attach.info file, change the application name to the new name, and then save the file.
    Warning Do not add content such as spaces when you modify the file. Otherwise, the modification may fail because it does not match the new application name modified in the preceding steps.

If the standard output shown in the following figure is displayed, the application name is modified.

Name modified

How do I modify the name of a Java application deployed in an ACK cluster?

You can check the version of the ARMS agent by viewing the Version file in the directory of the agent. For example, 2.5.8_cf020486_20190816150025 indicates that the version of the ARMS agent is 2.5.8 and was released on August 16, 2019.

Change the value of the armsPilotCreateAppName parameter in the Deployment and restart the pod.

Wait a moment after your application name is modified. Monitoring data of the application is reported by using the new name rather than the previous name.

[Back to the top]

How do I modify the name of a Java application deployed in EDAS?

The names of Java applications deployed in Enterprise Distributed Application Service (EDAS) cannot be modified.

[Back to the top]

How do I uninstall the ARMS agent that is manually installed?

For more information about how to manually install the ARMS agent on Java applications, see Manually install the ARMS agent for a Java application. You can perform the following steps to uninstall the ARMS agent that is manually installed:

  1. If you no longer want to use ARMS to monitor your Java applications, delete all parameters related to the AppName and LicenseKey parameters. These parameters are described in Step 8 in the Manually install the ARMS agent for a Java application topic.
  2. Restart the Java application.

[Back to the top]

How do I uninstall the ARMS agent that is installed with one click?

For more information about how to install the ARMS agent with one click, see Install the ARMS agent for a Java application by using scripts. You can perform the following steps to uninstall the ARMS agent that is installed with one click:

  1. If you no longer want to use ARMS to monitor your Java applications, run the jps -l command to view all processes and find the PID of com.alibaba.mw.arms.apm.supervisor.daemon.Daemon in the returned results.
    In this example, the PID of com.alibaba.mw.arms.apm.supervisor.daemon.Daemon is 62857. Kill process
  2. Run the kill -9 <PID> command.
    Example: kill -9 62857.
  3. Run the rm -rf /.arms /root/.arms command.
  4. Restart your application.

[Back to the top]

How do I uninstall the ARMS agent installed on a Java application deployed on an ECS instance?

  1. If you no longer want to use ARMS to monitor your Java applications, run the jps -l command to view all processes and find the PID of com.alibaba.mw.arms.apm.supervisor.daemon.Daemon in the returned results.
    In this example, the PID of com.alibaba.mw.arms.apm.supervisor.daemon.Daemon is 62857. Kill process
  2. Run the kill -9 <PID> command.
    Example: kill -9 62857.
  3. Run the rm -rf /.arms /root/.arms command.
  4. Restart your application.

[Back to the top]

How do I uninstall the ARMS agent installed on a Java application deployed in an ACK cluster?

If you no longer want to use ARMS to monitor your Java application deployed in an ACK cluster, perform the following steps to uninstall the ARMS agent:

  1. Log on to the Alibaba Cloud Container Service for Kubernetes console.
  2. In the left-side navigation pane, click Clusters. On the Clusters page, find the cluster that contains the Java application from which you want to uninstall the ARMS agent and click Applications in the Actions column.
  3. In the left-side navigation pane, choose Applications > Helm.
  4. On the Helm tab, find the release named arms-pilot of the ARMS agent, and click Delete in the Actions column.
  5. In the Delete message, click OK.
  6. Restart your business pod.

[Back to the top]

How do I uninstall the ARMS agent installed on a Java application deployed in an open source Kubernetes environment?

  1. If you no longer want to use ARMS to monitor your Java applications deployed in an open-source Kubernetes environment, run the following command to uninstall arms-pilot:
    helm del --purge arms-pilot
  2. Restart your business pod.

[Back to the top]

How do I uninstall the ARMS agent installed on a Java application deployed in a Docker cluster?

  1. If you no longer want to use ARMS to monitor your Java application deployed in a Docker cluster, delete the content of the Dockerfile edited in Step 1 in the Install the ARMS agent for a Java application deployed in a Docker cluster topic.
  2. Run the docker build command to construct the image.
  3. Run the docker run command to start the image.

[Back to the top]

How do I uninstall the ARMS agent installed on a PHP application?

If you no longer want to use ARMS to monitor your PHP application that is not deployed in an ACK cluster, perform the following steps to uninstall the ARMS agent:

  1. Delete the following four lines from the php.ini file:
    [arms] 
    extension=<php_extension_dir>/arms.so
    arms.trace_exception=true
    arms.config_full_name=/<php-agent-dir>/arms-agent.conf                  
  2. Restart your PHP application.

[Back to the top]

How do I uninstall the ARMS agent installed on a PHP application deployed in an ACK cluster?

If you no longer want to use ARMS to monitor your PHP application that is deployed in an ACK cluster, perform the following steps to uninstall the ARMS agent:

  1. Log on to the Alibaba Cloud Container Service for Kubernetes console.
  2. In the left-side navigation pane, click Clusters. On the Clusters page, find the cluster that contains the Java application from which you want to uninstall the ARMS agent and click Applications in the Actions column.
  3. In the left-side navigation pane, choose Applications > Helm.
  4. On the Helm tab, find the release named arms-pilot of the ARMS agent, and click Delete in the Actions column.
  5. In the Delete message, click OK.
  6. Restart your business pod.

[Back to the top]

What can I do if the data of my application that uses the OpenFeign component is incomplete in ARMS?

After your application that uses the OpenFeign component is connected to ARMS application monitoring, the application data may be incomplete and the data of downstream applications may fail to be viewed. This may be because that the OpenFeign component enables Hystrix that uses the RxJava asynchronous framework by default. However, ARMS does not support asynchronous frameworks.

Note This description applies to scenarios where the version of the ARMS agent for Java applications is earlier than 2.6.0. The ARMS agent for Java applications of the 2.6.0 version or later already supports asynchronous frameworks.

You can disable Hystrix and enable the OkHttp request class to resolve this issue.

  1. Add the following dependency to the pom.xml file:
    <!-- OKHttp supports Feign -->
    <dependency>
        <groupId>io.github.openfeign</groupId>
        <artifactId>feign-okhttp</artifactId>
    </dependency>                
  2. Add the following content to the SpringCloud configuration file:
    feign.okhttp.enabled: true
    feign.hystrix.enabled: false                 
  3. Configure the OkHttp request class.
    @Configuration
    @ConditionalOnClass(Feign.class)
    @AutoConfigureBefore(FeignAutoConfiguration.class)
    public class FeignClientOkHttpConfiguration {
    
        @Bean
        public OkHttpClient okHttpClient() {
            return new OkHttpClient.Builder()
                    // The connection times out.
                    .connectTimeout(20, TimeUnit.SECONDS)
                    // The response times out.
                    .readTimeout(20, TimeUnit.SECONDS)
                    // The write request times out.
                    .writeTimeout(20, TimeUnit.SECONDS)
                    // Indicates whether to enable automatic reconnection.
                    .retryOnConnectionFailure(true)
                    // The connection pool.
                    .connectionPool(new ConnectionPool())
                    .build();
        }                

[Back to the top]