Serverless App Engine:Custom configuration for Java applications

Access the feature

On the Application Monitoring page, click Application Configuration, and then click Custom Configuration to configure the following features.

Agent switch settings

In the Probe switch settings section, you can enable or disable application monitoring and select the plug-ins that you want to use.

Thread profiling settings

In the Thread Analysis Settings section, you can turn on or turn off Thread analysis master control switch and Periodically save local method stack information, and configure a threshold for time-consuming calls.

Application log association settings

In the Application log Association configuration section, you can configure a log source that you want to associate with the application, and specify whether to automatically fill in TraceId and SpanId in logs. For more information, see Log analysis.

• Automatically Fill TraceID and Automatically Fill SpanId: automatically injects TraceId and SpanId into logs without the need to modify the log configuration file.

  Configuration effect:

  As shown in the following figure, log4j, log4j2, and logback automatically print TraceId and SpanId in logs without modifying the log configuration file.

  

• Log MDC add traceId: specifies whether to obtain TraceId through org.slf4j.MDC.get(“EagleEye-TraceID”), which is not supported by the ARMS agent V4.x by default.

URL convergence settings

In the URL convergence Settings section, you can enable or disable convergence, and specify a convergence threshold and convergence rules. URL convergence displays similar URLs as a single object. For example, URLs prefixed with /service/demo/ are displayed as an object. The convergence threshold is the minimum number of URLs that can trigger URL convergence. For example, if the threshold is set to 100, URLs are converged only when 100 URLs meet the regular expression of the rule. For more information, see ARMS convergence policies.

Arthas monitoring

In the Arthas monitoring section, you can enable or disable the Arthas diagnostics feature. You can also specify the IP addresses on which you want to perform Arthas diagnostics. For more information, see Arthas diagnostics.

Continuous performance profiling settings

In the Continuous profiling settings section, you can turn the main switch, CPU hotspot, memory hotspot, and code hotspot features on or off. You can also set the effective IP addresses or CIDR block. For more information, see Continuous profiling for Java applications.

Information desensitization settings

In the Information desensitization section, you can configure a desensitization rule to desensitize JVM system parameters, Kubernetes YAML configurations, method input parameters, Arthas environment variables, and system variables during data collection. Each element in the rule represents a case-insensitive regular expression. For example, password indicates the regular expression .*password.*. Separate multiple elements with commas (,).

When the ARMS agent collects data, the agent desensitizes the data based on the key. If a key matches a regular expression, the corresponding value is desensitized. As shown in the following figure, if you specify licenseKey in the rule, the value of the -Darms.licenseKey key is desensitized.

Interface call settings

In the Interface call configuration section, you can specify a threshold for slow calls and adjust the exception whitelist, HTTP status code whitelist, and invalid interface filtering policy.

• Slow Call Threshold: Unit: ms. Default value: 500. If the response time of a request exceeds the threshold, the request is marked as slow.

• HTTP status code whitelist:

  By default, HTTP status codes greater than or equal to 400 are classified as error calls. If you do not want certain status codes to be classified as errors, you can set a whitelist to ignore these errors.

  This only affects the HTTP frameworks currently supported by Application Monitoring.

  Affected data: Error count metrics for HTTP server/client (arms_http_requests_error_count, arms_http_client_requests_error_count, arms_app_requests_error_count) and span status.

  Affected Features: Error count on the Overview, Provided Services, and Dependencies tabs, as well as span status, error count, and alerts on the Trace Explorer tab.

  Content Format: Enter single status codes, separate multiple status codes with an commas (,). Fuzzy matching is not supported.

  Example: 403,502

  Default Value: None

• Invalid interface call filtering:

  If you do not want to see some calls on the Provided Services tab, you can enter the names of interfaces whose call details you do not need to view, thereby hiding them. The ARMS agent will not report their observability data.

  Note

  • For the ARMS agent for Java earlier than v4.2.0, this feature will only apply to interfaces appearing on the Provided Services tab.

  • For the ARMS agent for Java v4.2.0 and later, this feature will apply to any LocalRootSpan.

  Affected Data: All metrics and spans corresponding to the interface will be ignored.

  Affected Features: All interface-related metrics on the Overview, Provided Services, and Dependencies tabs, as well as span count, interface call count, error count, and alerts on the Trace Explorer tab.

  Content Format: Use strings or AntPath expressions to match the full names of invalid interfaces. Separate multiple rules with commas (,). (The default AntPath expression is provided for compatibility with existing data and is not recommended to be deleted. New configurations should be appended to the existing rules.)

  Example: /api/test/*,/api/playground/create

  Default Value: //*.jpg,//.png,/**/.js,//*.jpeg,//.pdf,/**/.xlsx,//*.txt,//.docs,/**/.gif,/**/*.csv

• Whether the upstream interface name is recorded in the interface call indicator and Whether the upstream application name is recorded in the interface call indicator:

  Configuration purpose: Control whether to record the upstream applications and upstream interfaces that call this interface in the interface metrics. This primarily affects whether data exists on upstream and downstream traces in the services provided. When an application has many upstream applications, recording this information may lead to a significant increase in the amount of metric reports, thereby increasing costs.

  Configuration effect: As shown in the figure, after disabling the recording of upstream applications and interfaces in the interface metrics, the corresponding data drops to 0.

  

• Whether the interface call indicator records the original status code:

  Configuration purpose: Record the original response code in the HTTP interface-related metrics.

  Configuration effect:

  

• Record HTTP Method Name for Interface: When this switch is enabled, the API operation name on the interface call page includes the HTTP method. For example, the name changes from /api/v1/use/{userId} to GET /api/v1/use/{userId}.

Database call settings

In the Database call configuration section, you can specify a threshold for slow SQL queries and a maximum retention length for collected SQL statements. You can also specify whether to display the variable values and constant values in SQL statements, and whether to record the return value of MySQL queries.

• Show the variable binding value in SQL: captures the variable values bound to the PrepareStatement parameter. This configuration takes effect without the need to restart the application.

  The variable values bound to the PreparedStatement parameter will be recorded and stored as attributes in the corresponding database access span. In these attributes, the key is db.bindvalue and the value is a comma-separated list of variable values.

  • This configuration applies to all JDBC-based frameworks currently supported by ARMS.

  • The following database variable types are supported: Boolean, Byte, Short, Int, Long, Float, Double, BigDecimal, Date.

  • Enabling this configuration will incur additional overhead, which is directly proportional to the number of variables in the SQL statement. Under official stress test scenarios (with one variable), enabling this feature results in less than 0.5% additional CPU overhead, and negligible memory overhead.

  Actual effect when enabled:

  

• Show the constant values in SQL: The SQL statements are truncated without additional processing. This configuration takes effect without the need to restart the application. When different SQL statements contain varying constant values, this may cause SQL statements to appear more diverse and eventually be aggregated into {ARMS_OTHERS}. We do not recommend enabling this option.

Trace context propagation protocol settings

In the Trace context propagation settings section, you can select a trace context propagation protocol based on your business requirements.

By default, the ARMS agent selects a trace context propagation protocol based on whether the request header contains specified protocol headers.

In this section, you can select any protocol as the preferred one. After you select and save your preference, ARMS will prioritize checking for the presence of request headers specified by the selected protocol. For example, with the following configuration, the ARMS agent checks for the protocol header in the order of Jaeger, EagleEye, W3C, SkyWalking, and Zipkin.

You may also choose to enforce the use of a specific protocol. For example, with the following configuration, when a call arrives, the ARMS agent will only check for the presence of request headers defined by the Jaeger protocol. If these headers are not detected, it will not proceed to check for other protocols; instead, it will generate a new trace context.

Configuration purposes:

You may select a preferred or enforced trace context propagation protocol in the following scenarios:

• Requests carry trace contexts in multiple formats, and the default parsing order does not meet requirements. You want to specify a preferred protocol for parsing. For example, if requests carry both W3C and Zipkin trace contexts, by default, the system prioritizes parsing the W3C trace context. You can change this behavior by setting the propagation protocol to Zipkin, thereby prioritizing the Zipkin protocol instead.

• Requests carry trace headers of a specific protocol, but you do not want to reuse those headers and prefer to generate a new trace context using another protocol. For instance, if requests carry Zipkin trace contexts, by default, the system parses the Zipkin trace context. By setting the propagation protocol to W3C and changing the propagation mode to enforced, the system will ignore the Zipkin trace context and generate a new context according to the W3C protocol specifications. Note that if you do not change the propagation mode to enforced, the system will still attempt to parse the Zipkin trace context after failing to parse the W3C context.

Configuration effect:

After configuration, each LocalRootSpan in a trace will include an attribute with the key trace.protocol.type and the value as the current trace protocol used in the trace.

Message queue settings

In the Message Queuing Configuration section, you can customize consumer information.

• Custom RabbitMQ consumer: You can specify a class name for a custom consumer or a class name for an anonymous internal consumer to view the traces of the consumer. Separate multiple consumers with commas (,).

• Custom Kafka consumption method: You can specify a custom consumption method to view the traces and metrics when you consume messages by using the native Kafka SDKs.

• kafka sends automatic message pass-through context: Multiple headers are automatically added with Kafka messages to associate delivery traces with consumption traces.

Agent collection settings

In the Probe Acquisition Configuration section, you can set the maximum number of traces collected per second by the agent, the maximum QPS threshold that the agent can handle, and the log level of the agent.

• Collect call chain: Control whether trace data is reported. It is enabled by default. If disabled, trace data will no longer be reported.

• Maximum probe link acquisition per second: Specify the number of spans that the agent can report per second (for performance purposes, the actual effective threshold may have a deviation of up to 5% from the user-configured threshold). Spans exceeding this limit will not be reported.

• Probe maximum can handle QPS threshold: Specify the number of requests the agent can handle per second (for performance purposes, the actual effective threshold may have a deviation of up to 5% from the user-configured threshold). Requests exceeding this threshold will not be monitored, which indicates that no spans or metrics will be generated for these requests, and the log association with TraceId will also not work.

• Collect internal call data without entry: In ARMS, HTTP services provided by applications, RPC services, scheduled tasks triggered by the application, and message consumption are considered entry points. Database calls, HTTP requests sent within the business logic of these entry points are considered internal calls with entry points. Conversely, HTTP calls, database calls, NoSQL calls, message sends, and RPC calls initiated via JDK thread pools are considered internal calls without entry points. The ARMS agent V4.2.2 and later supports filtering out such data with one click. As shown in the figure below, in the Lettuce framework, commands are periodically executed automatically to ensure a normal connection with the Redis server. Such calls fall under internal calls without entry points. After disabling the collection of such data at the time point shown in the figure, the related data disappears.

  

• Probe log level: Adjust the logging level of the agent, which is useful for troubleshooting issues.

• Custom Metric Collection Configuration: Configure the metrics to be collected and defined by the OpenTelemetry SDK.

Exception advanced filtering settings

In the Exception Advanced Filtering Configuration section, you can configure exception collection rules.

• Collection plug-in exception: Specifies whether to collect plugin exceptions.

• Similar exception stack differentiation depth: This stack depth is used to identify similar types of exceptions. Default value: 2. Modifying this configuration may result in unexpected statistical behavior, so please proceed with caution.

• Exception filtering whitelist: If you do not want to see certain exceptions on the Scenario-based Analysis > Exceptions tab, you can enter the fully qualified class names of exceptions that should not be counted as exceptions, thereby hiding them.

  Affected Data: Exception count metrics (arms_exception_requests_count_raw, arms_exception_requests_seconds_raw)

  and exception information in spans.

  Affected Features: Metrics for corresponding exceptions on the Scenario-based Analysis > Exceptions tab, as well as Exception information in Spans and Exception count alerts for corresponding exceptions on the Trace Explorer tab.

  Content Format: Use the fully qualified class names to identify exceptions that need to be filtered. Separate multiple exceptions with commas (,).

  Example: java.lang.InterruptedException,java.lang.IndexOutOfBoundsException

  Default Value: None

• Exception filtering parent class inheritance: When enabled, if the currently collected exception is a subclass of an exception class configured in the exception filtering whitelist, it will also be filtered.

  Configuration effect: Exceptions that meet the filtering criteria will not be displayed in the ARMS console.

• Exception message filtering: After configuration, exceptions of a specified type whose message field meets the configured conditions will also be filtered.

  • Exception name: specifies which exception the filter applies to.

  • Message conditions: the conditions of the exception message, which can be startsWith, endsWith, or contains.

  • Message keywords: the keyword string.

  Configuration effect: Exceptions that meet the filtering criteria will not be displayed in the ARMS console.

Pooled monitoring settings

In the Pooled Monitoring Configuration section, you can configure collection rules for thread pools and connection pools.

• Thread pool and connection pool monitoring: Specify whether to monitor thread pool metrics in frameworks such as Apache Tomcat, Apache Dubbo, and High-speed Service Framework (HSF). To enable this feature, the ARMS agent must be updated to the latest version.

• Thread Pool Thread Name Pattern Extraction Policy: This feature, by default, replaces all numeric characters in the thread names of any running threads within a thread pool with *. You can also adjust this to only replace the ending characters of the thread names with *. In applications where multiple Dubbo Providers are initiated and these providers have different listening ports, if the default strategy is applied, two thread pools from two Dubbo Providers could be aggregated into one due to identical extracted thread name templates. At this point, adjusting this strategy can help distinguish between the two thread pools.

  Configuration effect: Taking the common Tomcat thread pool as an example, the displayed thread pool thread name would be http-nio-*-exec-* by default. After adjustment, the thread name becomes http-nio-9099-exec-*.

  

• Thread pool using scenario filtering and Thread Pool Thread Name Pattern Filtering: Metrics of certain thread pools can be excluded based on their usage scenarios and thread name patterns.

  Note

  This configuration only takes effect for the ARMS agent for Java v4.2.0 and later.

  • Thread pool usage scenario refers to the context in which the thread is used, currently supporting several types including Tomcat, Vert.x, Undertow, Dubbo, Jetty, AliyunJavaAgent, and default. Among these, AliyunJavaAgent represents the thread pool used by the agent, while "default" stands for other unclassified thread pools.

  • The thread pool thread name pattern indicates the pattern obtained after processing the thread names in the thread pool, such as transforming parts of the thread names that contain numbers into * to get a pattern like http-nio-*-exec-*.

  Affected Data: Thread pool metrics

  Affected Features: Instance Monitoring > Thread Pool Monitoring tab and alerts triggered based on thread pool-related metrics.

  Content Format:

  • Thread Pool Usage Scenario Filtering: The usage scenarios displayed on the Instance Monitoring > Thread Pool Monitoring tab. Separate multiple usage scenarios with commas (,).

  • Thread Pool Thread Name Pattern Filtering: The thread name patterns displayed in the thread pool monitoring tab. Separate multiple usage scenarios with commas (,). The matching method is exact matching and does not support rule-based matching.

  Examples:

  Thread Pool Usage Scenario Filtering: AliyunJavaAgent,Jetty

  Thread Pool Thread Name Pattern Filtering: Catalina-utility-*,DubboServerHandler-*-thread-*

  This means that all thread pool data for the scenario "AliyunJavaAgent" will not be reported. For other scenarios, if the thread pool pattern matches names like Catalina-utility-*, those will also not be reported.

  Default Value: None

Span attributes

• Document the OTel Spec convention attributes: The OpenTelemetry Specification stipulates the attributes that should be included in the spans generated by each plugin type. However, considering the amount of data reported, the ARMS agent does not record these attributes in spans by default. You can enable this based on your needs. For information about the additional attributes added by various frameworks after enabling, see OpenTelemetry Specification documentation.

  The following example shows a span for an HTTP server, where the attributes within the red box are the newly added attributes after enabling the switch.

  

• span Association application label configuration: This is used to control which spans will have the tags bound to the application on the Applications page. By default, all spans include application tags. Considering usage volume, you can attach application tags only to entry spans, such as spans of HTTP servers, RPC servers, message queues, or scheduled tasks.

  The effect after configuring to attach to entry spans is shown in the figure below. For instance, an application includes a pair of tags like test1:value1.

  

  For entry spans, you can see the corresponding tag attributes.

  

  For non-entry spans, they do not contain test1:value.

  

Advanced settings

In the Advanced Settings section, you can configure advanced settings, such as the interfaces that you want to exclude and the maximum length of method stacks.

• Quantile Statistics: You can enable or disable quantile statistics.

• Maximum retention length of method stack: The default value is 128 and the maximum value is 400.

• Call chain compression: Specify whether to simplify duplicated calls such as for loops.

• Maximum display length of request input parameters: The default length is 512 characters and the maximum length is 2,048 characters.

• Asynchronous pass-through scan package name: You can add a scan package for asynchronous propagation to the configurations of your application to monitor asynchronous tasks. After a Runnable object, a Callable object, or a Supplier object is created, the corresponding method in the scan package for asynchronous propagation automatically captures the trace context of the current thread. Then, when threads are used in asynchronous mode, the method propagates the captured trace context to the threads.

• The HTTP response contains TraceId: Specify whether to return the eagleeye-traceid field in the response headers for HTTP requests.

• JVM GC Trigger Root Cause: Specify whether to view the root causes of JVM garbage collection (GC) triggers. You can view the monitoring details on the JVM Monitoring page. For information about the GC trigger root causes, see GC metrics.

Copy application settings to other applications

You can copy the settings of an application to other applications.

Globally apply application settings

You can globally apply the current application settings. If you create a new application, the settings are used by default.

1. In the lower part of the page, click Save current app settings as global default configuration.

2. If the Current settings not saved message appears, click Save to save the configurations and then click Save current app settings as global default configuration.

3. In the message that appears, click OK.

FAQs