Statistical metrics - Application Real-Time Monitoring Service

Satisfaction

Application Performance Index (Apdex) is a global standard for evaluating application performance. The user experience of an application can be evaluated based on Apdex by three levels:

Satisfied (0 to T)
Tolerating (T to 4T)
Frustrated (greater than 4T)

The following formula is used to calculate the Apdex score:

Apdex score = (Number of satisfied samples + Number of tolerating samples/2)/Number of total samples

The preceding image is obtained from apdex.org.

The First Paint Time (FPT) is used as the metric to evaluate the Apdex of ARMS. The default value of T is 2 in seconds.

JS stability

In ARMS, JavaScript (JS) stability is evaluated by the JS error rate of pages.

If a JS error occurs in a page view (PV) cycle, this PV cycle is considered as an error sample.

Error rate = Number of error samples/Number of total samples

In addition to the JS errors that are automatically reported, page exceptions also include errors reported when you manually call methods described in API reference.

Access speed

In ARMS, access speed is evaluated by the FPT of a page.

All performance or speed statistics are collected by using the Navigation Timing API defined by the World Wide Web Consortium (W3C).

The preceding image is obtained from www.w3.org.

Table 1. Key performance metrics of web pages
Reported field	Description	Formula	Remarks
FMP	The First Meaningful Paint (FMP).	For more information, see Technical solution of FMP.	None
FPT	The FPT.	responseEnd - fetchStart	This field indicates the duration from the point in time when a request is initiated to the point in time when the browser begins to parse the bytes of the first batch of HTML documents.
TTI	The time to interact (TTI).	domInteractive - fetchStart	This field indicates the point in time when the browser starts to load resources after it resolves all HTML documents and constructs the document object model (DOM).
Ready	The time consumed to complete HTML loading, which is the time consumed to construct the DOM.	domContentLoadEventEnd - fetchStart	If a JS script is synchronously executed on the page, the execution time of the JS script can be calculated based on the following formula: Execution time of the JS script = Ready - TTI.
Load	The time consumed to completely load the page.	loadEventStart - fetchStart	This field can be calculated based on the following formula: Load = FPT + DOM + (Ready - TTI) + Res.
FirstByte	Time to first packet	responseStart - domainLookupStart	None

Table 2. Fields that describe the time consumed in each phase
Reported field	Description	Formula	Remarks
DNS	The time consumed for Domain Name System (DNS) query.	domainLookupEnd - domainLookupStart	None
TCP	The time consumed for Transmission Control Protocol (TCP) connection.	connectEnd - connectStart	None
TTFB	The time to first byte (TTFB), which indicates the time consumed to respond to a request.	responseStart - requestStart	TTFB can be calculated in different ways. For more information about how TTFB is calculated in ARMS, see Google development definition.
Trans	The time consumed for data transmission.	responseEnd - responseStart	None
DOM	The time consumed for DOM resolution.	domInteractive - responseEnd	None
Res	The time consumed for resource loading.	loadEventStart - domContentLoadedEventEnd	The time consumed to synchronously load resources on the page.
SSL	The time consumed for Secure Sockets Layer (SSL) connection.	connectEnd - secureConnectionStart	This field is valid only when HTTPS is used to transmit data.

Table 3. Key performance metrics of mini programs
Reported field	Description	Formula	Remarks
FPT	The FPT.	onShow (first page) - onLaunch (app)	The duration from the point in time when the mini program calls the onLaunch method to the point in time when the mini program calls the onShow method to display the first page.

Success rate of API calls

Success rate of API calls = Number of successful API calling samples/Number of total API calling samples

In addition to the AJAX requests that are automatically reported, samples for calculating the success rate of API calls also include the data reported when you manually call the methods described in API reference.

Log fields

The following tables describe the fields included in ARMS logs.

Table 4. Common fields
Field	Description
uid	The ID of the user. The value is an identifier of the user and can be used to search for the user. You can specify a custom value. If you do not set this field, the SDK automatically assigns an ID to the user and updates the ID every six months.
username	The username. The SDK does not automatically assign a username to the user. If you do not set this field, it is left empty.
release	The ID of the application version.
environment	The production environment.
page	The page.
sampling	The sampling rate.
tag	The custom tag.

Table 5. Fields related to API calls
Field	Description
api	The URL of the API call without parameters.
msg	The response string to the API call.
code	The HTTP status code.
time	The time consumed by the API call.
success	Indicates whether the API call is successful.

Table 6. Fields related to JS errors
Field	Description
msg	The error message.
stack	The stack where the error occurs.
cate	The error type. Valid values: EvalError: The error is related to the `eval()` function. RangeError: The error is reported because the value of a numeric variable or parameter is invalid. ReferenceError: The error is reported because invalid references exist. SyntaxError: The error is reported because the syntax is invalid. TypeError: The error is reported because the type of a variable or parameter is invalid. URIError: The error is reported because the parameters passed to the `encodeURI()` function or the `decodeURI()` function are invalid. AggregateError: Multiple errors are generated by one operation, such as `Promise.any()`. Custom error types.
file	The file where the error occurs.
line	The line where the error occurs.
col	The column where the error occurs.
times	The number of times the error occurs.

Log description

Table 7. Logs for multi-dimensional analysis
Log type	Type	Query field (Common metric fields can be used to query or filter all types of logs)
PV log	PV	When an onload event occurs, PV logs are uploaded to calculate PVs and unique visitors (UVs). Calculation methods: PV: the number of PV logs. UV: PV logs are deduplicated based on the UID.
Performance log	Perf	Performance metrics
Slow loading log (Performance logs in which the loading time is longer than 8 seconds)	RES	Performance metrics
JS error log	Error	The JS error message The URL of the JS error file The JS error type
API log	Fields related to API calls	The API name The message returned by the API call The HTTP status code The time consumed by the API call The domain name Whether the API call is successful TraceID
SUM log	SUM	Custom key: event name (example: scroll-count)
AVG log	AVG	Custom key: event name (example: scroll-time)
Resource error log	ResourceError	The resource error
None	Custom	None Note Custom logs are used for custom business monitoring. Custom logs are generated only if you call the ARMS Agent API to report data. Custom log data is not displayed in the ARMS console. You can query and analyze custom log data in the Log Service console.