All Products
Search
Document Center

Mobile Testing:Terms

Last Updated:Feb 28, 2022

Term

Description

compatibility testing

Compatibility testing is used to verify whether compatibility issues exist when operations such as installation, startup, logon, overwrite installation of different versions, and uninstallation are performed on an app across different devices. The system obtains the operational logs, error information, and performance data of the app when the app is running, provides a compatibility test report after a test is complete, and shows the running process of the app in the form of screenshots or videos.

functional testing

Functional testing is used to verify whether an app functions as expected when test cases for the app are executed on different devices. The system obtains the operational logs, error information, and performance data of the app when the app is running, provides a functional test report after a test is complete, and shows the running process of the app in the form of screenshots or videos.

performance testing

Performance testing is used to verify the basic performance data of an app when the app is running on different devices. The data includes but is not limited to the startup time, CPU utilization, memory usage, and network consumption. After a test is complete, the system provides a performance test report.

expert service

The expert service provides app testing services such as compatibility testing, functional testing, and remote debugging. Alibaba Cloud test experts offer one-on-one support to customize test scripts, provide professional test report analysis, and deliver test reports within 48 hours.

UI adaptation testing

UI adaptation testing is used in the expert service to verify whether the page layout and style of an app are displayed normally on different devices. It is also used to verify whether operations can be performed on the app, whether the interaction with controls works as expected, and whether the UI is user-friendly.

performance analysis

Performance analysis is used to monitor the basic performance data of an app when the app is running. The data includes but is not limited to the startup time, CPU utilization, memory usage, and network consumption.

intelligent exploration

In compatibility testing, Ripper enables autonomous decision-making on app compatibility exploration and greatly improves testing efficiency. Ripper is an intelligent AI program developed by Alibaba Group based on Machine Learning Platform for AI (PAI).

app

An Android or iOS app for compatibility testing, functional testing, performance testing, or remote debugging.

task

Every time an Android or iOS compatibility test or functional test is submitted, a test task is created.

use case library

Use case libraries are used for separate management of test cases based on versions. We recommend that you use a use case library version that corresponds to the current iteration of an app.

module

Modules can be used in combination with business logics of apps to further manage use case sets in a centralized manner. For example, Homepage, My, and Orders are commonly used modules.

use case set

The smallest executable test unit. A use case set consists of one or more use cases, and has different parameters to match different environments.

parameter group

A collection of parameters that have relevance to each other.

parameter

An input parameter that is used in conjunction with a script.

script

A set of instructions used to execute test cases. Mobile Testing supports two types of scripts: recorded script and uploaded script.

recorded script

A test script generated by means of online recording.

uploaded script

A test script written and uploaded by a user.

custom step

A step that is created by writing a script based on the Appium engine and Python language and adding the script to a recorded script to implement unconventional operations.

test status

The execution status of a compatibility test, functional test, or performance test. The status can be Waiting, Running, Completed, or Canceled.

execution result

The execution result of a compatibility test, functional test, or performance test. The result can be Passed, Failed, or Abnormal/Not Executed.

error type

The type of errors generated during a compatibility test, functional test, or performance test. Error types include Crash, Stuck, and Ignorable Exception.

CPU utilization

The percentage of CPU resources used by an app during the testing process.

memory usage

The amount of memory resources used by an app during the testing process.

network consumption

The amount of network resources used by an app during the testing process.

FPS

The number of frames transmitted per second, which is used to measure the number of displayed frames. The more frames transmitted per second, the smoother the displayed motion.

frame loss rate

The percentage of frames that are lost because the hardware is not powerful enough to keep up with the refresh frequency of the display.

power consumption

The device power consumed by an app during the testing process.

script log

An error log provided in the Python language.

Appium log

An error log provided by the Appium engine.

device log

An error log provided by the test device.

error level

A piece of information that tells how important an error message is. Error levels include Verbose, Error, Warn, Info, and Debug.

stuck

A phenomenon in which the system halts due to flaws in software design or insufficient memory space.

crash

A phenomenon in which an app exits abnormally while it is running.

ignorable exception

An exception that is captured by logs but is not a stuck or crash.

ANR

An acronym for application not responding. If an app does not respond to user input events such as key input, touch screen input, and gesture input within 5 seconds, or if a BroadcastReceiver does not finish executing within 10 seconds, an ANR error message appears. You can click Wait to keep the app running or click Force Close. If you perform time-consuming operations such as database, I/O, and network access in UI threads, ANR errors may occur.

jank

The result of an app not being able to keep up with the refresh rate of display due to hardware limitations or flaws in software design.

trendline

A statistical chart with time as the horizontal axis and observed variables as the vertical axis. A trendline is used to reflect the relationship between time and quantity as well as the trends and deviations of observed variables over time.

distribution chart

A chart that organizes all units in the gross sample by group on the basis of statistical grouping to reflect the distribution status of the units among groups.