Term | Description |
compatibility testing | Compatibility testing is used to verify whether compatibility issues exist when operations such as installation, startup, logon, overwrite installation of different versions, and uninstallation are performed on an app across different devices. The system obtains the operational logs, error information, and performance data of the app when the app is running, provides a compatibility test report after a test is complete, and shows the running process of the app in the form of screenshots or videos. |
functional testing | Functional testing is used to verify whether an app functions as expected when test cases for the app are executed on different devices. The system obtains the operational logs, error information, and performance data of the app when the app is running, provides a functional test report after a test is complete, and shows the running process of the app in the form of screenshots or videos. |
performance testing | Performance testing is used to verify the basic performance data of an app when the app is running on different devices. The data includes but is not limited to the startup time, CPU utilization, memory usage, and network usage. After a test is complete, the system provides a performance test report. |
performance analysis | Performance analysis is used to monitor the basic performance data of an app when the app is running. The data includes but is not limited to the startup time, CPU utilization, memory usage, and network usage. |
intelligent exploration | In compatibility testing, Ripper enables autonomous decision-making on app compatibility exploration and greatly improves testing efficiency. Ripper is an intelligent AI program developed by Alibaba Group based on Machine Learning Platform for AI (PAI). |
app | An Android or iOS app for compatibility testing, functional testing, performance testing, or remote debugging. |
task | Every time an Android or iOS compatibility test or functional test is submitted, a test task is created. |
test case library | Test case libraries are used for separate management of test cases based on versions. We recommend that you use a test case library version that corresponds to the current iteration of an app. |
Directory | Directories can be used in combination with business logics of apps to further manage test cases in a centralized manner. For example, Homepage, My, and Orders are common directories. |
test case | The smallest executable test unit. A test case consists of one or more scripts, and has different parameters to match different environments. |
parameter group | A collection of parameters that have relevance to each other. |
parameter | An input parameter that is used in combination with a script. |
script | A set of instructions used to execute test cases. Mobile Testing supports two types of scripts: recorded script and uploaded script. |
recorded script | A test script generated by means of online recording. |
uploaded script | A test script written and uploaded by a user. |
custom step | A step that is created by writing a script based on the Appium engine and Python language and adding the script to a recorded script to implement unconventional operations. |
test status | The execution status of a compatibility test, functional test, or performance test. The status can be Waiting, Running, Completed, or Canceled. |
execution result | The execution result of a compatibility test, functional test, or performance test. The result can be Passed, Failed, or Abnormal/Not Executed. |
error type | The type of errors generated during a compatibility test, functional test, or performance test. The error types include Crash, Stuck, and Ignorable Exception. |
CPU utilization | The percentage of CPU resources used by an app during the testing process. |
memory usage | The amount of memory resources used by an app during the testing process. |
network usage | The amount of network resources used by an app during the testing process. |
FPS | The number of frames transmitted per second, which is used to measure the number of displayed frames. The more frames transmitted per second, the smoother the displayed motion. |
frame loss rate | The percentage of frames that are lost because the hardware is not powerful enough to keep up with the refresh frequency of the display. |
power consumption | The device power consumed by an app during the testing process. |
script log | An error log provided in the Python language. |
Appium log | An error log provided by the Appium engine. |
device log | An error log provided by the test device. |
error level | A piece of information that tells how important an error message is. The error levels include Verbose, Error, Warn, Info, and Debug. |
stuck | A phenomenon in which the system halts due to flaws in software design or insufficient memory space. |
crash | A phenomenon in which an app exits abnormally while it is running. |
negligible exception | An exception that is captured by logs but is not a stuck or crash. |
ANR | An acronym for application not responding. If an app fails to respond to a user input event within 5 seconds or BroadReceiver does not receive a task within 10 seconds, the system displays an Application Not Responding (ANR) dialog box. The user can choose to wait or force the system to terminate the app. The user input events include keyboard inputs, touch screen inputs, and gesture inputs. If you perform time-consuming operations related to database, I/O, and networks in UI threads, ANR errors may occur. |
jank | The result of an app not being able to keep up with the refresh rate of display due to hardware limitations or flaws in software design. |
trend chart | A statistical chart with time as the horizontal axis and observed variables as the vertical axis. A trendline is used to reflect the relationship between time and quantity as well as the trends and deviations of observed variables over time. |
distribution chart | A chart that organizes all units in the gross sample by group on the basis of statistical grouping to reflect the distribution status of the units among groups. |