Designing a Flutter Tracking Framework for High Accuracy

By Cheng Zhe, nicknamed Lanhao at Alibaba.

User behavior tracking is used to record a series of behaviors of a user. It also provides the core data basis for judgments by business personnel. If data is missing or inaccurate, the business may suffer irrecoverable losses. When business code was migrated from Native to Flutter for Xianyu—Alibaba's second hand buy-sell market, the native tracking solution could be combined with Flutter. However, it is irresponsible to only migrate business features, so, after continuous study, we developed a user behavior tracking solution with high accuracy in Flutter.

What Is User Behavior Tracking

First, I will discuss how user behavior tracking is defined. On the user timeline shown in the following figure, after a user enters page A, the user will find button X and click this button. Then, the new page, page B, will appear.

The following five tracking events occurred on this timeline:

Entered page A: The first frame of page A is rendered and a focus was obtained.
Exposed placement X: Button X is displayed on the mobile screen for a period of time so that the user can see and tap the button.
Clicked placement X: The user is interested in the content of button X and clicked button X. Button X responds to the click and a new page is opened.
Exited page A: Page A losses the user's focus.
Entered page B: The first frame of page B is rendered and a focus is obtained.

Here, the most important thing is the tracking time, or to be more precise the time for triggering the tracking of a specific event. The following describes Xianyu's implementation solution in Flutter.

Implementation Solution

Entering and Exiting a Page

In Native, an Android client listens to the onResume and onPause events of an Activity as the events of entering and exiting a page. Similarly, an iOS client listens to the viewWillAppear and viewDidDisappear events of the UIViewController as the events of entering and exiting a page. Meanwhile, the entire page stack is maintained by the Android and iOS operating systems.

In Flutter, Android and iOS clients use FlutterActivity and FlutterViewController as the containers for carrying Flutter pages. Through the containers, a native page (FlutterActivity or FlutterViewController) can be used to switch native Flutter pages. In other words, Flutter maintains its own page stack. This means that the native solution we are most familiar with cannot directly work in Flutter.

To solve this problem, many people may think of registering a NavigatorObserver that can listen to Flutter so that they can detect the push and pop events on Flutter pages. However, two problems may occur in this case:

Assume that pages A and B enter the page stack in a sequence (A enter > A leave > B enter). Then, page B exits (B leave) and page A is visible again. In this case, no push event for page A (A enter) is received.
Assume that Dialog or BottomSheet pops up on page A. Although both the two operations trigger the push method to Navigator, page A does not exit in the sense of users.

Fortunately, the page stack of Flutter is not as complex as the page stack of Android Native. Therefore, to solve the first problem, we can maintain an index list that matches the page stack. When a push event for page A is received, an index of page A is inserted into the queue. When a push event for page B is received, the system checks whether the list contains pages. If yes, the system records a page exit event for the last page in the list, records a page entry event for page B, and then inserts an index of page B into the queue. When a pop event on page B is received, the system records a page exit event for page B and then judges whether the page (assuming it is page A) corresponding to the last index in the queue is on the top of the stack (ModalRoute.of(context).isCurrent).

If yes, the system records a page entry event for page A.

To solve the second issue, the Route class provides the member variable OverlayEntries, which can be used to obtain the OverlayEntry of all layers corresponding to the current route. The OverlayEntry object provides the member variable opaque, which can be used to determine whether the current layer covers the full screen so that Dialog and BottomSheet can be excluded. In combination with the first issue, a push operation must be added to the preceding solution to determine whether a new page is valid. If the new page is valid, the page exit event is recorded for the previous page in the index list and the valid page is added to the index list. If the new page is invalid, the index list is not operated on.

The preceding solution is not provided by Xianyu, but only a recommendation by the author. Xianyu did not use the native page stack management solution of Flutter when implementing the Flutter framework, but used the Native + Flutter hybrid solution instead. For more information, see the previous article Opened Source | Start Flutter Hybrid Development with FlutterBoost. Therefore, I will introduce Xianyu's solution based on the Native + Flutter hybrid solution.

The following figures show the Xianyu solution. For this, we use Android as an example, but this also applies to iOS.

Note that, if a page is opened for the first time, the page is a new page that is opened based on the hybrid stack. If a page is not opened for the first time, the page is moved from the background to the foreground by means of page rollback.

It seems that Flutter determines when to trigger the page entry and exit events. In fact, it retains consistency with the page stack management in Native. The tracking time in Native is notified to Flutter, and then Flutter immediately calls the tracking method in Native through a channel. You may ask why we don't have Native directly complete all management operations. Direct management by Native is suitable when a page is not opened for the first time, but is not applicable when opening a page for the first time. When opening a page for the first time, Flutter does not initialize the page or know the page information when onResume is called. In this case, the page entry tracking API in Native must be called during initialization (init) on the Flutter page. We directly trigger the page entry and exit events in Flutter so that developers do not have to pay attention to whether a Flutter page is opened for the first time.

Placement Exposure

We think that images and text are of exposure significance while what other users cannot see is of no exposure significance. Therefore, we define a valid exposure to occur if a placement meets the following two conditions:

The visible area of the placement on the screen is equal to or greater than half of the overall area.
The placement stays within the visible area of the screen for more than 500 milliseconds.

Based on this definition, we can quickly design a scenario like that shown in the following figure. A scrolling page contains placements A, B, C, and D. Among the four placements,

Placement A is outside of the visible area on the screen, and is therefore invisible;
Placement B is about to leave the visible area on the screen, and therefore will switch from visible to invisible;
Placement C is in the middle of the visible area on the screen, and is therefore visible;
Placement D is about to enter the visible area on the screen, and therefore will switch from invisible to visible.

We need to calculate the proportion of the exposed area of a placement on the screen. To calculate this proportion, we need to know the following values:

Offset of the container from the screen
Offset of the placement from the container
Location, width, and height of the placement
Location, width, and height of the container

It is easy to obtain and calculate the widths and heights of the placement and container.

Obtain the offset of the container from the screen.

//监听容器滚动，得到容器的偏移量
double _scrollContainerOffset = scrollNotification.metrics.pixels;

Obtain the offset of the placement from the container.

//曝光坑位Widget的context
final RenderObject childRenderObject = context.findRenderObject();
final RenderAbstractViewport viewport = RenderAbstractViewport.of(childRenderObject);
if (viewport == null) {
  return;
}
if (!childRenderObject.attached) {
  return;
}
//曝光坑位在容器内的偏移量
final RevealedOffset offsetToRevealTop = viewport.getOffsetToReveal(childRenderObject, 0.0);

Perform logical judgment.

if (当前坑位是invisible && 曝光比例 >= 0.5) {
  记录当前坑位是visible状态
  记录出现时间
} else if (当前坑位是visible && 曝光比例 < 0.5) {
  记录当前坑位是invisible状态
  if (当前时间-出现时间 > 500ms) {
    调用曝光埋点接口
  }
}

Placement Clicks

It is easy to track placement clicks by using the following solution:

Results

After several rounds of iteration and optimization, the tracking accuracy of online Flutter pages has reached 100%, providing strong support for business analysis and judgment. In addition, this solution means that the business personnel do not have to worry about page entry and exit or placement exposure during development. In other words, they do not need to care about the triggering time, making it easy to use and non-invasive methods.

Outlook

For the page entry and exit scenario, our solution is not universal because Xianyu also uses the Flutter Boost hybrid solution. However, with the increasing number of Flutter pages on Xianyu, we will continue to implement native solutions based on Flutter in the future.

Developing data-driven businesses on Xianyu is important and significant. However, tracking directly affects data collection, and tracking loss and errors may leave us without direction. We are accustomed to using data to direct our work by performing experiments, obtaining data for analysis, adjusting the experiments, obtaining new data for analysis, and adjusting the experiments again. With this loop, we aim to find the design that best suits our users.

Community

Designing a Flutter Tracking Framework for High Accuracy

What Is User Behavior Tracking

Implementation Solution

Entering and Exiting a Page

Placement Exposure

Placement Clicks

Results

Outlook

Read previous post:

Read next post:

XianYu Tech

You may also like

Comments

XianYu Tech

Related Products

ACK One

Container Registry

Container Service for Kubernetes

Elastic Container Instance