Best Practice of Continuous Integration of Xianyu Flutter Hybrid Project

1 Introduction

In the previous article "Flutter Hybrid Engineering Transformation Practice", some students left messages wanting to know some implementation details of extracting Flutter's dependencies to remote, so this article focuses on some specific implementations of Flutter's direct dependency removal in Flutter hybrid projects.

2. Think
Because our Xianyu is currently a mixed development model of Flutter and Native, there are some students who only do Native development and are not familiar with Flutter technology.

(1) If the Flutter engineering structure is directly used for daily development, then this part of the native development students also need to configure the Flutter environment and understand some Flutter technologies, and the cost is relatively high.

(2) Alibaba Group's build system currently does not support the direct construction of Flutter projects, which also requires us to release the native project's direct dependence on Flutter.

In view of these two reasons, we hope to design a Flutter dependency extraction module, which can extract Flutter's dependencies as a Flutter dependency library and publish it to a remote location for reference by pure Native projects. As shown below:

Flutter's direct dependency release

3. Implementation

3.1 Flutter analysis of native project dependencies

When we analyze the Flutter project, we will find that the Native project's dependence on the Flutter project mainly consists of three parts:
1. Flutter library and engine: Flutter's Framework library and engine library.
2. Flutter project: The function of the Flutter module implemented by ourselves is mainly implemented by the dart code in the lib directory of the Flutter project.
3. Self-implemented Flutter Plugin: Our own implementation of the Flutter Plugin.

We unpacked the Android and iOS APP files and found that the main files that Flutter depends on are shown in the following figure:

Flutter dependent files (Flutter products)

in,
The files that Android's Flutter depends on:

1. Flutter library and engine:
icudtl.dat, libflutter.so, and some class files. These are packaged in flutter.jar, which is located in the Flutter library directory.
2. Flutter engineering products:
isolate_snapshot_data, isolate_snapshot_instr, vm_snapshot_data, vm_snapshot_instr, flutter_assets.
3. Flutter Plugin:
The aar files compiled by each plugin.
in:

isolate_snapshot_data application data segment
isolate_snapshot_instr application instruction segment
vm_snapshot_data VM virtual machine data segment
vm_snapshot_instr VM virtual machine instruction segment

The files that Flutter of iOS depends on:
1. Flutter library and engine: Flutter.framework
2. The product of the Flutter project: App.framework
3. Flutter Plugin: The compiled framework of various plugins, other frameworks in the figure

Then we only need to extract the compilation results of these three parts, package them into a form of SDK dependency and provide them to the Native project, and then we can relieve the native project's direct dependence on the Flutter project.

3.2 Android-dependent Flutter library extraction

3.2.1 Analysis of Flutter Compilation Tasks in Android

The Android packaging of the Flutter project actually just inserts a flutter.gradle task into the Android Gradle task, and this flutter.gradle mainly does three things: (This file can be found in [flutter/packages/flutter_tools in the Flutter library] /gradle] directory.)

1. Increase the dependency of flutter.jar.
2. Insert the compilation dependencies of the Flutter Plugin.
3. Insert the compilation task of the Flutter project, and finally copy the products (two isolaate_snapshot files, two vm_snapshot files and the flutter_assets folder) to mergeAssets.outputDir, and finally merge to the assets directory of the APK.

3.2.2 Android's Flutter Dependency Extraction Implementation

After understanding the Android compilation product of the Flutter project, we extract the Flutter dependency on Android as follows:

1. Compile the Flutter project.
The main work of this part is to compile the dart and resource parts of Flutter, which can be compiled with AOT and Bundle commands.

2. Package flutter.jar and the product of the Flutter project into an aar.
The main work of this part is to encapsulate flutter.jar and the product compiled in step 1 into an aar.
(1) Add flutter.jar dependency
(2) Merge Flutter's products to assets

2. Publish this aar and the aar compiled by the Flutter Plugin to the maven repository at the same time.
(1) Publish the aar packaged by the Flutter engineering product

(2) Publish the aar of the Flutter Plugin

3. A pure Native project only needs to compile the aar we publish to maven.
In the usual development phase, we need to be able to rely on the latest aar in real time, so we use the SNAPSHOT version.

3.3 Extraction of Flutter libraries that iOS depends on

3.3.1 How to generate Flutter dependency files in iOS

Execute the compilation command "flutter build ios", and finally the compilation script [xcode_backend.sh] of Flutter will be executed, and this script mainly does the following things:
1. Obtain various parameters (such as project_path, target_path, build_mode, etc.), mainly from various definitions in Generated.xcconfig.
2. Delete App.framework and app.flx in the Flutter directory.
3. Compare Flutter/Flutter.framework with Flutter.framework in the directory. If they are not equal, use the latter to overwrite the former.
4. Obtain the parameters required to generate the App.framework command (build_dir, local_engine_flag, preview_dart_2_flag, aot_flags).
5. Generate App.framework, and copy the generated App.framework and AppFrameworkInfo.plist to the Flutter directory of the XCode project.

3.3.2 Implementation of Flutter Dependency Extraction for iOS

The extraction steps for iOS's Flutter dependencies are as follows:
1. Compile the Flutter project to generate App.framework.
2. Package each plug-in as a static library.
There are two main steps here: one is to make the plugin into a binary file, and the other is to make the registration entry of the plugin into a binary file.
3. Upload these to the remote repository and generate new tags.
4. Pure Native projects only need to update the pod dependencies.

4. Continuous Integration Process of Flutter Hybrid Project

According to the above method, we can relieve the native project's direct dependence on the Flutter project, but there are still some problems in daily development:
1. The Flutter project is updated, and the remote dependency library update is not timely.
2. When the version is integrated, it is easy to forget to update the remote dependency library, resulting in the version not integrating the latest Flutter features.
3. When multiple lines are developing Flutter in parallel, the version management is chaotic, and the problem of remote libraries being overwritten is prone to occur.
4. At least one student is required to follow up and release continuously, and the labor cost is high.
In view of these problems, we introduced our team's CI automation framework to solve it from two aspects:
(About the CI automation framework, we will write an article to share in the future)
On the one hand, there is automation, which reduces labor costs and human errors through automation.
On the other hand, do a good job of version control, and do version control in the form of automation.
Specific operations:
First of all, the compilation and release of the remote library corresponding to the Flutter project is automatically completed every time a pure Native project needs to be built, and the whole process does not require manual intervention.
Secondly, in the development and testing phase, a five-segment version number is used, and the last digit is automatically incremented, so as to ensure that the version numbers of all the Flutter libraries developed in parallel in the testing phase will not conflict.
Finally, in the release phase, a three-segment or four-segment version number is used, which can be consistent with the APP version number, which is convenient for follow-up problem tracing.

Build process in Standalone mode

5. Finally

The Xianyu technical team is a small and powerful engineering and technical team. We not only focus on the effective solution of business problems, but at the same time we are promoting the cutting-edge practice of breaking the division of labor in the technology stack (unification of android/iOS/Html5/Server programming models and languages) and computer vision technology on mobile terminals. As a software engineer on the Xianyu technical team, you have the opportunity to show all your talents and courage, proving that technology development is the driving force for changing lifestyles in the entire product evolution and user problem solving.