Golang Unit Testing: Misunderstandings and Practices

This article introduces some common problems encountered in the process of expanding the unit testing practice using a real-world application project.

Unit testing plays its value in all processes of software development as a powerful weapon of development. The process of the original development model should gradually change to the mode of DevOps. After the development, the software will be handed over to the test team for end-to-end tests. This article is an example of a specific practical process of transformation. This article introduces some common problems encountered in the process of expanding the unit testing practice using a real-world application project. This article also introduces several mock methods for reference in some businesses with more complex dependencies.

Background

Testing is an effective means to ensure code quality, while unit testing is the minimum verification of program modules. The importance of unit testing is self-evident. Compared with manual testing, unit testing features automatic execution, automatic regression, and high efficiency. Unit testing also has relatively high problem discovery efficiency. You can write a unit testing case in the development phase, daily push daily test, and measure the quality of the code by the success rate and coverage rate of unit testing. This ensures the overall quality of the project.

Unit Testing Rule

What is good unit testing? The Alibaba Java Coding Guidelines describes the characteristics of good unit testing:

A: (Automatic): Unit testing should be fully automatic and non-interactive.
I: (Independent): Unit testing cases must not be called between each other or rely on the execution order to ensure unit testing is stable, reliable, and easy to maintain.
R: (Repeatable): Unit testing is usually placed in continuous integration, and unit testing is executed each time there is a code check in. If unit testing depends on the external environment, such as a network, service, or middleware, the continuous integration mechanism may become unavailable.

The unit testing should be repeatable, and external dependencies and environmental changes should be blocked by mock or other means.

The article "On the Architecture for Unit Testing" [1] provides the following descriptions of good unit testing:

Brief: Only one test purpose
Easy: Easy data structure and cleansing
Fast: Execution within seconds
Standard: Adhering to strict conventions (prepare test context, execute key operations, and verify results)

Misunderstandings in Unit Testing

There is no assertion. Unit testing without assertion has no soul. If you only print out the results, unit testing is meaningless.
Continuous integration is not supported. Unit testing should not run locally. Instead, it should be integrated into the entire research and development process. The unit testing execution should be triggered and can be repeated when the code is merged and released.
The granularity is too large. The granularity of unit testing should be as small as possible, and computing logic should not be involved too much. It should only have input, output, and assertion.

Many people are not willing to write the unit testing because the project relies on a lot of calls between various functions and do not know how to test in an isolated test environment.

In practice, we investigated several mock methods. The following section describes these methods one by one.

Unit Testing Practice

The engineering project of this practice is an HTTP (gin-based HTTP framework) service. This article will introduce the unit testing process and take the function of the controller layer of the entrance as the measured function. The following function generates the CodeReview data of the code repository of the user based on the employee ID.

As we can see below, this function is relatively simple to use as the entry layer. It performs a parameter verification, calls the downstream, and reveals the result.

func ListRepoCrAggregateMetrics(c *gin.Context) {
    workNo := c.Query("work_no")
    if workNo == "" {
        c.JSON(http.StatusOK, errors.BuildRsp(errors.ErrorWarpper(errors.ErrParamError.ErrorCode, "work no miss"), nil))
        return
    }
    crCtx := code_review.NewCrCtx(c)
    rsp, err := crCtx.ListRepoCrAggregateMetrics(workNo)
    if err != nil {
        c.JSON(http.StatusOK, errors.BuildRsp(errors.ErrorWarpper(errors.ErrDbQueryError.ErrorCode, err.Error()), rsp))
        return
    }
    c.JSON(http.StatusOK, errors.BuildRsp(errors.ErrSuccess, rsp))
}

The rough results are listed below:

{
  "data": {
    "total": 10,
    "code_review": [
      {
        "repo": {
          "project_id": 1,
          "repo_url": "test"
        },
        "metrics": {
          "code_review_rate": 0.0977918,
          "thousand_comment_count": 0,
          "self_submit_code_review_rate": 0,
          "average_merge_cost": 30462.584,
          "average_accept_cost": 30388.75
        }
      }
    ]
  },
  "errorCode": 0,
  "errorMsg": "Success"
}

This function test will cover the following scenarios:

If workNo is empty, an error is returned.
If workNo is not empty, downstream calls are successful, and repos cr aggregates data.
If workNo is not empty, downstream calls fail, and an error message is returned.

Solution One: Do Not Mock Downstream Data. Mock Data Depends on Storage (Not Recommended).

This method connects all dependent storage systems, such as sqlite and redis, to a local host through configuration files. This way, the downstream has not been mocked but continues to call.

var db *gorm.DB
func getMetricsRepo() *model.MetricsRepo {
    repo := model.MetricsRepo{
        ProjectID:     2,
        RepoPath:      "/",
        FileCount:     5,
        CodeLineCount: 76,
        OwnerWorkNo:   "999999",
    }
    return &repo
}
func getTeam() *model.Teams {
    team := model.Teams{
        WorkNo: "999999",
    }
    return &team
}
func init() {
    db, err := gorm.Open("sqlite3", "test.db")
    if err != nil {
        os.Exit(-1)
    }
    db.Debug()
    db.DropTableIfExists(model.MetricsRepo{})
    db.DropTableIfExists(model.Teams{})
    db.CreateTable(model.MetricsRepo{})
    db.CreateTable(model.Teams{})
    db.FirstOrCreate(getMetricsRepo())
    db.FirstOrCreate(getTeam())
}
type RepoMetrics struct {
    CodeReviewRate           float32 `json:"code_review_rate"`            
    ThousandCommentCount     uint    `json:"thousand_comment_count"`       
    SelfSubmitCodeReviewRate float32 `json:"self_submit_code_review_rate"` 
}
type RepoCodeReview struct {
    Repo        repo.Repo   `json:"repo"`
    RepoMetrics RepoMetrics `json:"metrics"`
}
type RepoCrMetricsRsp struct {
    Total          int               `json:"total"`
    RepoCodeReview []*RepoCodeReview `json:"code_review"`
}
func TestListRepoCrAggregateMetrics(t *testing.T) {
    w := httptest.NewRecorder()
    _, engine := gin.CreateTestContext(w)
    engine.GET("/api/test/code_review/repo", ListRepoCrAggregateMetrics)
    req, _ := http.NewRequest("GET", "/api/test/code_review/repo?work_no=999999", nil)
    engine.ServeHTTP(w, req)
    assert.Equal(t, w.Code, 200)
    var v map[string]RepoCrMetricsRsp
    json.Unmarshal(w.Body.Bytes(), &v)
    assert.EqualValues(t, 1, v["data"].Total)
    assert.EqualValues(t, 2, v["data"].RepoCodeReview[0].Repo.ProjectID)
    assert.EqualValues(t, 0, v["data"].RepoCodeReview[0].RepoMetrics.CodeReviewRate)
}

We have not changed the tested code above. However, we must specify the configuration to the test configuration when we run the Go test. The project to be tested is set through environment variables.

RDSC_CONF=$sourcepath/test/data/config.yml go test -v -cover=true -coverprofile=$sourcepath/cover/cover.cover ./...

Initialize the test environment, clear the database data, and write the test data
Execute the test method
Assert the test result

Solution Two: Downstream is Mocked Through the Interface (Recommended)

Gomock [2], provided by Golang, is a mock framework used in Go. It can be integrated with the Go testing module and used in other test environments. Gomock includes two parts, the gomock, and the mockgen. Gomock manages the pile objects, and mockgen generates the corresponding mock files.

type Foo interface {
  Bar(x int) int
}
func SUT(f Foo) {
 // ...
}
ctrl := gomock.NewController(t)
  // Assert that Bar() is invoked.
  defer ctrl.Finish()
  //mockgen -source=foo.g
  m := NewMockFoo(ctrl)
  // Asserts that the first and only call to Bar() is passed 99.
  // Anything else will fail.
  m.
    EXPECT().
    Bar(gomock.Eq(99)).
    Return(101)
SUT(m)

In the example above, the Foo interface is mocked. Back to our project, in our test code above, it is called through an internal declaration object. When gomock is used, the code needs to be modified to expose dependencies through parameters, and then initialization is performed. The modified function that will be tested is listed below:

type RepoCrCRController struct {
    c     *gin.Context
    crCtx code_review.CrCtxInterface
}
func NewRepoCrCRController(ctx *gin.Context, cr code_review.CrCtxInterface) *TeamCRController {
    return &TeamCRController{c: ctx, crCtx: cr}
}
func (ctrl *RepoCrCRController)ListRepoCrAggregateMetrics(c *gin.Context) {
    workNo := c.Query("work_no")
    if workNo == "" {
        c.JSON(http.StatusOK, errors.BuildRsp(errors.ErrorWarpper(errors.ErrParamError.ErrorCode, "Employee ID information error"), nil))
        return
    }
    rsp, err := ctrl.crCtx.ListRepoCrAggregateMetrics(workNo)
    if err != nil {
        c.JSON(http.StatusOK, errors.BuildRsp(errors.ErrorWarpper(errors.ErrDbQueryError.ErrorCode, err.Error()), rsp))
        return
    }
    c.JSON(http.StatusOK, errors.BuildRsp(errors.ErrSuccess, rsp))
}

Now, we can test the mock interface generated by gomock:

func TestListRepoCrAggregateMetrics(t *testing.T) { 
    ctrl := gomock.NewController(t)
    defer ctrl.Finish()
    m := mock.NewMockCrCtxInterface(ctrl)
    resp := &code_review.RepoCrMetricsRsp{
    }
    m.EXPECT().ListRepoCrAggregateMetrics("999999").Return(resp, nil)
    w := httptest.NewRecorder()
    ctx, engine := gin.CreateTestContext(w)
    repoCtrl := NewRepoCrCRController(ctx, m)
    engine.GET("/api/test/code_review/repo", repoCtrl.ListRepoCrAggregateMetrics)
    req, _ := http.NewRequest("GET", "/api/test/code_review/repo?work_no=999999", nil)
    engine.ServeHTTP(w, req)
    assert.Equal(t, w.Code, 200)
    got := gin.H{}
    json.NewDecoder(w.Body).Decode(&got)
    assert.EqualValues(t, got["errorCode"], 0)
}

Solution Three: Mock Downstream Data Using the Monkey Patch (Recommended)

In the example above, we need to modify the code to implement the mock interface. For object member functions, mock is impossible. The monkey patch implements the mock of the instance method by modifying the contents of the underlying pointer at runtime. Note: The instance method must be accessible. The tests in the monkey mode are listed below:

func TestListRepoCrAggregateMetrics(t *testing.T) {
    w := httptest.NewRecorder()
    _, engine := gin.CreateTestContext(w)
    engine.GET("/api/test/code_review/repo", ListRepoCrAggregateMetrics)
    var crCtx *code_review.CrCtx
    repoRet := code_review.RepoCrMetricsRsp{
    }
    monkey.PatchInstanceMethod(reflect.TypeOf(crCtx), "ListRepoCrAggregateMetrics",
        func(ctx *code_review.CrCtx, workNo string) (*code_review.RepoCrMetricsRsp, error) {
            if workNo == "999999" {
                repoRet.Total = 0
                repoRet.RepoCodeReview = []*code_review.RepoCodeReview{}
            }
            return &repoRet, nil
        })
    req, _ := http.NewRequest("GET", "/api/test/code_review/repo?work_no=999999", nil)
    engine.ServeHTTP(w, req)
    assert.Equal(t, w.Code, 200)
    var v map[string]code_review.RepoCrMetricsRsp
    json.Unmarshal(w.Body.Bytes(), &v)
    assert.EqualValues(t, 0, v["data"].Total)
    assert.Len(t, v["data"].RepoCodeReview, 0)
}

The Mock of the Storage Layer

Go-sqlmock can mock the interface sql/driver [3]. It removes the need for a real database and simulates the behavior of the sql driver to implement a powerful underlying data test. The following example shows how to use table driven [4] to perform data-related tests:

package store
import (
    "database/sql/driver"
    "github.com/DATA-DOG/go-sqlmock"
    "github.com/gin-gonic/gin"
    "github.com/jinzhu/gorm"
    "github.com/stretchr/testify/assert"
    "net/http/httptest"
    "testing"
)
type RepoCommitAndCRCountMetric struct {
    ProjectID                 uint `json:"project_id"`
    RepoCommitCount           uint `json:"repo_commit_count"`
    RepoCodeReviewCommitCount uint `json:"repo_code_review_commit_count"`
}
var (
    w      = httptest.NewRecorder()
    ctx, _ = gin.CreateTestContext(w)
    ret    = []RepoCommitAndCRCountMetric{}
)
func TestCrStore_FindColumnValues1(t *testing.T) {
    type fields struct {
        g  *gin.Context
        db func() *gorm.DB
    }
    type args struct {
        table      string
        column     string
        whereAndOr []SqlFilter
        group      string
        out        interface{}
    }
    tests := []struct {
        name      string
        fields    fields
        args      args
        wantErr   bool
        checkFunc func()
    }{
        {
            name: "whereAndOr is null",
            fields: fields{
                db: func() *gorm.DB {
                    sqlDb, mock, _ := sqlmock.New(sqlmock.QueryMatcherOption(sqlmock.QueryMatcherEqual))
                    rs1 := sqlmock.NewRows([]string{"project_id", "repo_commit_count", "repo_code_review_commit_count"}).FromCSVString("1, 2, 3")
                    mock.ExpectQuery("SELECT project_id, sum(commit_count) as repo_commit_count, sum(code_review_commit_count) as repo_code_review_commit_count FROM `metrics_repo_cr` GROUP BY project_id").WillReturnRows(rs1)
                    gdb, _ := gorm.Open("mysql", sqlDb)
                    gdb.Debug()
                    return gdb
                },
            },
            args: args{
                table:      "metrics_repo_cr",
                column:     "project_id, sum(commit_count) as repo_commit_count, sum(code_review_commit_count) as repo_code_review_commit_count",
                whereAndOr: []SqlFilter{},
                group:      "project_id",
                out:        &ret,
            },
            checkFunc: func() {
                assert.EqualValues(t, 1, ret[0].ProjectID, "project id should be 1")
                assert.EqualValues(t, 2, ret[0].RepoCommitCount, "RepoCommitCount id should be 2")
                assert.EqualValues(t, 3, ret[0].RepoCodeReviewCommitCount, "RepoCodeReviewCommitCount should be 3")
            },
        },
        {
            name: "whereAndOr is not null",
            fields: fields{
                db: func() *gorm.DB {
                    sqlDb, mock, _ := sqlmock.New(sqlmock.QueryMatcherOption(sqlmock.QueryMatcherEqual))
                    rs1 := sqlmock.NewRows([]string{"project_id", "repo_commit_count", "repo_code_review_commit_count"}).FromCSVString("1, 2, 3")
                    mock.ExpectQuery("SELECT project_id, sum(commit_count) as repo_commit_count, sum(code_review_commit_count) as repo_code_review_commit_count FROM `metrics_repo_cr` WHERE (metrics_repo_cr.project_id in (?)) GROUP BY project_id").
                        WithArgs(driver.Value(1)).WillReturnRows(rs1)
                    gdb, _ := gorm.Open("mysql", sqlDb)
                    gdb.Debug()
                    return gdb
                },
            },
            args: args{
                table:  "metrics_repo_cr",
                column: "project_id, sum(commit_count) as repo_commit_count, sum(code_review_commit_count) as repo_code_review_commit_count",
                whereAndOr: []SqlFilter{
                    {
                        Condition: SQLWHERE,
                        Query:     "metrics_repo_cr.project_id in (?)",
                        Arg:       []uint{1},
                    },
                },
                group: "project_id",
                out:   &ret,
            },
            checkFunc: func() {
                assert.EqualValues(t, 1, ret[0].ProjectID, "project id should be 1")
                assert.EqualValues(t, 2, ret[0].RepoCommitCount, "RepoCommitCount id should be 2")
                assert.EqualValues(t, 3, ret[0].RepoCodeReviewCommitCount, "RepoCodeReviewCommitCount should be 3")
            },
        },
        {
            name: "group is null",
            fields: fields{
                db: func() *gorm.DB {
                    sqlDb, mock, _ := sqlmock.New(sqlmock.QueryMatcherOption(sqlmock.QueryMatcherEqual))
                    rs1 := sqlmock.NewRows([]string{"project_id", "repo_commit_count", "repo_code_review_commit_count"}).FromCSVString("1, 2, 3")
                    mock.ExpectQuery("SELECT project_id, sum(commit_count) as repo_commit_count, sum(code_review_commit_count) as repo_code_review_commit_count FROM `metrics_repo_cr` WHERE (metrics_repo_cr.project_id in (?))").
                        WithArgs(driver.Value(1)).WillReturnRows(rs1)
                    gdb, _ := gorm.Open("mysql", sqlDb)
                    gdb.Debug()
                    return gdb
                },
            },
            args: args{
                table:  "metrics_repo_cr",
                column: "project_id, sum(commit_count) as repo_commit_count, sum(code_review_commit_count) as repo_code_review_commit_count",
                whereAndOr: []SqlFilter{
                    {
                        Condition: SQLWHERE,
                        Query:     "metrics_repo_cr.project_id in (?)",
                        Arg:       []uint{1},
                    },
                },
                group: "",
                out:   &ret,
            },
            checkFunc: func() {
                assert.EqualValues(t, 1, ret[0].ProjectID, "project id should be 1")
                assert.EqualValues(t, 2, ret[0].RepoCommitCount, "RepoCommitCount id should be 2")
                assert.EqualValues(t, 3, ret[0].RepoCodeReviewCommitCount, "RepoCodeReviewCommitCount should be 3")
            },
        },
    }
    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            cs := &CrStore{
                g: ctx,
            }
            db = tt.fields.db()
            if err := cs.FindColumnValues(tt.args.table, tt.args.column, tt.args.whereAndOr, tt.args.group, tt.args.out); (err != nil) != tt.wantErr {
                t.Errorf("FindColumnValues() error = %v, wantErr %v", err, tt.wantErr)
            }
            tt.checkFunc()
        })
    }
}

Continuous Integration

Aone, the Alibaba internal project collaboration management platform, provides a function similar to travis-ci [5]: the test service [6]. We can perform unit testing integration by creating a unit testing task or using the lab.

# Run the test command.
mkdir -p $sourcepath/cover
RDSC_CONF=$sourcepath/config/config.yaml go test -v -cover=true -coverprofile=$sourcepath/cover/cover.cover ./...
ret=$?; if [[ $ret -ne 0 && $ret -ne 1 ]]; then exit $ret; fi

Incremental coverage can be converted to xml reports through gocov/gocov-xml incremental report output through diff_cover:

cp $sourcepath/cover/cover.cover /root/cover/cover.cover
pip install diff-cover==2.6.1
gocov convert cover/cover.cover | gocov-xml > coverage.xml
cd $sourcepath
diff-cover $sourcepath/coverage.xml --compare-branch=remotes/origin/develop > diff.out

Then, set the triggered integration phase: