This topic describes how to install and register a GitLab runner in a Kubernetes cluster and add a Kubernetes executor to build an application. The topic also provides step-by-step examples to implement a CI/CD pipeline that includes stages such as source code compilation, image build, and application deployment.

Background

In the following example, a Java project is built and deployed in a Kubernetes cluster in Container Service for Kubernetes. The example shows how to use GitLab CI to run GitLab runners, set a Kubernetes executor, and execute a CI/CD pipeline.

Create a GitLab project and upload sample code

  1. Create a GitLab project.
    In this example, the address of the GitLab project is as follows:
    http://xx.xx.xx.xx/demo/gitlab-java-demo.git
  2. Run the following commands to create a copy of the sample code and upload the copy to GitLab:
    $ git clone https://github.com/haoshuwei/gitlab-ci-k8s-demo.git
    $ git remote add gitlab http://xx.xx.xx.xx/demo/gitlab-java-demo.git
    $ git push gitlab master

Install GitLab Runner in a Kubernetes cluster

  1. Obtain registration information about GitLab runners.
    1. Obtain registration information about project specific runners.
      1. Log on to GitLab.
      2. In the top navigation bar, choose Projects > Your projects.
      3. On the Your projects tab, select the target project.
      4. In the left-side navigation pane, choose Settings > CI / CD.
      5. Click Expand on the right of Runners.
      6. Copy the URL and registration token.
    2. Obtain registration information about group runners.
      1. In the top navigation bar, choose Groups > Your groups.
      2. On the Your groups tab, select the target group.
      3. In the left-side navigation pane, choose Settings > CI / CD.
      4. Click Expand on the right of Runners.
      5. Copy the URL and registration token.
    3. Obtain registration information about shared runners.
      Note Only the administrator has the permission to perform this step.
      1. In the top navigation bar, click to go to the Admin Area page.
      2. In the left-side navigation pane, choose Overview > Runners.
      3. Copy the URL and registration token.
  2. Run the following command to obtain a copy of the Helm chart of GitLab Runner.
    $ git clone https://github.com/haoshuwei/gitlab-runner.git
    Replace the gitlabUrl and runnerRegistrationToken fields as follows:
    ## GitLab Runner image
    ##
    image: gitlab/gitlab-runner:alpine-v12.1.0
    
    ## Specify a imagePullPolicy
    ##
    imagePullPolicy: IfNotPresent
    
    ## Default container image to use for initcontainer
    init:
      image: busybox
      tag: latest
    
    ## The GitLab Server URL (with protocol) that want to register the runner against
    ##
    gitlabUrl: http://xx.xx.xx.xx/
    
    ## The Registration Token for adding new Runners to the GitLab Server. This must
    ## be retreived from your GitLab Instance.
    ##
    runnerRegistrationToken: "AMvEWrBTBu-d8czE****"
    ## Unregister all runners before termination
    ##
    unregisterRunners: true
    
    ## Configure the maximum number of concurrent jobs
    ##
    concurrent: 10
    
    ## Defines in seconds how often to check GitLab for a new builds
    ##
    checkInterval: 30
    
    ## For RBAC support:
    ##
    rbac:
      create: true
      clusterWideAccess: false
    
    ## Configure integrated Prometheus metrics exporter
    ##
    metrics:
      enabled: true
    
    ## Configuration for the Pods that that the runner launches for each new job
    ##
    runners:
      ## Default container image to use for builds when none is specified
      ##
      image: ubuntu:16.04
    
      ## Specify the tags associated with the runner. Comma-separated list of tags.
      ##
      tags: "k8s-runner"
    
      ## Run all containers with the privileged flag enabled
      ## This will allow the docker:dind image to run if you need to run Docker
      ## commands. Please read the docs before turning this on:
      ##
      privileged: true
    
      ## Namespace to run Kubernetes jobs in (defaults to the same namespace of this release)
      ##
      namespace: gitlab
    
      cachePath: "/opt/cache"
    
      cache: {}
      builds: {}
      services: {}
      helpers: {}
    
    resources: {}
  3. Run the following commands to install GitLab Runner.
    $ helm package .
    Successfully packaged chart and saved it to: /root/gitlab/gitlab-runner/gitlab-runner-0.1.37.tgz
    
    $ helm install --namespace gitlab gitlab-runner *.tgz
    Check whether the related deployment or pods have been started. If the related deployment or pods have been started, the registered GitLab runners will be displayed in GitLab as follows:

Cache settings

GitLab Runner has a limited cache capacity. You need to mount a volume to store cache data. In the preceding example, the /opt/cache directory is used as the cache directory by default after GitLab Runner is installed. You can modify the runners.cachePath field in the values.yaml file to change the cache directory.

For example, to create a maven cache, add the MAVEN_OPTS variable to the variables field and specify a local cache directory as follows:
variables:
  KUBECONFIG: /etc/deploy/config
  MAVEN_OPTS: "-Dmaven.repo.local=/opt/cache/.m2/repository"
Modify the following fields in the templates/configmap.yaml file to mount docker.sock and the volume for caching:
cat >>/home/gitlab-runner/.gitlab-runner/config.toml <<EOF
            [[runners.kubernetes.volumes.pvc]]
              name = "gitlab-runner-cache"
              mount_path = "{{ .Values.runners.cachePath }}"
            [[runners.kubernetes.volumes.host_path]]
              name = "docker"
              mount_path = "/var/run/docker.sock"
              read_only = true
              host_path = "/var/run/docker.sock"
    EOF

Set global variables

  1. In the top navigation bar, choose Projects > Your projects.
  2. On the Your projects tab, select the target project.
  3. In the left-side navigation pane, choose Settings > CI / CD.
  4. Click Expand on the right of Variables. Add environment variables for GitLab Runner. In this example, add the following three variables:
    • REGISTRY_USERNAME: The username of the image repository.
    • REGISTRY_PASSWORD: The password of the image repository.
    • kube_config: The KubeConfig as an encoded string.
    Run the following command to convert KubeConfig into an encoded string:
    echo $(cat ~/.kube/config | base64) | tr -d " "

Edit the .gitlab-ci.yml file

Use the .gitlab-ci.yml file to compile and build the Java demo project, push the image, and deploy the application. For more information, see .gitlab-ci.yml.example of the gitlabci-java-demo project.

A sample .gitlab-ci.yml file is as follows:
image: docker:stable
stages:
  - package
  - docker_build
  - deploy_k8s
variables:
  KUBECONFIG: /etc/deploy/config
  MAVEN_OPTS: "-Dmaven.repo.local=/opt/cache/.m2/repository"
mvn_build_job:
  image: maven:3.6.2-jdk-14
  stage: package
  tags:
    - k8s-runner
  script:
    - mvn package -B -DskipTests
    - cp target/demo.war /opt/cache
docker_build_job:
  image: docker:latest
  stage: docker_build
  tags:
    - k8s-runner
  script:
    - docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD registry.cn-beijing.aliyuncs.com
    - mkdir target
    - cp /opt/cache/demo.war target/demo.war
    - docker build -t registry.cn-beijing.aliyuncs.com/haoshuwei24/gitlabci-java-demo:$CI_PIPELINE_ID .
    - docker push registry.cn-beijing.aliyuncs.com/haoshuwei24/gitlabci-java-demo:$CI_PIPELINE_ID
deploy_k8s_job:
  image: registry.cn-hangzhou.aliyuncs.com/haoshuwei24/kubectl:1.16.6
  stage: deploy_k8s
  tags:
    - k8s-runner
  script:
    - mkdir -p /etc/deploy
    - echo $kube_config |base64 -d > $KUBECONFIG
    - sed -i "s/IMAGE_TAG/$CI_PIPELINE_ID/g" deployment.yaml
    - cat deployment.yaml
    - kubectl apply -f deployment.yaml
The .gitlab-ci.yml file defines a pipeline that consists of the following three stages:
image: docker:stable  # If no image is specified for each stage of the pipeline, the docker:stable image is used by default.
stages:
  - package                # package the source code
  - docker_build         # build, package, and push the image
  - deploy_k8s           # deploy the application
variables:
  KUBECONFIG: /etc/deploy/config  # define the global variable, namely, KUBECONFIG
  • Package the source code with Maven
    mvn_build_job:     # the job name
      image: maven:3.6.2-jdk-14  # the image used in this stage
      stage: package      # the stage name
      tags:                     # the tags of GitLab Runner
        - k8s-runner
      script:
        - mvn package -B -DskipTests  # run the build script
        - cp target/demo.war /opt/cache  # save the build output to cache
  • Build, package, and push the image
    docker_build_job:  # the job name
      image: docker:latest # the image used in this stage
      stage: docker_build   # the stage name
      tags:                     # the tags of GitLab Runner
        - k8s-runner
      script:
        - docker login -u $REGISTRY_USERNAME -p $REGISTRY_PASSWORD registry.cn-beijing.aliyuncs.com   # log on to the image repository
        - mkdir target
        - cp /opt/cache/demo.war target/demo.war
        - docker build -t registry.cn-beijing.aliyuncs.com/haoshuwei24/gitlabci-java-demo:$CI_PIPELINE_ID .     # package the Docker image. Use the pipeline ID as its tag
        - docker push registry.cn-beijing.aliyuncs.com/haoshuwei24/gitlabci-java-demo:$CI_PIPELINE_ID      # push the Docker image
  • Deploy the application
    deploy_k8s_job:   # the job name
      image: registry.cn-hangzhou.aliyuncs.com/haoshuwei24/kubectl:1.16.6   # the image used in this stage
      stage: deploy_k8s  # the stage name
      tags:                     # the tags of GitLab Runner
        - k8s-runner
      script:
        - mkdir -p /etc/deploy
        - echo $kube_config |base64 -d > $KUBECONFIG    # set the config file that is used to connect the Kubernetes cluster
        - sed -i "s/IMAGE_TAG/$CI_PIPELINE_ID/g" deployment.yaml  # dynamically replace the image tag in the deployment file
        - cat deployment.yaml
        - kubectl apply -f deployment.yaml

Execute the pipeline

After you submit the .gitlab-ci.yml file, the gitlab-java-demo project will automatically detect this file and execute the pipeline, as shown in the following figures:

Access the service

If no namespace is specified in the deployment file, the application is deployed under the GitLab namespace by default.
$ kubectl -n gitlab get svc 
NAME        TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE
java-demo   LoadBalancer   172.19.9.252   xx.xx.xx.xx   80:32349/TCP   1m

Visit xx.xx.xx.xx/demo in your browser to access the service.

For more information about Container Service, see Container Service.

For more information about GitLab CI, see GitLab CI.