Container Compute Service：在ACS叢集中使用GitLab Runner構建CI/CD環境

背景資訊

GitLab Runner是一個開源的、用於執行GitLab CI/CD流水線作業的專案，作為一個可外置的任務執行架構，它提供了多樣的執行器如Shell、Kubernetes等，讓您可以在私人環境或雲上執行具體的CI作業，並將結果反饋回GitLab中。

Gitlab Runner的核心配置分為兩部分，一部分是Manager Pod的配置和初始化設定，另一部分是Kubernetes Executor的配置。更多資訊，請參見Configure GitLab Runner。

實踐路徑

本文的全部流程如下：

前提條件

已使用kubectl串連Kubernetes叢集。具體操作，請參見擷取叢集kubeconfig並通過kubectl工具串連叢集。

安裝步驟

本文將基於17.3.1版本的gitlab-runner進行，對應Chart版本為0.68.1。關於常規gitlab-runner安裝步驟，請參見GitLab Runner Helm chart。

1. 擷取GitLab Chart包。

   1. 添加GitLab Helm倉庫。

      helm repo add gitlab https://charts.gitlab.io

   2. 更新Chart。

      helm repo update gitlab

   3. 擷取GitLab。

      helm pull gitlab/gitlab-runner --version 0.68.1 && tar zvxf gitlab-runner-0.68.1.tgz

2. 註冊Runner。

   您需要在GitLab控制台Registering runners，並記錄Token資訊。

3. 初始化設定values.yaml。

   展開查看YAML內容

   ## GitLab Runner Image
   ##
   ## By default it's using registry.gitlab.com/gitlab-org/gitlab-runner:alpine-v{VERSION}
   ## where {VERSION} is taken from Chart.yaml from appVersion field
   ##
   ## ref: https://gitlab.com/gitlab-org/gitlab-runner/container_registry/29383?orderBy=NAME&sort=asc&search[]=alpine-v&search[]=
   ##
   ## Note: If you change the image to the ubuntu release
   ##       don't forget to change the securityContext;
   ##       these images run on different user IDs.
   ##
   ...
   
   ## The GitLab Server URL (with protocol) that want to register the runner against
   ## ref: https://docs.gitlab.com/runner/commands/index.html#gitlab-runner-register
   ##
   gitlabUrl: https://jihulab.com/
   
   ## DEPRECATED: The Registration Token for adding new Runners to the GitLab Server.
   ##
   ## ref: https://docs.gitlab.com/ee/ci/runners/new_creation_workflow.html
   ##
   # runnerRegistrationToken: ""
   
   ## The Runner Token for adding new Runners to the GitLab Server. This must
   ## be retrieved from your GitLab instance. It is the token of an already registered runner.
   ## ref: (we don't have docs for that yet, but we want to use an existing token)
   ##
   runnerToken: "glrt-t3_sz6xxxxxxxxxDsWF77"
   #
   
   ## Unregister all runners before termination
   ##
   ## Updating the runner's chart version or configuration will cause the runner container
   ## to be terminated and created again. This may cause your Gitlab instance to reference
   ## non-existant runners. Un-registering the runner before termination mitigates this issue.
   ## ref: https://docs.gitlab.com/runner/commands/index.html#gitlab-runner-unregister
   ##
   unregisterRunners: true
   
   ## When stopping the runner, give it time to wait for its jobs to terminate.
   ##
   ## Updating the runner's chart version or configuration will cause the runner container
   ## to be terminated with a graceful stop request. terminationGracePeriodSeconds
   ## instructs Kubernetes to wait long enough for the runner pod to terminate gracefully.
   ## ref: https://docs.gitlab.com/runner/commands/#signals
   terminationGracePeriodSeconds: 3600
   
   ## Set the certsSecretName in order to pass custom certficates for GitLab Runner to use.
   ## Provide resource name for a Kubernetes Secret Object in the same namespace,
   ## this is used to populate the /home/gitlab-runner/.gitlab-runner/certs/ directory
   ## ref: https://docs.gitlab.com/runner/configuration/tls-self-signed.html#supported-options-for-self-signed-certificates-targeting-the-gitlab-server
   ##
   # certsSecretName:
   
   ## Configure the maximum number of concurrent jobs
   ## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section
   ##
   concurrent: 10
   
   ...
   
   ## For RBAC support:
   rbac:
     ## Specifies whether a Role and RoleBinding should be created
     ## If this value is set to `true`, `serviceAccount.create` should also be set to either `true` or `false`
     ##
     create: true
     ## Define the generated serviceAccountName when create is set to true
     ## It defaults to "gitlab-runner.fullname" if not provided
     ## DEPRECATED: Please use `serviceAccount.name` instead
     generatedServiceAccountName: ""
   ...
   
   ## Configuration for the Pods that the runner launches for each new job
   ##
   runners:
     # runner configuration, where the multi line string is evaluated as a
     # template so you can specify helm values inside of it.
     #
     # tpl: https://helm.sh/docs/howto/charts_tips_and_tricks/#using-the-tpl-function
     # runner configuration: https://docs.gitlab.com/runner/configuration/advanced-configuration.html
     config: |
       [[runners]]
         [runners.kubernetes]
           namespace = "{{.Release.Namespace}}"
           image = "alpine"
   
     ## Absolute path for an existing runner configuration file
     ## Can be used alongside "volumes" and "volumeMounts" to use an external config file
     ## Active if runners.config is empty or null
     configPath: ""
   ...

   配置項	配置描述
   gitlabUrl	註冊Runner的極狐GitLab伺服器的完整URL。例如，https://gitlab.example.com。
   runnerToken	上一步中註冊Runner時產生的Token。此Token是每一個Runner的身份標識，Manager通過此Token資訊關聯由其建立的Pod、Secret等資訊。
   rbac	是否啟用RBAC支援。啟用後會自動建立相關ServiceAccount。 rbac: create: true
   concurrent	配置作業的並發度，預設值是10。Manager Pod本身在管理作業時會有一定的記憶體開銷，如果您的並發度較高，需要適當調大Manager Pod的資源規格。
   unregisterRunners	在Manager Pod退出時執行unregister，此配置一般用於使用runnerRegistrationToken進行註冊的情境，避免每次重建token導致的作業失聯問題。詳細資料，請參見FAQ。
   runners.config	Runner回合組態，以多行字串形式作為runner運行模板，您可以按需調整執行器的相關配置。

4. 安裝GitLab Runner到ACS叢集。

   helm install --namespace default gitlab-runner -f values.yaml --version 0.68.1 gitlab/gitlab-runner

5. 執行以下命令，確認GitLab Runner的運行狀態。

   kubectl get pod | grep gitlab

   預期輸出：

   gitlab-runner-7c5b4xxxxx-xxxxx   1/1     Running     0          5m17s

   出現此輸出表明安裝已經完成。

鏡像構建

本節我們將結合ACS的部分特性，分別以Docker-in-Docker和使用Kaniko的方式進行鏡像構建。使用到的樣本工程，請參見Java demo。

結合ACS的彈性策略降低CI成本

您可以通過以下幾種方式使用ACS提供的best-effort執行個體，進一步降低CI/CD流程中的作業成本。

1. 配置全域或工程維度使用best-effort執行個體。

   建議您全域預設採用best-effort類型，部分特殊情境按需覆蓋配置使用其他執行個體類型。

   1. 通過runners配置進行全域配置。

      ...
      runners:
        config: |
          [[runners]]
            [runners.kubernetes]
              ...
              pod_labels_overwrite_allowed = ".*" #允許工程內通過variable覆蓋Labels
            [[runners.kubernetes.pod_labels]]
            "app" = "acs-gitlab-runner"
            "alibabacloud.com/compute-class" = "general-purpose"
            "alibabacloud.com/compute-qos" = "best-effort" #聲明使用best-effort型
      ...      

   2. 您也可以在每個倉庫下通過gitlab-ci.yml進行按需配置，例如使用Kaniko以非特權模式進行鏡像構建中的variables配置。

2. 配置ResourcePolicy，提高彈性確定性。

   ResourcePolicy可以提供自訂的調度策略，您可以使用其提供的成本優先策略，對CI作業優先使用best-effort執行個體。當地區內此執行個體類型售罄時，可以自動降級至default型執行個體，確保CI作業的連續可用。

   apiVersion: scheduling.alibabacloud.com/v1alpha1
   kind: ResourcePolicy
   metadata:
     name: rp-demo
     namespace: default
   spec:
     selector: # 在selector中標記Pod，表示帶有app=stress標籤的Pod將遵循此調度策略
       app: acs-gitlab-runner
     units: # 在units中定義調度順序
     - resource: acs # 優先申請best-effort類型的資源
       podLabels:
         alibabacloud.com/compute-class: general-purpose
         alibabacloud.com/compute-qos: best-effort
     - resource: acs # 前者庫存不足時，申請default類型的資源
       podLabels:
         alibabacloud.com/compute-class: general-purpose
         alibabacloud.com/compute-qos: default

FAQ

當Manager Pod重建或重啟時，如何處理叢集內出現Pod殘留？

問題原因

一種可能的情況是，重啟或重建Manager Pod後使用了和之前不同的一個RunnerToken進行啟動，RunnerToken作為唯一的身份標識資訊，當產生變化後，無法繼續接管原有的作業Pod。

解決辦法

• 您可以在GitLab控制台重新建立Runner並將RunnerToken持久化到安裝的設定檔中。

• 如果是通過registrationToken在啟動期間進行的Runner註冊，建議掛載一個Secret，並將首次註冊產生的Token資訊持久化到Secret中，確保每次重啟時優先進行讀取操作。

• 建議開啟featureGate FF_USE_POD_ACTIVE_DEADLINE_SECONDS，為每個worker增加TTL，作為備用的資源回收策略。