Istio Practice in Alibaba Cloud Container Service for Kubernetes: Automatic Sidecar Injection

This article describes how to enable or disable automatic sidecar injection in Alibaba Cloud Container Service for Kubernetes, and analyzes the working principle of automatic sidecar injection.

Alibaba Cloud Container Service for Kubernetes supports integration of Istio with Log Service. To learn more information about integration of Istio and Log Service based distributed tracing system, see Istio Practice in Alibaba Cloud Container Service for Kubernetes: Integrated Log Service.

This article describes how to enable or disable automatic sidecar injection in Alibaba Cloud Container Service for Kubernetes, and analyzes the working principle of automatic sidecar injection.The automatic injection function can simplify the application deployment logic.In addition, Alibaba Cloud Container Service for Kubernetes provides the configuration option so that you can enable or disable automatic sidecar injection based on your own requirements.

Review the Kubernetes Webhook Mechanism

Admission is a term used in Kubernetes. It refers to a phase in the resource request process of the Kubernetes API server.As shown in the figure below, when the API server receives a resource creation request, the request is authenticated and authorized, goes through the admission phase, and then is saved to the etcd.

As shown in the figure, admission has two important phases: mutating and validating. The logic executed in each of the two phases is as follows:

Mutating: The request content can be modified in the mutating phase.
Validating: The request content cannot be modified in the validating phase, but the Kubernetes API server can determine whether to accept or reject this request based on the request content.

Kubernetes V1.9 or later introduces the admission webhook extended mechanism. With the webhook callback function, developers can flexibly extend functions of the Kubernetes API server, and verify or modify resources when the API server creates resources.

The advantage of webhook is that you can extend the functions of the API server without modifying or re-compiling the source code of the API server.The inserted logic is implemented as an independent process, and is passed in to Kubernetes as parameters. Kubernetes calls back the extended logic when processing its own logic.

With the admission webhook, you can add two types of webhook plugin: Mutation and Validation.

MutatingAdmissionWebhook allows you to mutate the resource objects in the webhook.
ValidatingAdmissionWebhook conducts some checks and can reject users' requests to add additional admission policies.

When the cluster administrator needs to forcibly check or modify some or all requests, he/she can consider the use of ValidatingAdmissionWebhook or MutatingAdmissionWebhook.

Enable the Webhook Plugins

On Kubernetes, the prerequisite for normal operation of some advanced features is that the related admission module is in enabled state.If the admission module is not properly configured, the Kubernetes API server cannot run properly or some functions may fail.

The Kubernetes API server has a flag: --enable-admission-plugins. Its value is a list of admission modules that are separated by commas. After this flag is configured, admission modules can be called according to the preconfigured sequence before any object operations.

--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota

In the cluster kube-apiserver created by Alibaba Cloud Container Service for Kubernetes (V1.10.4) by default, the MutatingAdmissionWebhook and ValidatingAdmissionWebhook plugins are enabled.You can run the following command to check the pod enabling parameters corresponding to kube-apiserver:

kubectl describe po -l component=kube-apiserver -n kube-system
Command:
      kube-apiserver
      --apiserver-count=500
      --runtime-config=admissionregistration.k8s.io/v1alpha1
      ......
      --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota
      ......

Use the Webhook to Automatically Inject Istio Sidecars

Istio fully utilizes the Kubernetes webhook mechanism to realize automatic injection of envoys, proxies, and sidecars.

Sidecar Injection Configuration Item

First, create the sidecar injection configuration item istio-sidecar-injector, as shown below:

kubectl describe configmap istio-sidecar-injector -n istio-system
Name:         istio-sidecar-injector
Namespace:    istio-system
Labels:       app=ack-istio
              chart=ack-istio-1.0.0
              heritage=Tiller
              istio=sidecar-injector
              release=myistio
Annotations:  <none>

Data
====
config:
----
policy: enabled
template: |-
  initContainers:
  - name: istio-init
    image: "registry.cn-hangzhou.aliyuncs.com/aliacs-app-catalog/proxy_init:1.0.0"
    args:
    ......
    imagePullPolicy: IfNotPresent
    securityContext:
      capabilities:
        add:
        - NET_ADMIN
      privileged: true
    restartPolicy: Always

  containers:
  - name: istio-proxy
    image: [[ if (isset .ObjectMeta.Annotations "sidecar.istio.io/proxyImage") -]]
    ......

As shown in the figure, ConfigMap stores the default injection policy and sidecar injection template.

Policy

disabled: The sidecar injector is not injected to the pod by default.The injection function is enabled if you set sidecar.istio.io/inject in the comment of the pod template to true.
enabled: The sidecar injector is injected to the pod by default.The injection function is disabled if you set sidecar.istio.io/inject in the comment of the pod template to false.

Sidecar Injection Webhook

Second, deploy the sidecar injection webhook, as shown below:

mutatingwebhook.yaml:

apiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingWebhookConfiguration
metadata:
  name: istio-sidecar-injector
  namespace: {{ .Release.Namespace }}
  labels:
    app: istio-sidecar-injector
    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
    release: {{ .Release.Name }}
    heritage: {{ .Release.Service }}
webhooks:
  - name: sidecar-injector.istio.io
    clientConfig:
      service:
        name: istio-sidecar-injector
        namespace: {{ .Release.Namespace }}
        path: "/inject"
      caBundle: ""
    rules:
      - operations: [ "CREATE" ]
        apiGroups: [""]
        apiVersions: ["v1"]
        resources: ["pods"]
    failurePolicy: Fail
    namespaceSelector:
{{- if .Values.enableNamespacesByDefault }}
      matchExpressions:
      - key: istio-injection
        operator: NotIn
        values:
        - disabled
{{- else }}
      matchLabels:
        istio-injection: enabled
{{- end }}

Note that if enableNamespacesByDefault is set to true, the matching rules of the namespace selector are used to determine whether to enable automatic sidecar injection by default, that is, the value of istio-injection is not disabled. If enableNamespacesByDefault is set to false, automatic sidecar injection is enabled only when the flag istio-injection is set to enabled in the namespace.

Sidecar Injection Deployment

You can view the deployed container istio-sidecar-injector on the following page:

The corresponding deployment.yaml file is as follows:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: istio-sidecar-injector
  namespace: {{ .Release.Namespace }}
  labels:
    app: {{ template "sidecar-injector.name" .}}
    chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
    release: {{ .Release.Name }}
    heritage: {{ .Release.Service }}
    istio: sidecar-injector
spec:
  replicas: {{ .Values.replicaCount }}
  template:
    metadata:
      labels:
        istio: sidecar-injector
    spec:
      serviceAccountName: istio-sidecar-injector-service-account
      containers:
        - name: sidecar-injector-webhook
          image: "{{ .Values.global.hub }}/{{ .Values.image }}:{{ .Values.global.tag }}"
          imagePullPolicy: {{ .Values.global.imagePullPolicy }}
          args:
            - --caCertFile=/etc/istio/certs/root-cert.pem
            - --tlsCertFile=/etc/istio/certs/cert-chain.pem
            - --tlsKeyFile=/etc/istio/certs/key.pem
            - --injectConfig=/etc/istio/inject/config
            - --meshConfig=/etc/istio/config/mesh
            - --healthCheckInterval=2s
            - --healthCheckFile=/health

Enable the Namespace for which Sidecars Need to be Automatically Injected

Based on the preceding rule for enabling automatic sidecar injection, automatic sidecar injection is enabled only when enableNamespacesByDefault is set to false and the flag istio-injection is set to enabled in the namespace.

Run the following command to add the flag istio-injection to the namespace default and set this flag to enabled.

kubectl label namespace default istio-injection=enabled

Now, the name space default supports automatic sidecar injection. Running the following command is equivalent to manual injection described in the previous article. When you run the command, the system automatically injects the envoy container as the sidecar.

kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml

Summary

This article describes how to enable or disable automatic sidecar injection in Alibaba Cloud Container Service for Kubernetes, and analyzes the working principle of automatic sidecar injection.

This article series introduces Istio and its core components, as well as describes how to quickly build an Istio open platform for connecting, managing, and securing microservices on the basis of Alibaba Cloud Container Service for Kubernetes. These articles also use an official example to demonstrate how to deploy an application in the Istio environment; how to configure intelligent routing and distributed tracing; and how to configure Istio functions of collecting, querying, and visualizing the telemetry data.

To review these articles, see:

Community

Istio Practice in Alibaba Cloud Container Service for Kubernetes: Automatic Sidecar Injection

Review the Kubernetes Webhook Mechanism

Enable the Webhook Plugins

Use the Webhook to Automatically Inject Istio Sidecars

Sidecar Injection Configuration Item

Sidecar Injection Webhook

Sidecar Injection Deployment

Enable the Namespace for which Sidecars Need to be Automatically Injected

Summary

Read previous post:

Read next post:

Xi Ning Wang(王夕宁)

You may also like

Comments

Xi Ning Wang(王夕宁)

Related Products

Container Service for Kubernetes

ACK One

Cloud-Native Applications Management Solution

Container Registry