Alicloud Image Builder is an image building tool provided by Alibaba Cloud that simplifies and automates image building. You can use OS images created by using Alicloud Image Builder as custom images to create node pools in Container Service for Kubernetes (ACK) clusters. This allows you to quickly add nodes to ACK clusters. This topic describes how to run Alicloud Image Builder as a Job to create custom OS images in ACK clusters.
Prerequisites
An ACK cluster is created. For more information, see Create an ACK managed cluster.
A kubectl client is connected to the cluster. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.
Background information
The node pools in ACK clusters support auto scaling. By default, when you create a node pool, you can select OS images such as CentOS and Alibaba Cloud Linux 2. These OS images meet the requirements of most scenarios. However, in scenarios that require preinstallation or high performance, these images may be unable to meet your requirements. Alibaba Cloud provides Alicloud Image Builder to help you build custom OS images and facilitate auto scaling in complex scenarios.
To use Alicloud Image Builder to create custom images, you can create a Job or a CronJob to distribute the image building task in the cluster.
Create a Job to quickly build a custom OS image
In this example, a ConfigMap named build-config and a Job named build are created to show how to use Alicloud Image Builder to quickly build a custom OS image.
Create a ConfigMap named build-config to specify the parameters for the OS image.
Create a YAML file named build-config.yaml and copy the following content to the file:
The following table describes the parameters in the YAML file.
Table 1. Alicloud Image Builder parameters Parameter
Example
Description
variables{"<variable1>":"<value>"}
variables{"access_key":"{{env ALICLOUD_ACCESS_KEY}}"}
The
variables
that are used by Alicloud Image Builder.NoteIf you write sensitive information such as AccessKey pairs (
access_key
andsecret_key
) to the configuration file, the information may be leaked. To ensure data security, you can specify AccessKey pairs as variables. The values of the variables are based on the input values of the runtime.builders{"type":"<value>"}
builders{"type":"alicloud-ecs"}
The image
builders
. When type is set to aliyun-ecs, a temporary Elastic Compute Service (ECS) instance is created to build the image. The ECS instance is automatically released after the image is built.provisioners{"type":"<value>"}
provisioners{"type":"shell"}
The image
provisioners
that are used to specify the operations that need to be performed on the temporary instance. When type is set to shell, a shell provisioner is used. A shell command is automatically run after the Linux instance is connected. For example, you can run theyum install redis.x86_64 -y
command to install Redis.For more information about how to configure provisioners, see Provisioner configuration.
Table 2. Image building parameters Parameter
Example
Description
Importance
access_key
LTAInPyXXXXQ****
The AccessKey ID that is used to create the custom image. For more information, see Obtain an AccessKey pair.
Required
secret_key
CM1ycKrrCekQ0dhXXXXXXXXXl7y****
The AccessKey secret that is used to create the custom image.
Required
region
cn-beijing
The region where the custom image is to be created.
Required
image_name
ack-custom_image
The name of the custom image to be created. The name must be globally unique.
Required
source_image
aliyun_2_1903_x64_20G_alibase_20200904.vhd
The ID of the Alibaba Cloud public image based on which the custom image is created. The created custom image contains the same operating system as the public image.
Required
instance_type
ecs.c6.xlarge
The instance type of the ECS instance that is created from the base image specified in the source_image parameter. The ECS instance is used to run the preinstallation task and build the custom image. If you want to use a GPU-accelerated image, specify a GPU-accelerated instance type.
Required
RUNTIME
docker
The container runtime. Valid values: docker and containerd.
Required
SKIP_SECURITY_FIX
true
Specifies whether to skip security patching.
Required
KUBE_VERSION
1.22.3-aliyun.1
The Kubernetes version of the cluster.
Required
PRESET_GPU
true
Specifies whether to preinstall a GPU driver to accelerate startup.
Optional
NVIDIA_DRIVER_VERSION
460.91.03
The version of the preinstalled GPU driver. The default value is 460.91.03.
Optional
OS_ARCH
amd64
The CPU architecture. Valid values: amd64 and arm64.
Required
ImportantBefore you specify a custom image for a node pool, make sure that the configurations of the node pool are the same as the build settings of the custom image. Otherwise, nodes created from the custom image cannot be added to the cluster. The configurations of the node pool include the cluster version, container runtime, and GPU-accelerated instance type.
When you verify the custom image, select a regular node pool that uses the same build settings as the custom image. After you use the custom image to create nodes and add the nodes to the node pool, check whether your application can run on the nodes as expected.
Run the following command to deploy Alicloud Image Builder in the cluster:
kubectl apply -f build-config.yaml
Create a Job to build a custom OS image.
Use the following YAML template to grant permissions to the account that uses the AccessKey pair.
Run the following command to generate encrypted strings for the AccessKey pair.
echo -n "AKxxxxxxxxxxxxxxx" | base64 echo -n "SKxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" | base64
Use the following YAML template to create a Secret named my-secret.
apiVersion: v1 kind: Secret metadata: name: my-secret namespace: default type: Opaque data: ALICLOUD_ACCESS_KEY: TFRxxxxxxxxxxxxxRTkx // The Base64-encoded string in the previous substep. ALICLOUD_SECRET_KEY: a0zxxxxxxxxxxxxxx2UThl
Create a YAML file named build.yaml and add the following content to the file:
Configure variables to run the Job. The ECS instance of the specified instance type (instance_type) created from the base image (source_image) is used to build the custom image. The ECS instance belongs to the Alibaba Cloud account to which the AccessKey pair belongs. The system then runs the configurations of the provisioner and pushes the image built by the ECS instance to the specified region as a custom image. The custom image also belongs to the Alibaba Cloud account to which the AccessKey pair belongs.
Run the following command to deploy the Job and start building the image:
kubectl apply -f build.yaml
Optional:Log on to the ACK console and check the image building log.
A log is generated during the image building process. The log records all image building operations, including checking parameters, creating temporary resources, pre-installing software, creating target resources, and releasing temporary resources. You can check the image building log by performing the following steps:
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, click the name of the cluster that you want to manage and choose in the left-side navigation pane.
On the Jobs page, find the Job that you created and click Details in the Actions column.
On the Job details page, click the Logs tab to check the image building log.
Provisioner configuration
A provisioner is a component used to install and configure software in a running operating system before the operating system is packaged into an OS image. A provisioner is often used to install software in images in the following scenarios:
Install software.
Patch kernels.
Create a user.
Download application code.
Build a custom Alibaba Cloud Linux 3 image.
Common operations by using provisioners:
Execute shell scripts.
"provisioners": [{ "type": "shell", "script": "script.sh" }]
Execute orchestration scripts by using Ansible.
"provisioners": [ { "type": "ansible", "playbook_file": "./playbook.yml" } ]
Install the Cloud Paralleled File System (CPFS) client.
The installation of CPFS requires multiple installation packages, some of which involve real-time compilation and may require an extended period of time to install. The use of a custom image can greatly reduce the cost of installing the CPFS client on a large number of nodes. The following code block provides a sample configuration.
Customize the OS image of a GPU-accelerated node.
ImportantYou cannot deploy images with GPU drivers preinstalled on CPU-accelerated nodes.
Add the application image to the system image.
Pull an image from a private repository when the runtime is Docker:
docker login <Image address> -u user -p password docker pull nginx
Pull an image from a private repository when the runtime is containerd:
ctr -n k8s.io i pull --user=username:password nginx
Pull an image from a private repository after the custom image is built.
Run the following
docker login
command on a Linux server that has Docker installed to generate a certificate:docker login --username=zhongwei.***@aliyun-test.com --password xxxxxxxxxx registry.cn-beijing.aliyuncs.com
After the
docker login
command succeeds, a certificate named config.json is created in the/root/.docker
directory.Create a ConfigMap based on the certificate named config.json.
apiVersion: v1 kind: ConfigMap metadata: name: docker-config data: config.json: |- { "auths": { "registry.cn-beijing.aliyuncs.com": { "auth": "xxxxxxxxxxxxxx" } }, "HttpHeaders": { "User-Agent": "Docker-Client/19.03.15 (linux)" } }
Modify the YAML file of the Job to mount the ConfigMap to the pod.
Add the content in the following figure to the build-config ConfigMap.
Run the Job.
Specify the maximum numbers of concurrent uploads and downloads for the image.
Log on to the ACK console and click Clusters in the left-side navigation pane.
On the Clusters page, click the name of the cluster that you want to manage and choose
in the left-side navigation pane.Click the name of the node pool that you want to manage. Click the Overview tab. In the Node Pool Information section, click the link next to Scaling Group.
Click the Instance Configuration Sources tab. Find the scaling configuration that you want to manage and click Edit in the Actions column.
In the Note message, click OK.
On the Modify Scaling Configuration page, modify the parameters and click Advanced Settings to show the advanced settings. Record the data in the Instance User Data box. Decode the data in the Instance User Data box by using Base64.
After you decode the data, append the following code to the end of the decoded data.
yum install -y jq echo "$jq '. += {"max-concurrent-downloads": 20,"max-concurrent-uploads": 20}' /etc/docker/daemon.json" > /etc/docker/daemon.json service docker restart
Encode the modified data in Base64 and overwrite the original data in the Instance User Data box with the modified data.
Click Modify. In the dialog box that appears, click Modify.
Use the following template to build a custom Alibaba Cloud Linux 3 image:
Use the following template to build a custom Red Hat Enterprise Linux 9 (RHEL 9) image:
What to do next
After you create a custom image by using Alicloud Image Builder, you can create an elastic node pool based on the custom image to quickly add nodes to the cluster. For more information about how to create an elastic node pool, see Auto scaling of nodes.