To ensure the stability and reliability of your cluster, you must select appropriate Elastic Compute Service (ECS) instance types for cluster nodes. This topic describes the recommended ECS instance types for creating an ACK cluster.
Cluster planning
If you use many small ECS instances when you create an ACK cluster, the following issues may occur:
Network issues: Small worker nodes have limited network resources.
Capacity issues: To ensure cluster stability and reliability, the system reserves some node resources, such as CPU, memory, and disk space, for cluster management and infrastructure components. Small instance types can affect cluster performance and availability. For more information about the node resource reservation policy for ACK, see Node resource reservation policy.
Fragmentation issues: When node resources are allocated, if a container occupies a small ECS instance, the remaining resources on that instance may become unusable for new or restored containers. This happens because of inconsistent or non-contiguous resource needs, leading to waste. For example, a node might only allocate whole CPU cores. If an application needs only a fraction of a core, the rest of that core is wasted.
Using large ECS instances provides the following benefits:
Network benefits: Large instance types have high network bandwidth, which improves resource utilization for bandwidth-intensive applications. Additionally, more containers can communicate within a single ECS instance, which reduces network traffic.
Image pulling benefits: Image pulling is more efficient because an image is pulled only once and can be used by multiple containers on the same instance. In contrast, small ECS instances require more frequent image pulls. This increases the time required to scale a cluster by adding more ECS instances, which negates the benefit of a quick response.
For more information about selecting ECS instance types, see the following sections.
Choose worker node specifications
Select nodes with 4 CPU cores and 8 GB of memory or higher.
Determine the total number of cores and the availability requirements for your cluster's daily operations.
For example, assume a cluster needs a total of 160 CPU cores and can tolerate a 10% failure rate. In this case, select at least 10 ECS instances with 16 CPU cores each. Do not let the peak load exceed 144 cores (160 × 90% = 144). If the fault toleration is 20%, select at least five ECS instances with 32 CPU cores each. Do not let the peak load exceed 128 cores (160 × 80% = 128). This way, if one ECS instance fails, the remaining instances can still support your business.
If your cluster's daily scale reaches about 1,000 cores, consider using ECS Bare Metal Instances. For more information, see Scenarios and benefits of ECS Bare Metal Instances.
Based on pod resource requirements, determine the CPU-to-memory ratio, such as 1:2 or 1:4. For memory-intensive applications, such as Java applications, consider using a 1:8 ratio.
Instructions for persistent memory-optimized instances
When a worker node is a persistent memory-optimized instance, such as an instance of the re6p family, it uses a hybrid memory architecture. This architecture includes both standard memory and persistent memory. To implement persistent storage, see Non-volatile memory volumes. For more information about persistent memory-optimized instances, see Instance families.
Choose master node specifications
When you create an ACK cluster, the master nodes run core components such as etcd, kube-apiserver, and kube-controller. For ACK dedicated clusters in a production environment, select appropriate master node specifications to ensure cluster stability. The required master node specifications depend on the cluster size. Larger clusters require higher specifications.
Cluster size can be measured by the number of nodes, number of pods, deployment frequency, and request volume. For simplicity, this topic uses the number of nodes to measure cluster size.
For personal testing and learning environments, you can use small ECS instance types. However, for production-scale clusters, select master node specifications from the following table to keep the master node load at a safe level.
Number of nodes | Recommended master node specifications |
1 to 5 nodes | 4 CPU cores, 8 GB of memory (Instance types with 2 CPU cores and 4 GB of memory or less are not recommended) |
6 to 20 nodes | 4 CPU cores, 16 GB of memory |
21 to 100 nodes | 8 CPU cores, 32 GB of memory |
100 to 200 nodes | 16 CPU cores, 64 GB of memory |
200 to 500 nodes (Estimate the blast radius risk) | 64 CPU cores, 128 GB of memory |
Scenarios and benefits of ECS Bare Metal Instances
ECS Bare Metal Instance is an innovative computing service developed by Alibaba Cloud based on state-of-the-art virtualization 2.0 technology. Virtualization 2.0 endows ECS bare metal instances with the elasticity of virtual machines (ECS instances), the performance and features of physical machines, and full support for nested virtualization.
ECS Bare Metal Instances have advantages in areas such as dedicated compute resources, encrypted computing, and building new hybrid clouds. For a detailed description of ECS Bare Metal Instances and their supported instance families, see Overview of ECS Bare Metal Instances.
Typical scenarios for ECS Bare Metal Instances include but are not limited to the following:
For clusters that scale to 1,000 cores daily, you can use ECS Bare Metal Instances. An ECS Bare Metal Instance provides at least 96 cores, which lets you build a large-scale cluster with only 10 or 11 instances.
ECS Bare Metal Instances are ideal for scenarios that require rapid scaling of containers. For example, during e-commerce sales promotions, they offer better performance than physical machines with the same configuration and can provide millions of vCPUs of computing power to handle traffic spikes.
Unsupported ECS instance types
General limits
For stability and security reasons, ACK does not support the instance types in the following table for worker or master nodes.
Unsupported instance family or family category | Example of an unsupported instance type | Description | Notes |
Burstable instance family t5 | ecs.t5-lc2m1.nano | Unstable instance performance may cause cluster instability. | None. |
Burstable instance family t6 | ecs.t6-c4m1.large | Unstable instance performance may cause cluster instability. | None. |
Instance types with fewer than 4 CPU cores | ecs.g6.large | Low instance specifications may cause cluster instability. | You can go to the Quota Center to apply for permission to use low-specification ECS instance types for creating clusters and node pools. |
Security-enhanced compute-optimized instance family c6t | ecs.c6t.large | Not supported. | None. |
Security-enhanced general-purpose instance family g6t | ecs.g6t.large | Not supported. | None. |
Super Computing Cluster (SCC) instance family | ecs.sccg7.32xlarge | Not supported. | None. |
For information about the GPU instance types that ACK supports, see GPU-accelerated instance families supported by ACK.
Limits on the Terway network plug-in
If you use the Terway network plug-in, the maximum number of pods that a single node can support is calculated based on the number of elastic network interfaces (ENIs) supported by the node's ECS instance type. Therefore, different Terway modes support different ECS instance types. For more information, see Use the Terway network plug-in.
Shared ENI mode or Shared ENI + Trunk ENI: The pod limit for a single node must be greater than 11, that is,
(<a baseurl="t71560_v1_6_0.xdita" data-node="9548" data-root="84794" data-tag="xref" href="t9548.xdita#concept-sx4-lxv-tdb" id="7dec22dd9eofr">Number of ENIs supported by the ECS instance type</a> - 1) × Number of private IP addresses supported by a single ENI > 11.For example, an ecs.g6.large instance supports 2 ENIs, and each ENI supports 6 private IPv4 addresses. The pod limit for a single node is
(2 - 1) × 6 = 6. This instance type cannot be used.Exclusive ENI pattern: The pod limit for a single node must be >6, that is,
<a baseurl="t71560_v1_6_0.xdita" data-node="9548" data-root="84794" data-tag="xref" href="t9548.xdita#concept-sx4-lxv-tdb" id="027def5f024gd">Number of ENIs supported by the ECS instance type</a>-1 > 6.For example, an ecs.g6.xlarge instance supports 3 ENIs. The pod limit for a single node is
3 - 1 = 2. This instance type cannot be used.