All Products
Search

This Product

    Container Service for Kubernetes:Remove a node

    Document Center

    Container Service for Kubernetes:Remove a node

    Last Updated:Oct 20, 2023

    This topic describes how to remove an idle node from a Container Service for Kubernetes (ACK) cluster.

    Prerequisites

    How to remove a node

    After you remove nodes from a node pool, the expected number of nodes in the node pool is reduced. The new expected number of nodes equals the original expected number of nodes minus the number of nodes that are removed. For more information about how to modify the expected number of nodes, see Scale a node pool.

    When you remove a node, you can select the following options:

    • Release ECS Instance: Release the Elastic Compute Service (ECS) instance after the node is removed.

      Important

      If the ECS instance uses the subscription billing method, the system does not release the instance after the node is removed.

    • Drain Node: Automatically drain the node before the node is removed.

      Note

      The timeout period of a node removal task is 30 minutes. If the terminationGracePeriodSeconds parameter (graceful shutdown period) of the pods on a node is set to a large value, such as 40 minutes, node draining cannot be completed before the node removal task times out. This is because the timeout period of the node removal task takes precedence over the graceful shutdown period and the graceful shutdown period is longer than the timeout period. In this case, node draining will be terminated after 30 minutes and considered failed. You need to manually drain the node.

      When node draining fails, the node removal task is immediately canceled. If your application pods on the node require a longer graceful shutdown period, you can run the kubectl drain [node-name] [options] command to manually evict the pods from the node. After the node is drained, you can then remove the node. In this case, do not select the Drain Node option when you remove the node.

    The system performs the following steps to remove a node from a node pool:

    1. If you select the Drain Node option, the system first drains the node. If the system fails to drain the node, the node removal task is canceled. Make sure that the cluster has sufficient node resources for the pods that are evicted from the node that you want to remove. You can run the kubectl drain [node-name] [options] command to manually drain the node before you remove the node. In this case, do not select the Drain Node option when you remove the node.

    2. Remove the node from the Kubernetes API server of the cluster.

    3. Remove the node from the scaling group. If the ECS instance uses the subscription billing method, the system does not release the instance even if you select the Release ECS Instance option. You can remove the instance in the ECS console or by calling the ECS API. If you select the Release ECS Instance option and the ECS instance uses the pay-as-you-go billing method, the system releases the instance after the node is removed. We recommend that you back up the data on the node before you remove the node.

    Precautions

    • When you remove a node, the pods that run on the node are migrated to other nodes. This may cause service interruptions. We recommend that you remove nodes during off-peak hours.
    • Unknown errors may occur when you remove nodes. Before you remove nodes, back up the data on the nodes.
    • Nodes remain in the Unschedulable state when they are being removed.
    • You can remove only worker nodes. You cannot remove master nodes.
    • We recommend that you remove nodes in the ACK console. If you run the kubectl delete node command to remove nodes from an ACK cluster, take note of the following limits:
      • The removed nodes cannot be added to other ACK clusters.
      • After you delete a cluster, the Elastic Compute Service (ECS) instances where the removed nodes are deployed are automatically released.
    • If you use the ECS or Auto Scaling console or use the ECS or Auto Scaling API to remove nodes or release instances in a node pool, the node pool automatically scales to the expected number of nodes. A node pool also automatically scales to the expected number of nodes when subscription instances in the node pool expire and are automatically released. Therefore, if you want to change the number of nodes in a node pool, modify the expected number of nodes or manually remove nodes from the node pool in the ACK console.

    Procedure

    1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

    2. On the Clusters page, click the name of the cluster that you want to manage and choose Nodes > Node Pools in the left-side navigation pane.

    3. On the Node Pools page, click the name of the node pool that you want to manage. On the Nodes tab, select the node that you want to remove and click Remove Node.
    4. Optional:In the Remove Node dialog box, select Release ECS Instance and Drain the Node.
      • Release ECS Instance:
        • Only pas-as-you-go ECS instances are released. The system continues to bill ECS instances that are not released.
        • Subscription ECS instances are automatically released after the subscription expires.
        • If you do not select Release ECS Instance, you are still charged for the ECS instance where the node is deployed.
      • Drain the Node: Select this option to migrate pods that run on the nodes to be removed to other nodes in the cluster. If you select this option, make sure that the other nodes have sufficient resources for these pods.
        You can also run the kubectl drain node-name [options] command to migrate pods that run on the nodes to be removed to other nodes in the cluster.
        Note
        • node-name must be in the format of your-region-name.node-id.

          your-region-name specifies the region where the cluster that you want to manage is deployed. node-id specifies the ID of the ECS instance where the node to be removed is deployed. Example: cn-hangzhou.i-xxx.

        • options specifies the optional parameters of the command. Example: --force --ignore-daemonsets --delete-local-data. You can run the kubectl drain --help command to view help information.
    5. In the message that appears, click Confirm.
      Note If a message that indicates a node removal failure appears, clear Release ECS Instance and Drain the Node, and then remove the nodes again.

    FAQ

    Issue

    The node that you want to remove from a cluster is not displayed on the Nodes page.

    Solution

    Perform the following steps to remove the node:

    1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

    2. On the Clusters page, click the name of the cluster that you want to manage and choose Nodes > Nodes in the left-side navigation pane.

    3. On the Nodes page, select the nodes that you want to remove and click Batch Remove.
    4. In the Remove Node dialog box, confirm the information and click OK.

    References