Container Service for Kubernetes:Troubleshoot cluster access exceptions from the console

Symptoms

The ACK console displays the following error when you access a cluster resource:

An error occurred while processing your request to the API server of the current cluster.

Error code: ErrorQueryClusterNamespace or APIServer.500

Cause

The load balancing configuration of the API server is invalid or the API server status is abnormal, causing ACK management services to fail to connect to the API server.

Solution

Step 1: Verify that the SLB instance exists

1. Log on to the ACK consoleACK console. In the left navigation pane, click Clusters.

2. Click the target cluster name. In the left navigation pane, click Cluster Information.

3. On the Basic Information tab, click the link next to API server SLB to open the Server Load Balancer (SLB) console.

If the console displays The specified SLB ID does not exist., the SLB instance for the API server has been deleted or released and the cluster cannot be restored. Recreate the cluster. For details, see Create an ACK managed cluster.

Step 2: Check the SLB instance status

Check whether the Status of the SLB instance is Running.

If the status is not Running, check for overdue payments or subscription expiry. Settle any overdue payments or renew the subscription, then restart the SLB instance. For billing details, see CLB billing.

Step 3: Verify the listener configuration

Click the Listener tab and check for a listener where both Frontend Protocol/Port and Backend Protocol/Port are set to TCP:6443 and Status is Running.

If no such listener exists or the listener is not running:

• If the listener exists but its Status is Stopped, select it and click Enable.

• If the listener does not exist:

  • For ACK managed clusters, submit a ticketsubmit a ticketsubmit a ticketsubmit a ticketsubmit a ticket. ACK manages the master nodes for managed clusters, so the platform must restore the listener configuration.

  • For ACK dedicated clusters, add all master nodes to the default server group. Then create a listener with Frontend Protocol/Port and Backend Protocol/Port both set to TCP:6443, associate it with the default server group, and start the listener. For details, see Add a TCP listener.

Step 4: Check the health check status

Check whether the Health Check Status column of the listener shows Normal.

If the status is not Normal, the backend servers of the SLB instance are abnormal:

• For ACK managed clusters, submit a ticketsubmit a ticketsubmit a ticketsubmit a ticketsubmit a ticket. ACK manages the master nodes for managed clusters and you cannot access them directly.

• For ACK dedicated clusters, troubleshoot as follows. If the issue persists after troubleshooting, submit a ticketsubmit a ticketsubmit a ticketsubmit a ticketsubmit a ticket.

  1. On the Nodes > Nodes page of the ACK consoleACK console, click the ECS instance ID of each master node and verify that the instance is in the Running state in the ECS console.

  2. Log on to each master node via the ECS console and check the API server container status. For how to log on to a master node, see Connection method overview.

     • Docker runtime:

       docker ps | grep kube-apiserver

       Then run docker inspect <container-id> to check the container status.

     • containerd runtime:

       crictl ps | grep kube-apiserver

       Then run crictl inspect <container-id> to check the container status.

Step 5: Check listener access control

Check whether access control is enabled for the listener.

If access control is enabled, the whitelist is missing the internal IP range used by ACK management services. Add the CIDR block 100.104.0.0/16 to the whitelist. This CIDR block covers the source IP addresses of internal requests sent by ACK management services to the API server. For details, see Access control.

Step 6: If the issue persists

If none of the above steps resolves the issue, submit a ticketsubmit a ticketsubmit a ticketsubmit a ticketsubmit a ticket.

Cause

The security group on the node hosting the pod does not allow inbound traffic on TCP port 10250, which the API server uses to retrieve pod logs.

Solution

1. Check whether the pod Status is Running. If not, resolve the pod issue first. For details, see Pod troubleshooting.

2. On the Nodes > Nodes page, find the node where the pod is deployed and click its ECS instance ID. In the ECS console, click Network & Security > Security Groups.

3. Review all security group rules and confirm that inbound access from VPCs to TCP port 10250 is permitted. If not, add the required rule. For details, see Add a security group rule.

4. If the issue persists, submit a ticketsubmit a ticketsubmit a ticketsubmit a ticketsubmit a ticket.

Symptoms

The ACK console displays the following error:

The current account does not have the required RBAC permissions to perform the operation.

Error code: ForbiddenQueryClusterNamespace or APISERVER.403

Cause

The account does not have the required RBAC permissions.

Solution

1. Log on to the ACK consoleACK console using an Alibaba Cloud account or an account with administrator permissions. In the left navigation pane, click Authorizations.

2. On the RAM Users tab, find the RAM user that encountered the error and click Modify Permissions.

3. In the Permission Management panel, click Add Permissions. Select a cluster, a namespace, and a predefined RBAC role, then click Submit.

Symptoms

The ACK console displays the following error:

The current account does not have the required RAM permissions to perform the operation.

Error code: StatusForbidden

Cause

The account does not have the required RAM permissions.

Solution

1. Log on to the RAM console using an Alibaba Cloud account or an account with RAM permissions.

2. Grant the required permissions based on the CS action shown in the error message (for example, cs:DescribeKubernetesVersionMetadata). For details, see Create a custom RAM policy.