Container Service for Kubernetes:Access multi-cluster resources with the Go SDK

Core mechanism

All examples use the same approach to target a member cluster: override the API server host to the Cluster Gateway proxy path for that cluster.

configCopy.Host = fmt.Sprintf(
    "%s/apis/cluster.core.oam.dev/v1alpha1/clustergateways/%s/proxy",
    fleetConfig.Host,
    cluster.Name,
)

This single line routes any standard Kubernetes API call through Cluster Gateway to the named member cluster, using your Fleet credentials.

Full example

The following example, based on OCM v0.14.0, lists all member clusters from the Fleet and prints the UID of the kube-system namespace for each one.

package main

import (
	"context"
	"fmt"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/apimachinery/pkg/util/errors"
	"k8s.io/client-go/kubernetes"
	"k8s.io/client-go/tools/clientcmd"
	ocmcluster "open-cluster-management.io/api/client/cluster/clientset/versioned"
)

// Print the UID of the kube-system namespace for each member cluster.
func main() {
	fleetKubeconfigPath := "<your-fleet-kubeconfig-path>"
	memberClients, err := GenerateMemberClusterKubeClients(fleetKubeconfigPath)
	if err != nil {
		panic(err.Error())
	}

	for cluster, kubeClient := range memberClients {
		ns, err := kubeClient.CoreV1().Namespaces().Get(context.TODO(), "kube-system", metav1.GetOptions{})
		if err != nil {
			fmt.Println(fmt.Sprintf("failed to get kube-system for member cluster %s: %v", cluster, err))
			continue
		}
		fmt.Println(fmt.Sprintf("Cluster %s kube-system namespace UID: %s", cluster, ns.UID))
	}
}

// Generate a kubeClient for each member cluster.
func GenerateMemberClusterKubeClients(fleetKubeconfigPath string) (map[string]kubernetes.Interface, error) {
	fleetConfig, err := clientcmd.BuildConfigFromFlags("", fleetKubeconfigPath)
	if err != nil {
		return nil, err
	}

	// Get all ManagedCluster resources by using the OCM cluster client.
	ocmClusterClient, err := ocmcluster.NewForConfig(fleetConfig)
	if err != nil {
		return nil, fmt.Errorf("failed to create ocm cluster client: %v", err)
	}
	managedClusters, err := ocmClusterClient.ClusterV1().ManagedClusters().List(context.Background(), metav1.ListOptions{})
	if err != nil {
		return nil, err
	}

	var errs []error
	memberClients := make(map[string]kubernetes.Interface)
	for _, cluster := range managedClusters.Items {
		configCopy := *fleetConfig
		configCopy.Host = fmt.Sprintf("%s/apis/cluster.core.oam.dev/v1alpha1/clustergateways/%s/proxy", fleetConfig.Host, cluster.Name)
		kubeClient, err := kubernetes.NewForConfig(&configCopy)
		if err != nil {
			errs = append(errs, fmt.Errorf("failed to create kube client for cluster %s: %v", cluster.Name, err))
			continue
		}
		memberClients[cluster.Name] = kubeClient
	}

	return memberClients, errors.NewAggregate(errs)
}

Replace <your-fleet-kubeconfig-path> with the local path to your Fleet kubeconfig file.

go.mod configuration

The following go.mod configuration includes the dependencies required by this example. The key direct dependencies are k8s.io/client-go v0.29.0 and open-cluster-management.io/api v0.14.0. To confirm which OCM version your Fleet supports, check the ACK One release notes or contact your cluster administrator.

module example.com/ack-one-demo

go 1.21.0


require (
	k8s.io/apimachinery v0.29.0
	k8s.io/client-go v0.29.0
	open-cluster-management.io/api v0.14.0
)

require (
	github.com/davecgh/go-spew v1.1.1 // indirect
	github.com/emicklei/go-restful/v3 v3.11.0 // indirect
	github.com/go-logr/logr v1.4.1 // indirect
	github.com/go-openapi/jsonpointer v0.19.6 // indirect
	github.com/go-openapi/jsonreference v0.20.2 // indirect
	github.com/go-openapi/swag v0.22.3 // indirect
	github.com/gogo/protobuf v1.3.2 // indirect
	github.com/golang/protobuf v1.5.4 // indirect
	github.com/google/gnostic-models v0.6.8 // indirect
	github.com/google/gofuzz v1.2.0 // indirect
	github.com/google/uuid v1.3.0 // indirect
	github.com/imdario/mergo v0.3.6 // indirect
	github.com/josharian/intern v1.0.0 // indirect
	github.com/json-iterator/go v1.1.12 // indirect
	github.com/mailru/easyjson v0.7.7 // indirect
	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
	github.com/modern-go/reflect2 v1.0.2 // indirect
	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
	github.com/spf13/pflag v1.0.5 // indirect
	golang.org/x/net v0.23.0 // indirect
	golang.org/x/oauth2 v0.12.0 // indirect
	golang.org/x/sys v0.18.0 // indirect
	golang.org/x/term v0.18.0 // indirect
	golang.org/x/text v0.14.0 // indirect
	golang.org/x/time v0.3.0 // indirect
	google.golang.org/appengine v1.6.7 // indirect
	google.golang.org/protobuf v1.33.0 // indirect
	gopkg.in/inf.v0 v0.9.1 // indirect
	gopkg.in/yaml.v2 v2.4.0 // indirect
	gopkg.in/yaml.v3 v3.0.1 // indirect
	k8s.io/api v0.30.0 // indirect
	k8s.io/klog/v2 v2.120.1 // indirect
	k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340 // indirect
	k8s.io/utils v0.0.0-20230726121419-3b25d923346b // indirect
	sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
	sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
	sigs.k8s.io/yaml v1.4.0 // indirect
)

Insufficient permissions error

If your RAM user lacks the required RBAC permissions on a member cluster, Cluster Gateway returns the following error:

stream error: stream ID 3; INTERNAL_ERROR; received from peer

This error means Cluster Gateway received the request but the member cluster's kube-apiserver rejected it due to insufficient permissions. To resolve this:

1. Log on to the ACK One console as the Alibaba Cloud account (root account).

2. In the left-side navigation pane, choose Fleet > Permissions.

3. On the Associated Clusters tab, verify that the RAM user has the correct RBAC role binding for the target member cluster.