MseIngressConfig is a Custom Resource Definition (CRD) provided by the MSE Ingress Controller to manage the lifecycle of MSE cloud-native gateway instances and configure Ingress listener options and global settings. This topic describes common uses of MseIngressConfig, including how to create, reuse, and delete MSE cloud-native gateway instances, configure global instance-level IP blacklists and whitelists for access control, enable Simple Log Service (SLS) logs, and enable OpenTelemetry tracing.
Background information
The MSE Ingress Controller listens for MseIngressConfig resources created in a cluster. It dynamically maintains the lifecycle of the corresponding cloud-native gateway instance and its association with an ACK managed cluster, ACK Serverless cluster, or ACS cluster.
By associating with the API Server of an ACK managed cluster, ACK Serverless cluster, or ACS cluster, the control plane of the MSE cloud-native gateway can detect changes to Ingress resources and dynamically update the routing rules of the gateway. When a request is received, the MSE cloud-native gateway matches the Ingress forwarding rules and forwards the request to the pod that corresponds to the backend Service.
The following list describes the relationships between Service, Ingress, IngressClass, MseIngressConfig, and the MSE Ingress Controller in Kubernetes:
Service: An abstraction of a real backend service. A Service can represent multiple identical backend services.
Ingress: A reverse proxy rule that specifies which Service an HTTP or HTTPS request should be forwarded to. For example, requests can be forwarded to different Services based on the host and URL path in the request.
IngressClass: A description of an Ingress processor that is used to declare an Ingress processor implementation in a Kubernetes cluster. Ingress resources associated with this IngressClass are parsed by the declared Ingress processor. You must also associate an MseIngressConfig (MSE cloud-native gateway) through the Parameter field of the IngressClass to apply the traffic management rules defined in the parsed Ingress resource.
MseIngressConfig: A CRD provided by the MSE Ingress Controller that describes the basic information of a cloud-native gateway instance.
MSE Ingress Controller: The control plane that manages MSE cloud-native gateway instances and configurations. It is not a network data plane. It listens for MseIngressConfig resources in the cluster and coordinates MSE cloud-native gateway instances to apply the traffic management rules defined in Ingress resources.
The following figure shows how the MSE Ingress Controller works.
MseIngressConfig description
Configuration description
MseIngressConfig is a CRD provided by the MSE Ingress Controller to manage the lifecycle and global configurations of an MSE cloud-native gateway instance.
One MseIngressConfig corresponds to one MSE cloud-native gateway instance. To use multiple MSE cloud-native gateway instances, you must create multiple MseIngressConfig configurations. Except for the reuse scenario, deleting an MseIngressConfig configuration also deletes the corresponding MSE cloud-native gateway instance through a cascade delete.
The following sample code shows a complete MseIngressConfig configuration.
apiVersion: mse.alibabacloud.com/v1alpha1
kind: MseIngressConfig
metadata:
name: test
spec:
name: mse-ingress
common:
pay:
payType: POSTPAY
instance:
spec: 4c8g
replicas: 3
network:
vSwitches:
- "vsw-1"
- "vsw-2"
publicSLBSpec: slb.s2.small
securityGroupType: normal
global:
downstreamIdleTimeout: 15
enableHttp2: true
enableXffTrustedCidrs: |-
{"enable":true,"ipListContent":"x.x.x.x/24
x.x.x.x"}
preserveHeaderFormat: true
upstreamIdleTimeout: 30
tls:
enableHardwareAcceleration: true
ipAccessControl:
whitelist:
- 1.1.XX.XX
- 2.2.XX.XX
monitor:
logging:
sls:
reuseProject: "xxx" # An empty value indicates that the default project is used.
compatibleNginx: false
tracing:
openTelemetry:
sampleRate: "100"
ingress:
local:
ingressClass: mse
watchNamespace: "" # An empty value indicates all namespaces.Parameter | Description | Optional | Default value |
name | The name of the gateway. | Optional | mse-ingress |
common.pay.payType | The billing method. Currently, only pay-as-you-go is supported. | Optional If not configured, the system defaults to pay-as-you-go. | POSTPAY |
common.instance.spec | The gateway instance type. Valid values:
| Optional | 4c8g |
common.instance.replicas | The number of gateway instance replicas. The valid range for the number of replicas is 0 to 30. | Optional | 3 |
common.network.vSwitches | The primary and standby vSwitches. The primary vSwitch is listed first, followed by the standby. Specify at least one and at most two vSwitches. If you do not specify any, the vSwitches are obtained from the node where the Controller pod is located. | Optional | None |
common.network.publicSLBSpec | The specification of the Internet-facing SLB instance purchased by the gateway. Valid values:
| Optional | slb.s2.small |
common.network.privateSLBSpec | The specification of the internal-facing SLB instance purchased by the gateway. Valid values:
| Optional | slb.s2.small |
common.securityGroupType | The type of security group. Valid values:
| Optional | normal |
global.tls.enableHardwareAcceleration | The switch for TLS hardware acceleration. This can significantly improve the performance of HTTPS traffic. | Optional | Enabled |
global.ipAccessControl.whitelist | The global IP access whitelist. | Optional | Not configured |
global.ipAccessControl.blacklist | The global IP access blacklist. | Optional | Not configured |
global.downstreamIdleTimeout | The idle timeout for connections between downstream clients and the gateway. If there is no read or write activity within this period, the gateway actively closes the connection. Unit: seconds. | Optional | 15 |
global.enableHttp2 | Specifies whether to enable HTTP/2 protocol support for the gateway. | Optional | false |
global.enableXffTrustedCidrs | Configures trusted intermediate proxy IP address ranges in CIDR format, such as Alibaba Cloud WAF or CDN. This configuration allows the gateway to skip the IPs of these trusted proxies when parsing the | Optional | Not configured |
global.preserveHeaderFormat | Specifies whether to preserve the original case format of the client request header. | Optional | false |
global.upstreamIdleTimeout | The idle timeout for connections between the gateway and upstream backend services. If there is no activity within this period, the gateway actively closes it. Unit: seconds. | Optional | 30 |
monitor.logging.sls | The switch for the SLS log service. To enable this, grant the Controller SLS-related permissions. | Optional | Disabled |
monitor.logging.sls.reuseProject | The destination Project for delivering access logs in the SLS log service. Valid values:
| Optional | Empty |
monitor.tracing.xTrace.sampleRate | The sample rate for OpenTelemetry tracing. | Optional | 0 |
monitor.tracing.openTelemetry | The switch for OpenTelemetry tracing. | Optional | Disabled by default. This is mutually exclusive with OpenTelemetry. |
monitor.tracing.openTelemetry.sampleRate | The sample rate for OpenTelemetry tracing. | Optional | 100 |
ingress.local.ingressClass | The cloud-native gateway listens for Ingress resources under a specific IngressClass in the cluster. Valid values:
Note This configuration has a lower priority than associating an MseIngressConfig through an IngressClass resource. | Optional | Not configured |
ingress.local.watchNamespace | The namespaces in the cluster from which the cloud-native gateway listens for Ingress resources. Valid values:
| Optional | Empty |
Status description
After you create an MseIngressConfig resource, you can view its status by running the kubectl get mseingressconfig command. The MseIngressConfig status changes in the following order: Pending > Running > Listening. The statuses are described as follows:
Pending: The cloud-native gateway is being created. This process takes about 3 minutes.
Running: The cloud-native gateway is created and running.
Listening: The cloud-native gateway is running and listening for Ingress resources in the cluster.
Failed: The cloud-native gateway is in an invalid state. Check the Message in the Status field for the cause of the failure.
Tag description
MSE cloud-native gateway instances that are created or reused through MseIngressConfig are automatically assigned resource tags. You can view the tags for a gateway in the basic information section of the MSE console.
Do not edit the tags in the following table in the MSE cloud-native gateway console. Editing these tags may cause issues with your gateway instance.
Tag name | Description |
ack.aliyun.com | The ingress traffic of the Container Service for Kubernetes (ACK) cluster managed by the MSE cloud-native gateway. |
ingress.k8s.alibaba/MseIngressConfig | Associates the MSE cloud-native gateway with an MseIngressConfig. |
kubernetes.reused.by.user | Indicates whether the MSE cloud-native gateway is being reused. If it is, the MSE cloud-native gateway instance is not deleted when the associated MseIngressConfig is deleted. |
Create an MSE cloud-native gateway instance
Configure the MseIngressConfig.
Use the following sample code to create an MSE cloud-native gateway instance named mse-ingress with three replicas and an instance type of 2-core 4 GB. Modify other MseIngressConfig settings as needed.
apiVersion: mse.alibabacloud.com/v1alpha1 kind: MseIngressConfig metadata: name: test spec: name: mse-ingress common: instance: spec: 2c4g replicas: 3Create an IngressClass resource in the ACK cluster and associate it with the MseIngressConfig.
Create an IngressClass resource in the ACK cluster and associate it with the MseIngressConfig from the previous step using spec.parameters. This declares an Ingress processor in the ACK cluster. As a result, Ingress resources in the cluster that are associated with this IngressClass are processed by the MSE cloud-native gateway instance that is associated with the MseIngressConfig.
apiVersion: networking.k8s.io/v1 kind: IngressClass metadata: name: mse spec: controller: mse.alibabacloud.com/ingress parameters: apiGroup: mse.alibabacloud.com kind: MseIngressConfig name: test
Reuse an existing MSE cloud-native gateway instance
If you want to reuse an existing MSE cloud-native gateway instance, specify its unique ID in the `gw-xxx` format in spec.id when you create the MseIngressConfig. You can use the spec.override parameter to control whether to overwrite the configurations of the reused MSE cloud-native gateway instance with the configurations from the MseIngressConfig.
The following sample code reuses an existing MSE cloud-native gateway instance using an MseIngressConfig. It associates the gateway instance with a container cluster and configures it to listen for Ingress resources in that cluster that have `ingressClass` set to `mse`.
In the following configuration,
spec.overrideis set tofalse. This means that the existing Ingress listener options and global configurations of the MSE cloud-native gateway are not overwritten. If the existing gateway instance is not associated with this container cluster, the instance is automatically associated with the cluster. Its Ingress listener options are then set to listen for Ingress resources where `ingressClass` matches the value ofspec.ingress.local.ingressClass, which is `mse` in this case. If the existing gateway instance is already associated with the container cluster, its original Ingress listener options are not overwritten.If you want to overwrite the existing Ingress listener options and global configurations of the MSE cloud-native gateway, you must ensure that the gateway parameter configurations in the MseIngressConfig are correct before you set
spec.overrideto `true`. These configurations include Ingress listener options, hardware acceleration, global blacklists and whitelists, and observability. If these configurations are not specified in the MseIngressConfig, settingspec.overrideto `true` overwrites the gateway's original parameter configurations with empty values, which may disrupt your traffic.
apiVersion: mse.alibabacloud.com/v1alpha1
kind: MseIngressConfig
metadata:
name: reuse
spec:
id: gw-xxxx
override: false
ingress:
local:
ingressClass: mseParameter | Description |
spec.id | The ID of the target MSE cloud-native gateway instance to reuse, starting with gw-. |
spec.override | Specifies whether to overwrite the configurations of the reused MSE cloud-native gateway instance based on the MseIngressConfig.
|
Delete an MSE cloud-native gateway instance
One MseIngressConfig corresponds to one MSE cloud-native gateway instance. Except in the reuse scenario, deleting an MseIngressConfig configuration also deletes the corresponding MSE cloud-native gateway instance through a cascade delete.
The deletion policy is shown in the following table.
Gateway billing method | Automatically created by MSE Ingress Controller | Reused gateway purchased in the console |
Pay-as-you-go | Deleting the MseIngressConfig automatically deletes the gateway instance. | Deleting the MseIngressConfig retains the gateway instance. |
Subscription | Not applicable. | Deleting the MseIngressConfig retains the gateway instance. |
Run the following command to delete the MseIngressConfig configuration.
kubectl delete mseingressconfig your-config-nameConfigure global instance-level IP access control
Configure an IP whitelist
You can configure a global instance-level IP whitelist to allow only specified source IP addresses to access the MSE Ingress.
The following sample code configures the MSE Ingress gateway instance to allow access only from the source IP address 1.1.XX.XX or the IP range 2.0.XX.XX.
apiVersion: mse.alibabacloud.com/v1alpha1
kind: MseIngressConfig
metadata:
name: test
spec:
...
global:
ipAccessControl:
whitelist:
- 1.1.XX.XX
- 2.0.XX.XX/8
...Configure an IP blacklist
You can configure a global instance-level IP blacklist to deny access from specified source IP addresses to the MSE Ingress.
The following sample code configures the MSE Ingress gateway instance to deny access from the source IP address 1.1.XX.XX or the IP range 2.0.XX.XX.
apiVersion: mse.alibabacloud.com/v1alpha1
kind: MseIngressConfig
metadata:
name: test
spec:
...
global:
ipAccessControl:
blacklist:
- 1.1.XX.XX
- 2.0.XX.XX/8
...Enable SLS logs
Before you enable SLS logs, you must grant the necessary SLS permissions to the MSE Ingress Controller.
If you use a managed or dedicated ACK cluster, see Grant permissions to MSE Ingress Controller in a dedicated ACK cluster.
If your cluster is an ACK Serverless cluster, see Grant permissions to MSE Ingress Controller.
If your cluster is an ACS cluster, see Grant permissions to MSE Ingress Controller in an ACS cluster.
The following sample code configures the MSE Ingress gateway to deliver access logs to a Project named demo.
apiVersion: mse.alibabacloud.com/v1alpha1
kind: MseIngressConfig
metadata:
name: test
spec:
...
monitor:
logging:
sls:
# An empty value indicates that the default project is used.
reuseProject: "demo"
...You can configure the reuseProject field to enable SLS logs. If you want to use the default Project, leave the reuseProject field empty.
Enable OpenTelemetry tracing
You can configure the Alibaba Cloud OpenTelemetry tracing service for the gateway to build an end-to-end tracing and monitoring system. This helps you quickly diagnose and locate online issues.
The following sample code configures OpenTelemetry tracing for the MSE Ingress gateway with a sample rate of 100%.
apiVersion: mse.alibabacloud.com/v1alpha1
kind: MseIngressConfig
metadata:
name: test
spec:
...
monitor:
tracing:
openTelemetry:
sampleRate: "100"
...More configurations
The cloud-native gateway instance that is used by the MSE Ingress supports additional configurations.
To configure these settings, log on to the MSE Gateway Management Console. On the page, find the gateway instance that is used by the MSE Ingress Controller. The instance ID is in the format `mse_ingresspost-*****`. Click the instance ID, and then in the navigation pane on the left, click Parameter Configuration.
Configuration item | Type | Description |
EnableGenerateRequestId | Bool | Applies to the request scope. When enabled, it generates a requestId in the request header to trace the request. |
EnableGzip | Bool | Applies to the request and response stages. Enabling gzip compresses requests and responses, which reduces gateway traffic but increases gateway CPU consumption. |
EnableSlashMerge | Bool | Applies to the request stage. Specifies whether to merge extra slashes ('/') in the request. When enabled, extra slashes in `www.a.com//b` are merged, making it identical to `www.a.com/b`. |
DownstreamConnectionBufferLimits | Int Range: [0, 2147483647] | Applies to gateway connections. The buffer size for a single connection. This setting affects throughput and the gateway's memory usage. |
XffTrustedNum | Int Range: [0, 10] | Applies to the request stage. The number of trusted proxies in front of the gateway. This affects whether the gateway uses client-generated headers such as x-forwarded-for and x-request-id. |
DownstreamHttp2MaxConcurrentStream | Int Range: [0, 2147483647] (byte) | Applies to the request stage. The maximum number of concurrent streams on a single connection when the client uses HTTP/2. |
InitialStreamWindowSize | Int Range: [65535, 2147483647] (byte) | Applies to the request stage. The initial stream window size negotiated between the gateway and the client when using HTTP/2. |
InitialConnectionWindowSize | Int Range: [65535, 2147483647] (byte) | Applies to the request stage. The connection-level initial window size when the gateway and client use HTTP/2. |
EnableHttp3 | Bool | Specifies whether to support the HTTP/3 protocol. (HTTP/3 is not compatible with the hardware acceleration feature. Disable hardware acceleration before enabling HTTP/3 support.) |
PathWithEscapedSlashes | String Range: [KEEP_UNCHANGED, REJECT_REQUEST, UNESCAPE_AND_REDIRECT, UNESCAPE_AND_FORWARD] | Determines the action for requests whose URI path contains escaped characters such as %2F, %2f, %5C, or %5c. The default is KEEP_UNCHANGED (keep as is). |
ZipAlgorithm | List<String> Range: [brotli, gzip] | The algorithm to use when compression is enabled. You can choose brotli or gzip. If EnableGzip is not enabled at this time, it will be enabled automatically. |
EnableProxyProtocol | Bool | Specifies whether to enable the Proxy Protocol. If the gateway's ingress traffic uses NLB, the client's originating IP address cannot be obtained unless this is enabled. Enabling it does not affect non-Proxy Protocol requests. |
EnableCustomAuthConfigPush | Bool | Applies to scenarios using a self-built authorization service. When enabled, changes to authorization rules do not cause connection interruptions, making it suitable for WebSocket and online business scenarios. |
KeepaliveHeaderTimeout | Int Range: [0, 600] (s) | Used to generate a Keep-Alive response header to inform the client of the connection's keepalive time. For example, a value of 10 returns the response header: `keep-alive: timeout=10`. A value of 0 means this response header is not returned. |
WebsocketTermGracePeriod | Int Range: [20, 900] (s) | The keepalive time maintained for WebSocket connections when connections need to be closed, such as during gateway instance upgrades or restarts. |
EnableGzipHardwareAccelerate | Bool | Performs Gzip compression based on dedicated hardware. When enabled, it compresses requests and responses, significantly reducing gateway traffic with higher performance and lower CPU consumption than software Gzip. This cannot be enabled in unsupported regions or if Gzip hardware acceleration was not specified at the time of purchase. When enabled, EnableGzip and ZipAlgorithm will not take effect. |
EnableK8sSourceWorkloadFilter | Bool | Filters Ingress, Service, and Pod resources based on specified labels when adding a service source. Supports inverse filtering. |