Annotations supported by APIG Ingress - Container Service for Kubernetes

APIG Ingress supports core and common Nginx Ingress annotations. This enables a seamless migration from Nginx Ingress to an APIG Ingress gateway. APIG Ingress also introduces additional annotations to provide traffic governance capabilities that Nginx Ingress annotations do not support. This topic describes the annotations that APIG Ingress currently supports.

Annotation feature overview

Standard Kubernetes Ingress provides only basic routing for Layer 7 HTTP traffic and TLS encryption. Ingress controllers typically use annotations to enhance the traffic governance and security protection capabilities of an Ingress.

Ingress annotation support status

Nginx Ingress annotations

To help Nginx Ingress users seamlessly migrate to APIG Ingress, APIG Ingress provides extensive support for Nginx Ingress annotations. The following table describes the support status.

Nginx Ingress annotation	Total supported	Notes
Supported annotations	51	Covers 90% of user scenarios.
Annotations that do not affect features	15	No configuration required.
Temporarily unsupported annotations	48	Support is in progress. These are used in few scenarios.
Unsupported annotations	5	Mainly involves Nginx-specific code snippets.

Note

Higress and Nginx have different implementations, which results in some differences.

Configurations that use NGINX variables and code snippets in Nginx Ingress annotations are not compatible.
Nginx Ingress uses the nginx.ingress.kubernetes.io/proxy-body-size annotation to configure the size limit of a client request body. If a request body exceeds the specified size, Nginx returns an error. The Higress cloud-native gateway uses chunked streaming and does not require you to specify the request body size in advance. For large file transfers, you can adjust the `DownstreamConnectionBufferLimits` parameter in the parameter settings of the gateway instance.

APIG Ingress annotations

APIG Ingress introduces additional annotations to provide administration capabilities that are not available in Nginx Ingress annotations. The following table describes the current support status.

APIG Ingress annotation	Total supported	Notes
Extended annotations	40	Enhances traffic governance and security protection features beyond Nginx.

Scope definitions

Ingress: An Ingress-scoped annotation applies only to the routing rules that are defined in the current Ingress.
Domain name: A domain-scoped annotation applies to the hosts that are specified in the current Ingress. This scope also affects other Ingresses where the same host is specified.
Service: A service-scoped annotation applies to the services that are specified in the current Ingress. This scope also affects other Ingresses where the same service is specified.

Annotation prefixes

APIG Ingress supports all Nginx Ingress annotations. For example, nginx.ingress.kubernetes.io/xxx has the same effect as higress.ingress.kubernetes.io/xxx. You can use either the nginx or higress prefix. However, you cannot use the nginx prefix for annotations that are unique to APIG Ingress.

Annotation support summary

This topic covers two main modules: traffic governance and security protection.

Traffic governance

Canary release

Annotation	Scope	Support level	Description
nginx.ingress.kubernetes.io/canary	Ingress	Compatible	Enables or disables a canary release.
nginx.ingress.kubernetes.io/canary-by-header	Ingress	Compatible	Splits traffic based on a `Request Header Key`.
nginx.ingress.kubernetes.io/canary-by-header-value	Ingress	Compatible	Splits traffic based on a `Request Header Value`. The value is an exact match.
nginx.ingress.kubernetes.io/canary-by-header-pattern	Ingress	Compatible	Splits traffic based on a `Request Header Value`. The value is a regular expression match.
higress.ingress.kubernetes.io/canary-by-query	Ingress	Higress extension	Splits traffic based on a `URL Query Parameter`.
higress.ingress.kubernetes.io/canary-by-query-value	Ingress	Higress extension	Splits traffic based on a `URL Query Parameter`. The value is an exact match.
higress.ingress.kubernetes.io/canary-by-query-pattern	Ingress	Higress extension	Splits traffic based on a `URL Query Parameter`. The value is a regular expression match.
nginx.ingress.kubernetes.io/canary-by-cookie	Ingress	Compatible	Splits traffic based on a `Request Cookie Key`.
higress.ingress.kubernetes.io/canary-by-cookie-value	Ingress	Higress extension Requires gateway version 1.2.30	Splits traffic based on a `Request Cookie Value`. The value is an exact match.
nginx.ingress.kubernetes.io/canary-weight	Ingress	Compatible	Splits traffic based on weight.
nginx.ingress.kubernetes.io/canary-weight-total	Ingress	Compatible	The sum of the weights.

Multiple services

Annotation

Scope

Support level

Description

higress.ingress.kubernetes.io/destination

Ingress

Higress extension

Configures weight-based service distribution for a route.

The syntax is {weight}% {serviceName}.{serviceNamespace}.svc.cluster.local:{port}.

Note

After you configure this annotation, the destination service for all routing rules in the Ingress is changed to the service specified by this annotation.
If the configuration syntax of this annotation is invalid, the annotation is ignored. The destination service for all routing rules in the Ingress is not changed.

Syntax example:

annotations:
  # 60% of traffic is routed to the foo service, and 40% is routed to the bar service.
  higress.ingress.kubernetes.io/destination: |
    60% foo.default.svc.cluster.local:8080
    40% bar.default.svc.cluster.local:9090

Service subset

Annotation

Scope

Support level

Description

higress.ingress.kubernetes.io/service-subset

Ingress

Higress extension
Requires gateway version 1.2.25

Applies to scenarios where one service manages multiple deployments. Specifies that the route defined by the Ingress is forwarded to a subset of the pods under this service.

If the subset-labels annotation is not configured, the value of this annotation has the following meanings:
- If the value is "" or base, requests are forwarded to the pod collection that has the label `opensergo.io/canary: ""` or does not have any label key with the prefix `opensergo.io/canary`. This includes pods with an empty label and pods without a label.
- If the value is set to another value, requests are forwarded to the pod collection that has the label `opensergo.io/canary-{value}: "{value}"`. For example, if the value is `gray`, requests are forwarded to the pod collection that has the label `opensergo.io/canary-gray: gray`.
If the subset-labels annotation is configured, requests are forwarded only to the pod collection that has the key-value pair defined in the subset-labels annotation.

Note

If the service does not have a pod with the specified label, traffic is automatically routed to all pods under the service for disaster recovery.

higress.ingress.kubernetes.io/subset-labels

Ingress

Higress extension
Requires gateway version 1.2.25

Optional. This annotation works with service-subset to explicitly configure which labels belong to the same subset of pods.

Fallback (disaster recovery)

Annotation	Scope	Support level	Description
nginx.ingress.kubernetes.io/default-backend	Ingress	Compatible	The disaster recovery service. When the service defined in the Ingress has no active nodes, requests are automatically forwarded to this disaster recovery service.
nginx.ingress.kubernetes.io/custom-http-errors	Ingress	Compatible	This annotation works with `default-backend`. When the backend service returns a specified HTTP response code, the original request is forwarded again to the disaster recovery service. Important When a request is forwarded to the disaster recovery service, the request path is rewritten to /, which is consistent with the behavior of `ingress-nginx`.

Regex match

Annotation	Scope	Support level	Description
nginx.ingress.kubernetes.io/use-regex	Ingress	Compatible	Regex match. Indicates that the path defined in the Ingress is matched using a regular expression. The regular expression uses the RE2 syntax.

Rewrite

Annotation	Scope	Support level	Description
nginx.ingress.kubernetes.io/rewrite-target	Ingress	Compatible	Rewrites the original path defined in the Ingress to a specified target. Supports `Group Capture`.
nginx.ingress.kubernetes.io/upstream-vhost	Ingress	Compatible	When a request that matches the route defined in the Ingress is forwarded to the backend service, the Host header is changed to the specified value.

Redirection

Note

The feature sets of Nginx and Nginx Ingress are different. Nginx provides a broader range of features. The official Nginx Ingress documentation does not mention support for redirection that uses NGINX variables. Although some versions of Nginx Ingress may support the configuration of NGINX variables, using them can cause compatibility issues because this feature is not officially documented. We recommend that you do not use NGINX variables in Nginx Ingress.

Annotation	Scope	Support level	Description
nginx.ingress.kubernetes.io/ssl-redirect	Ingress	Compatible	Redirects HTTP to HTTPS.
nginx.ingress.kubernetes.io/force-ssl-redirect	Ingress	Compatible	Redirects HTTP to HTTPS.
nginx.ingress.kubernetes.io/permanent-redirect	Ingress	Compatible	Permanent redirect.
nginx.ingress.kubernetes.io/permanent-redirect-code	Ingress	Compatible	The status code for a permanent redirect.
nginx.ingress.kubernetes.io/temporal-redirect	Ingress	Compatible	Temporary redirect.
nginx.ingress.kubernetes.io/app-root	Ingress	Compatible	Changes the application root path. Requests to / are redirected to the new path.

Cross-domain

Annotation	Scope	Support level	Description
nginx.ingress.kubernetes.io/enable-cors	Ingress	Compatible	Enables or disables cross-domain access.
nginx.ingress.kubernetes.io/cors-allow-origin	Ingress	Compatible	The allowed third-party sites.
nginx.ingress.kubernetes.io/cors-allow-methods	Ingress	Compatible	The allowed request methods, such as GET, POST, and PUT.
nginx.ingress.kubernetes.io/cors-allow-headers	Ingress	Compatible	The allowed request headers.
nginx.ingress.kubernetes.io/cors-expose-headers	Ingress	Compatible	The allowed response headers that can be exposed to the browser.
nginx.ingress.kubernetes.io/cors-allow-credentials	Ingress	Compatible	Specifies whether to allow credentials to be carried.
nginx.ingress.kubernetes.io/cors-max-age	Ingress	Compatible	The maximum cache duration for preflight results.

Header control

Note

Header control annotations defined on a standard route do not apply to a canary route. The header control annotations defined on standard and canary routes are independent and take effect separately. This lets you set different header operation policies for standard and canary routes.

Annotation	Scope	Support level	Description
higress.ingress.kubernetes.io/request-header-control-add	Ingress	Higress extension	Adds a specified header when a request is forwarded to the backend service. If the header exists, its value is appended to the original value. The syntax is as follows: Single header: Key Value. Multiple headers: Use the YAML special character `\|` to place each key-value pair on a separate line.
higress.ingress.kubernetes.io/request-header-control-update	Ingress	Higress extension	Changes a specified header when a request is forwarded to the backend service. If the header exists, its value overwrites the original value. The syntax is as follows: Single header: Key Value. Multiple headers: Use the YAML special character `\|` to place each key-value pair on a separate line.
higress.ingress.kubernetes.io/request-header-control-remove	Ingress	Higress extension	Deletes a specified header when a request is forwarded to the backend service. The syntax is as follows: Single header: Key. Multiple headers: Separate them with commas (,).
higress.ingress.kubernetes.io/response-header-control-add	Ingress	Higress extension	Adds a specified header before forwarding a response from the backend service to the client. If the header exists, its value is appended to the original value. The syntax is as follows: Single header: `Key Value`. Multiple headers: Use the YAML special character `\|` to place each key-value pair on a separate line.
higress.ingress.kubernetes.io/response-header-control-update	Ingress	Higress extension	Changes a specified header before forwarding a response from the backend service to the client. If the header exists, its value overwrites the original value. The syntax is as follows: Single header: `Key Value`. Multiple headers: Use the YAML special character `\|` to place each key-value pair on a separate line.
higress.ingress.kubernetes.io/response-header-control-remove	Ingress	Higress extension	Deletes a specified header before forwarding a response from the backend service to the client. The syntax is as follows: Single header: Key. Multiple headers: Separate them with commas (,).

Timeout

Annotation	Scope	Support level	Description
higress.ingress.kubernetes.io/timeout	Ingress	Higress extension	The request timeout period in seconds. By default, no timeout period is configured. Note The timeout setting applies to the application layer, not the transport-layer TCP.

Retry

Annotation	Scope	Support level	Description
nginx.ingress.kubernetes.io/proxy-next-upstream-tries	Ingress	Compatible	The maximum number of retries for a request. The default is 3.
nginx.ingress.kubernetes.io/proxy-next-upstream-timeout	Ingress	Compatible	The timeout period for request retries in seconds. By default, no timeout period is configured.
nginx.ingress.kubernetes.io/proxy-next-upstream	Ingress	Compatible	The conditions for retrying a request. For more information, see Nginx retry mechanism.

Traffic mirroring

Annotation

Scope

Support level

Description

higress.ingress.kubernetes.io/mirror-target-service

Ingress

Higress extension

Copies and forwards traffic to a specified mirror service. The service format is namespace/name:port.

namespace: The namespace where the Kubernetes service resides. This is optional. The default is the namespace of the Ingress.
name: The name of the Kubernetes service. This is required.
port: The port of the Kubernetes service to which traffic is forwarded. This is optional. The default is the first port.

higress.ingress.kubernetes.io/mirror-percentage

Ingress

Higress extension
Requires gateway version 1.2.32

The percentage of traffic to copy. The value can range from 0 to 100. The default is 100.

Domain name alias

Annotation	Scope	Support level	Description
nginx.ingress.kubernetes.io/server-alias	Domain name	Partially compatible (only exact-match and wildcard domain names are supported) Requires gateway version 1.2.30	Defines an alias for the domain name that appears in the Ingress spec. The domain name alias shares the TLS, routing, and traffic governance configurations of the source domain name.

Single-instance throttling (to be deprecated)

Annotation	Scope	Support level	Description
higress.ingress.kubernetes.io/route-limit-rpm	Ingress	Higress extension	The maximum number of requests per minute for the route defined by this Ingress on each gateway instance. The maximum instantaneous number of requests is this value multiplied by `limit-burst-multiplier`. When throttling is triggered, the response body is `local_rate_limited`. The response status code is as follows: For gateway versions earlier than 1.2.23: The status code is 503. For gateway versions 1.2.23 and later: The status code is 429.
higress.ingress.kubernetes.io/route-limit-rps	Ingress	Higress extension	The maximum number of requests per second for the route defined by this Ingress on each gateway instance. The maximum instantaneous number of requests is this value multiplied by `limit-burst-multiplier`. When throttling is triggered, the response body is `local_rate_limited`. The response status code is as follows: For gateway versions earlier than 1.2.23: The status code is 503. For gateway versions 1.2.23 and later: The status code is 429.
higress.ingress.kubernetes.io/route-limit-burst-multiplier	Ingress	Higress extension	The factor for the maximum instantaneous number of requests. The default is 5.

Global throttling (recommended)

Annotation	Scope	Support level	Description
higress.ingress.kubernetes.io/rate-limit	Ingress	Higress extension Requires gateway version 1.2.25	The global throttling for the route defined by this Ingress on the gateway instance, which is the maximum number of requests per second.
higress.ingress.kubernetes.io/rate-limit-fallback-custom-response-code	Ingress	Higress extension Requires gateway version 1.2.25	The response status code when throttling is triggered for the route defined by this Ingress. The default is 429. Note This annotation is mutually exclusive with rate-limit-fallback-redirect-url. You can choose either a custom response or a redirection.
higress.ingress.kubernetes.io/rate-limit-fallback-custom-response-body-type	Ingress	Higress extension Requires gateway version 1.2.25	The format of the response body when throttling is triggered for the route defined by this Ingress. The default is text. If set to text: The Content-Type of the response is text/plain; charset=UTF-8. If set to JSON: The Content-Type of the response is application/json; charset=UTF-8.
higress.ingress.kubernetes.io/rate-limit-fallback-custom-response-body	Ingress	Higress extension Requires gateway version 1.2.25	The response body when throttling is triggered for the route defined by this Ingress. The default is `sentinel rate limited`.
higress.ingress.kubernetes.io/rate-limit-fallback-redirect-url	Ingress	Higress extension Requires gateway version 1.2.25	The redirection address when throttling is triggered for the route defined by this Ingress. Note This annotation is mutually exclusive with rate-limit-fallback-custom-response-code. You can choose either a redirection or a custom response.

Global concurrency control

Annotation	Scope	Support level	Description
higress.ingress.kubernetes.io/concurrency-limit	Ingress	Higress extension Requires gateway version 1.2.25	The global concurrency control for the route defined by this Ingress on the gateway instance, which is the maximum number of instantaneous requests being processed.
higress.ingress.kubernetes.io/concurrency-limit-fallback-custom-response-code	Ingress	Higress extension Requires gateway version 1.2.25	The response status code when concurrency control is triggered for the route defined by this Ingress. The default is 429. Note This annotation is mutually exclusive with concurrency-limit-fallback-redirect-url. You can choose either a custom response or a redirection.
higress.ingress.kubernetes.io/concurrency-limit-fallback-custom-response-body-type	Ingress	Higress extension Requires gateway version 1.2.25	The format of the response body when concurrency control is triggered for the route defined by this Ingress. The default is text. If set to text: The Content-Type of the response is text/plain; charset=UTF-8. If set to JSON: The Content-Type of the response is application/json; charset=UTF-8.
higress.ingress.kubernetes.io/concurrency-limit-fallback-custom-response-body	Ingress	Higress extension Requires gateway version 1.2.25	The response body when concurrency control is triggered for the route defined by this Ingress. The default is `sentinel rate limited`.
higress.ingress.kubernetes.io/concurrency-limit-fallback-redirect-url	Ingress	Higress extension Requires gateway version 1.2.25	The redirection address when concurrency control is triggered for the route defined by this Ingress. Note This annotation is mutually exclusive with concurrency-limit-fallback-custom-response-code. You can choose either a redirection or a custom response.

Protocol used by the backend service

Annotation

Scope

Support level

Description

nginx.ingress.kubernetes.io/backend-protocol

Service

Partially compatible. AJP and FCGI are not supported.

Specifies the protocol used by the backend service. The default is HTTP. Supported values are:

HTTP
HTTP2
HTTPS
gRPC
gRPCS

Load balancing

Annotation

Scope

Support level

Description

nginx.ingress.kubernetes.io/load-balance

Service

Partially compatible. The ewma algorithm is not supported. If set to the EWMA algorithm, it falls back to the round_robin algorithm.

The standard load balancing algorithm for the backend service. The default is round_robin. Valid values are:

round_robin: Round-robin load balancing.
least_conn: Least-connections load balancing.
random: Random load balancing.

nginx.ingress.kubernetes.io/upstream-hash-by

Service

Partially compatible. The combined use of NGINX variables and constants is not supported.

The consistent hashing load balancing algorithm. APIG Ingress supports the following forms:

APIG Ingress supports the configuration of some NGINX variables:
- $request_uri: The URI of the request (including path parameters) is used as the hash key.
- $host: The request host is used as the hash key.
- $remote_addr: The client IP address of the request is used as the hash key.
Consistent hashing based on a request header. Configure it as $http_headerName.
Consistent hashing based on a request path parameter. Configure it as $arg_varName.

Service prefetch (graceful start)

Annotation	Scope	Support level	Description
higress.ingress.kubernetes.io/warmup	Service	Higress extension	The service ramp-up period in seconds. This is disabled by default. Important Service prefetch depends on the selected load balancing algorithm. Currently, only round_robin and least_conn are supported.

Cookie affinity

Annotation	Function	Support level	Description
nginx.ingress.kubernetes.io/affinity	Service	Compatible	The affinity type. Currently, only Cookie is supported. The default is `cookie`.
nginx.ingress.kubernetes.io/affinity-mode	Service	Partially compatible. The `persistent` mode is not supported.	The affinity mode. APIG Ingress currently supports only the Balanced mode. The default is the `balanced` mode.
nginx.ingress.kubernetes.io/session-cookie-name	Service	Compatible	Configures the value of a specified cookie as the hash key.
nginx.ingress.kubernetes.io/session-cookie-path	Service	Compatible	The path value for the generated cookie when the specified cookie does not exist. The default is `/`.
nginx.ingress.kubernetes.io/session-cookie-max-age	Service	Compatible	The time-to-live (TTL) of the generated cookie when the specified cookie does not exist. The unit is seconds. The default is session level.
nginx.ingress.kubernetes.io/session-cookie-expires	Service	Compatible	The TTL of the generated cookie when the specified cookie does not exist. The unit is seconds. The default is session level.

IP access control

Annotation	Scope	Support level	Description
nginx.ingress.kubernetes.io/whitelist-source-range	Ingress	Compatible	Specifies the IP address whitelist for the route. Supports IP addresses or Classless Inter-Domain Routing (CIDR) blocks, separated by commas (,).
nginx.ingress.kubernetes.io/denylist-source-range	Ingress	Compatible Requires gateway version 1.2.31	Specifies the IP blacklist for the route. Supports IP addresses or CIDR blocks, separated by commas (,). Note This annotation has a higher priority than the Higress extension annotation higress.ingress.kubernetes.io/blacklist-source-range.
higress.ingress.kubernetes.io/blacklist-source-range	Ingress	Higress extension	Specifies the IP blacklist for the route. Supports IP addresses or CIDR blocks, separated by commas (,).
higress.ingress.kubernetes.io/domain-whitelist-source-range	Ingress	Higress extension	Specifies the IP address whitelist for the domain name. The domain name has a lower priority than the route level. Supports IP addresses or CIDR blocks, separated by commas (,).
higress.ingress.kubernetes.io/domain-blacklist-source-range	Ingress	Higress extension	Specifies the IP blacklist for the domain name. The domain name has a lower priority than the route level. Supports IP addresses or CIDR blocks, separated by commas (,).

Connection pool configuration between the gateway and backend services

Annotation	Scope	Support level	Description
higress.ingress.kubernetes.io/connection-policy-tcp-max-connection	Service	Higress extension	The maximum number of connections that can be established between the gateway and the backend service.
higress.ingress.kubernetes.io/connection-policy-tcp-max-connection-per-endpoint	Service	Higress extension	The maximum number of connections that can be established between the gateway and a single node of the backend service.
higress.ingress.kubernetes.io/connection-policy-http-max-request-per-connection	Service	Higress extension	The maximum number of requests on a single connection between the gateway and the backend service.

Security protection

Encrypted communication between client and gateway

Annotation	Scope	Support level	Description
higress.ingress.kubernetes.io/tls-min-protocol-version	Domain name	Higress extension	Specifies the minimum TLS version. The default is TLSv1.0. Valid values are: TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
higress.ingress.kubernetes.io/tls-max-protocol-version	Domain name	Higress extension	Specifies the maximum TLS version. The default is TLSv1.3. Valid values are: TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
nginx.ingress.kubernetes.io/ssl-cipher	Domain name	Compatible	Specifies the TLS cipher suite. You can specify multiple cipher suites separated by colons (:). This takes effect only when TLSv1.0-1.2 is used for the TLS handshake. The default cipher suites are: ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-ECDSA-AES128-SHA ECDHE-RSA-AES128-SHA AES128-GCM-SHA256 AES128-SHA ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-GCM-SHA384 ECDHE-ECDSA-AES256-SHA ECDHE-RSA-AES256-SHA AES256-GCM-SHA384 AES256-SHA
higress.ingress.kubernetes.io/auth-tls-secret	Domain name	Partially compatible. The secret name must be in the format: (name of the secret where the domain name certificate is located)`-cacert`.	The CA certificate used by the gateway to verify the certificate provided by the client during an mTLS handshake. This annotation is mainly used in scenarios where the gateway needs to verify the client identity.

Encrypted communication between gateway and backend service

Annotation	Scope	Support level	Description
nginx.ingress.kubernetes.io/proxy-ssl-secret	Service	Compatible	The client certificate used by the gateway for identity authentication by the backend service.
nginx.ingress.kubernetes.io/proxy-ssl-name	Service	Compatible	The SNI used during the TLS handshake.
nginx.ingress.kubernetes.io/proxy-ssl-server-name	Service	Compatible	Enables or disables the SNI used during the TLS handshake.

Authentication

Basic

Annotation	Scope	Support level	Description
nginx.ingress.kubernetes.io/auth-type	Ingress	Partially compatible. Only Basic is supported.	The authentication type.
nginx.ingress.kubernetes.io/auth-secret	Ingress	Compatible	The secret name. The format `<namespace>/<name>` is supported. It contains the usernames and passwords of users who are granted access privileges to the routes defined on this Ingress.
nginx.ingress.kubernetes.io/auth-secret-type	Ingress	Compatible	The secret content format. `auth-file`: The key in Data is `auth`, and the value is the username and password. Multiple accounts are separated by carriage returns. `auth-map`: The key in Data is the username, and the value is the password.
nginx.ingress.kubernetes.io/auth-realm	Ingress	Compatible	The protection domain. The same protection domain shares usernames and passwords.

For more information about Nginx Ingress annotations, see the official documentation.