By Chengtan
This article introduces the capabilities of combining Higress and Nacos as a microservice gateway and highlights two emerging trends in the development of microservice gateways, aiding in the selection process:
Higress and Nacos both originate from Alibaba's middleware team. During its usage in supporting Alibaba Cloud's internal business operations, Higress, along with Nacos, served as a microservice gateway, effectively handling the high traffic load during the Double 11 shopping festival. As cloud products were commercialized, Higress and Nacos continued to collaborate closely, evolving their features based on Alibaba Cloud Microservices Engine (MSE). With Higress now being open source, opting for Higress and Nacos to build a microservice gateway provides the following advantages:
For more information about the installation and deployment of Higress, see the documentation on the official website of Higress. Higress is installed by default here. You can use it with Nacos in the following ways:
apiVersion: networking.higress.io/v1
kind: McpBridge
metadata:
name: default
namespace: higress-system
spec:
registries:
# Define a service source named "production".
- name: production
# The registry type is Nacos 2.x. The gRPC protocol is supported.
type: nacos2
# The access address of the registry can be a domain name or an IP address.
domain: 192.xxx.xx.32
# The Nacos access port of the registry is 8848 by default.
port: 8848
# Nacos namespace ID.
nacosNamespaceId: d8ac64f3-xxxx-xxxx-xxxx-47a814ecf358
# Nacos service group.
nacosGroups:
- DEFAULT_GROUP
# Define a service source named "uat".
- name: uat
# The registry type is Nacos 1.x. Only HTTP protocol is supported.
type: nacos
# The access address of the registry can be a domain name or an IP address.
domain: 192.xxx.xx.31
# The Nacos access port of the registry is 8848 by default.
port: 8848
# Nacos namespace ID.
nacosNamespaceId: 98ac6df3-xxxx-xxxx-xxxx-ab98115dfde4
# Nacos service group.
nacosGroups:
- DEFAULT_GROUPWe have configured two service sources, named production and uat, using McpBridge resources. It is important to note that Higress supports both HTTP and gRPC protocols when connecting to Nacos. We recommend upgrading Nacos to version 2.x. By specifying nacos2 in the configured type, the gRPC protocol can be utilized. This allows for faster detection of service changes and reduces the resource consumption on the Nacos server.
Higress can easily connect to multiple service sources of different types, such as Nacos, Zookeeper, Eureka, and Consul, based on the registries array configuration in McpBridge. For Nacos service sources, Higress supports configuring multiple namespaces. This enables microservices in different namespaces to share the same gateway, reducing the resource costs associated with building individual microservice gateways.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
higress.io/destination: service-provider.DEFAULT-GROUP.d8ac64f3-xxxx-xxxx-xxxx-47a814ecf358.nacos
name: demo
namespace: default
spec:
rules:
- http:
paths:
- backend:
resource:
apiGroup: networking.higress.io
kind: McpBridge
name: default
path: /
pathType: PrefixUnlike traditional Ingress, which defines a service in the backend, the Ingress-based resource backend is associated with the previously defined service source in McpBridge. It specifies the destination service to which the route is ultimately forwarded using the "higress.io/destination" annotation. For a service sourced from Nacos, the format of the destination service is service name.service group.namespace ID.nacos. Please note that the DNS domain name format must be followed, and any underscores _ in the service group should be converted to hyphens -.
Built on microservice discovery, Higress offers a range of practical microservice gateway capabilities, including "canary release" and "custom extension." For more details on these capabilities, please refer to the original text and consult the documentation on Higress's official website.
Higress fully supports the canary annotations of Nginx Ingress. The following code demonstrates how to route requests with an HTTP header of x-user-id: 100 to the canary service.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
higress.io/destination: service-provider.DEFAULT-GROUP.98ac6df3-xxxx-xxxx-xxxx-ab98115dfde4.nacos
nginx.ingress.kubernetes.io/canary: 'true'
nginx.ingress.kubernetes.io/canary-by-header: x-user-id
nginx.ingress.kubernetes.io/canary-by-header-value: '100'
name: demo-uat
namespace: default
spec:
rules:
- http:
paths:
- backend:
resource:
apiGroup: networking.higress.io
kind: McpBridge
name: default
path: /
pathType: PrefixYou can also use OpenKruise Rollout to link the canary release and service deployment processes to implement progressive delivery.
Microservice gateways often need to handle common logic in microservice architectures, such as authentication and security protection. However, this common logic may not be sufficient for diverse business scenarios. To address this issue, Higress allows developers to add custom processing logic. Unlike traditional microservice gateways like Spring Cloud Gateway, which require developers to add filters to the gateway's code, Higress enables developers to write Wasm plugins in multiple languages and dynamically load them for immediate effect. These plugins can be applied without restarting the gateway, and modifying their logic has no impact on ongoing traffic.
The following example demonstrates a Wasm plugin that blocks specific requests. If the request URL contains swagger.html, access will be denied. For more information on implementing the plugin's code, please refer to: https://github.com/alibaba/higress/tree/main/plugins/wasm-go/extensions/request-block
apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
name: request-block
namespace: higress-system
spec:
selector:
matchLabels:
higress: higress-system-higress-gateway
pluginConfig:
block_urls:
- "swagger.html"
url: oci://higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/request-block:1.0.0For more information about the development, compilation, and image push modes of Wasm plug-in, see Higress Practice: Use 30 Lines of Code to Write a Wasm Go Plugin
Different implementations can be based on a set of APIs, which not only prevents users from being locked into specific implementations but also bridges the gap in technology evolution. APIs are considered the foundation of the entire cloud-native architecture. While Kubernetes may eventually disappear, abstraction-oriented API standards will undoubtedly endure. In the field of traffic gateways, the Ingress API has become the standard. However, for more complex usage scenarios like microservice gateways, Ingress is limited by its simple protocol fields and requires extensions through Ingress annotations, making standardization challenging. As a result, gateways such as Contour, Emissary, Kong, and APISIX have defined their own HTTP routes and other Custom Resource Definitions (CRDs), leading to fragmented gateway API definitions.
Against this backdrop, the Gateway API has emerged and progressed from alpha to beta over the past year. While the Gateway API has not been finalized and its protocol is subject to change (not recommended for production), the trend towards unified APIs is inevitable and will become a reality in the future. The figure below illustrates a use case scenario of the Gateway API. Unlike the Ingress API, the Gateway API separates cluster operations and business operations, relieving business developers from concerns about cluster-level details such as website certificates. They can focus solely on the DevOps of the business itself, while cluster operations and maintenance tasks are handled by SRE personnel through a unified process.
Higress extends its capabilities using Ingress annotations and is compatible with most common Nginx Ingress annotations, enabling smooth migration to the Gateway API.
Higress allows traditional microservices architectures to progressively evolve into Kubernetes-based cloud-native microservices architectures. By using Nacos to discover services deployed outside of Kubernetes, Higress decouples gateway backend microservices from Kubernetes, enabling business teams to migrate microservices to Kubernetes one by one without worrying about traffic impact at the gateway layer.
Migrating from traditional microservice gateways to Higress and gradually transforming the entire microservices architecture into a cloud-native environment is a wise choice.
Higress introduces a three-in-one concept that combines security, traffic, and microservices gateways. Let's first examine a typical multi-layer gateway architecture:
In this architecture, a WAF gateway is used to implement security capabilities, an Ingress gateway is used for cluster ingress gateway capabilities (Nginx may be deployed in non-Kubernetes scenarios), and SCG (Spring Cloud Gateway) is used for microservice gateways. Evaluating the capacity of each gateway layer is necessary in this architecture. Each layer of gateways represents a potential bottleneck that may require scaling out, resulting in significant resource and operations and maintenance costs. Moreover, each additional layer of gateways introduces availability risks. In the event of availability issues, multi-layer gateways significantly increase the complexity of problem identification, leading to a longer Mean Time to Recovery (MTTR).
The three-in-one architecture can significantly reduce costs and improve the overall system availability. It aligns with the DevSecOps trend in microservice evolution, where microservice developers can focus on security from the perspective of business interfaces rather than adopting a one-size-fits-all WAF prevention mode for all routes.
Behind the evolution of technical architecture is the evolution of organizational architecture, which is emphasized in microservice DevOps: focusing on developers to enhance microservice development efficiency. The evolution towards DevSecOps has no shortcuts and requires collaboration between development and operations roles to break down barriers, forming a fully-featured and agile team in terms of development, deployment, and security operations.
Using Higress as a means to merge gateways paves the way for the evolution towards DevSecOps and is a wise choice.
Exploration and Practice of Cloud-native Observability of Apache Dubbo
481 posts | 48 followers
FollowAlibaba Cloud Native Community - October 31, 2023
Alibaba Cloud Native Community - October 31, 2023
Alibaba Cloud Native Community - September 18, 2023
Alibaba Cloud Native Community - December 6, 2022
Alibaba Cloud Native Community - February 2, 2024
Alibaba Cloud Native Community - June 30, 2023
481 posts | 48 followers
Follow
Best Practices
Follow our step-by-step best practices guides to build your own business case.
Learn More
Microservices Engine (MSE)
MSE provides a fully managed registration and configuration center, and gateway and microservices governance capabilities.
Learn More
Managed Service for Prometheus
Multi-source metrics are aggregated to monitor the status of your business and services in real time.
Learn More
Cloud-Native Applications Management Solution
Accelerate and secure the development, deployment, and management of containerized applications cost-effectively.
Learn MoreMore Posts by Alibaba Cloud Native Community