By Yuanyi, Senior Development Engineer at Alibaba
You may be familiar with the RocketMQ service. This article will explore the effect of combining RocketMQ and serverless. I will describe how to drive cloud-native serverless applications based on RocketMQ + Knative. The article will cover the following content:
The Cloud-Native Computing Foundation (CNCF) defines cloud-native as:
Cloud-native technologies empower organizations to build and run scalable applications in modern, dynamic environments, such as the public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructures, and declarative APIs exemplify this approach.
These techniques enable us to build loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, cloud-native technology allows engineers to make high-impact changes frequently, predictably, and effortlessly.
Cloud-native aims to use standardized cloud services to connect cloud providers and customers. This approach helps cloud customers reduce cloud deployment and cross-cloud migration costs and gives them bargaining power with cloud providers. Due to low cross-cloud migration costs, cloud providers can easily acquire a large number of customers by providing cost-effective cloud services.
Serverless means that developers implement the server logic and applications run in stateless computing containers. In containers, applications are triggered by events and fully managed by a third party with business layer statuses stored in a database or other storage media.
Serverless represents an advanced stage of cloud-native technology development. It allows developers to focus on business logic and pay less attention to infrastructure.
Here, we are talking about serverless functions, but there is another type of serverless technology, serverless containers. Compared with serverless functions, serverless containers have stronger portability, helping developers better control complex applications. For those who have experienced the container era, serverless containers may be a better choice.
There are a few things you need to understand about serverless:
We have already mentioned serverless containers, but is there such a thing as a serverless framework? The answer is Knative.
Knative is a Kubernetes-based, serverless orchestration engine that was released at the Google Cloud Next Conference in 2018. One important goal of Knative is to establish a cloud-native, cross-platform, serverless orchestration standard. Knative integrates container or function building, elastic workload management, and event modeling to implement a serverless standard. Major contributors to the Knative community include Google, Pivotal, IBM, and Red Hat. PaaS providers, including CloudFoundry and OpenShift, also actively participate in Knative building.
Core Knative modules include Eventing, an event-driven framework, and Serving for deploying workloads.
The core capability of Knative Serving is its simple and efficient application hosting service, which is also the foundation for its support for serverless. The application hosting service greatly reduces the complexity and risks involved in operating Kubernetes resources. It also improves the efficiency of application iteration and service delivery. As a serverless framework, Knative can allocate resources based on demand. It automatically scales out instances when your application has a large number of requests and scales in instances when the request volume falls, helping you cut costs.
Together with Istio, Serving also provides powerful traffic management and flexible phased release capabilities. Traffic management can split traffic by percentages. Phased release allows you to send a certain percentage of traffic to a new version, allowing you to test it by customizing tags before the full release. This function can be easily integrated with your CI-CD system.
CloudEvents as the event transfer protocol:
CloudEvents describes event data in a common format and provides cross-platform service interaction. Knative Eventing uses
CloudEvents as the event transfer standard, which greatly improves the cross-platform portability of applications.
2) External event source access and registration: Knative Eventing supports event sources, such as Github, RocketMQ, and Kafka. You can also customize event sources.
3) Event subscription and triggering: Knative Eventing introduces the Broker and Trigger models to screen complex event processing and provide a wide range of event subscription and filtering mechanisms.
4) Compatible with existing messaging systems: Knative Eventing fully decouples messaging system implementation. In addition to the memory-based message channel
InMemoryChannel, Knative Eventing also supports message services, such as Kafka and NATSStreaming, and can easily connect to existing messaging systems.
This section introduces the Broker and Trigger models in Eventing, which are simple to use. The external event source sends an event to Broker, and Broker sends it to the corresponding channel, that is, the message caching and forwarding components, such as Kafka or
InMemoryChannel. You can create a Trigger and subscribe to Broker to subscribe to events. In addition, you can define corresponding services in Trigger to implement event-driven services.
Message Queue for Apache RocketMQ is a distributed message middleware with low latency, high concurrency, high availability, and high reliability built by Alibaba Cloud based on Apache RocketMQ. Message Queue for Apache RocketMQ provides asynchronous decoupling and load shifting for distributed application systems and supports features for Internet applications, including massive message accumulation, high throughput, and reliable retry.
RocketMQSource is the RocketMQ event source of Knative. It can forward RocketMQ cluster messages to Knative in real-time in
CloudEvents format. It is a connector between Apache RocketMQ and Knative.
Here, we will use a food delivery application as an example. Food delivery scenarios have the following characteristics:
We can use an event-driven serverless framework to automatically scale out resources in peak hours and scale in resources in off-peak hours. This on-demand resource allocation significantly improves resource utilization and reduces costs.
As shown in the preceding figure, when meal times approach, customers order food. Order messages are generated and sent to RocketMQ. You can obtain order messages through
RocketMQSource, convert them to events, and send the events to Broker. You can then subscribe to order events through Trigger and drive the ordering service to generate orders. This solution has the following advantages:
For more information, see Deploy Knative.
On the Knative Component Management page, select
RocketMQSource, and click Deploy.
For more information, see the example code repository.
The following figure shows the command for one-click service deployment.
kubectl apply -f 200-serviceaccount.yaml -f 202-clusterrolebinding.yaml -f 203-secret.yaml -f alirocketmqsource.yaml -f broker.yaml -f ksvc-order-service.yaml -f trigger.yaml
Use simulated ordering to concurrently send messages to RocketMQ. The following figure shows the message format.
The following figure shows the effect.
Has the preceding introduction to serverless applications driven by RocketMQ + Knative inspired you? You can try it out in your own application scenario. I believe it will give you a unique experience.
Alibaba Clouder - August 5, 2020
Alibaba Clouder - June 20, 2017
Alibaba Clouder - June 30, 2020
Alibaba Clouder - September 18, 2019
Alibaba Developer - September 22, 2020
Aliware - October 20, 2020
Alibaba Cloud Container Service for Kubernetes is a fully managed cloud container management service that supports native Kubernetes and integrates with other Alibaba Cloud products.Learn More
Accelerate and secure the development, deployment, and management of containerized applications cost-effectively.Learn More
A secure image hosting platform providing containerized image lifecycle managementLearn More
AlibabaMQ for Apache RocketMQ is a distributed message queue service that supports reliable message-based asynchronous communication among microservices, distributed systems, and serverless applications.Learn More
More Posts by Alibaba Developer