Access Redis by Using Higress Based on Custom Plug-ins

This article introduces Higress and describes how Redis plugins can be utilized for implementing sophisticated features such as rate limiting, caching, and session management.

By Yucheng

This article introduces Higress, a gateway that leverages WebAssembly (WASM) for edge computing and allows users to create plug-ins in Go, C++, or Rust to expand its functionality. It specifically highlights how Redis plugins can be utilized for implementing sophisticated features such as rate limiting, caching, and session management.

Introduction

Higress provides excellent extensibility based on the WASM mechanism. Users can write WASM plugins using Go, C++, or Rust and customize the request processing logic to meet their personalized needs. Currently, the plugins have incorporated support for Redis calls, enabling users to write stateful plugins and further enhancing the extensibility of Higress.

The documentation Use a Plug-in to Call ApsaraDB for Redis Services [1] provides a complete example of how to use a gateway to call ApsaraDB for Redis by using plug-ins, including the processes of creating and configuring an ApsaraDB for Redis instance, writing plug-in code, uploading and configuring plug-ins, and testing samples. Next, this article focuses on several plug-ins based on ApsaraDB for Redis.

Global Throttling for Multiple Gateways

The gateway has already provided the sentinal throttling [2], which can effectively protect backend business applications. The ApsaraDB for Redis plug-in throttling allows you to implement global quota management for multiple gateways.

The following code provides an example of how to use the plug-in to check the number of requests in the current period at the request header stage. If the number of requests exceeds the quota, the plug-in directly returns a 429 response.

func onHttpRequestHeaders(ctx wrapper.HttpContext, config RedisCallConfig, log wrapper.Log) types.Action {
    now := time.Now()
    minuteAligned := now.Truncate(time.Minute)
    timeStamp := strconv.FormatInt(minuteAligned.Unix(), 10)
    // If "err != nil" is returned by the ApsaraDB for Redis API, the gateway fails to find the ApsaraDB for Redis backend service. In this case, check whether the ApsaraDB for Redis backend service is deleted.
    err := config.client.Incr(timeStamp, func(response resp.Value) {
        if response.Error() != nil {
            log.Errorf("call redis error: %v", response.Error())
            proxywasm.ResumeHttpRequest()
        } else {
            ctx.SetContext("timeStamp", timeStamp)
            ctx.SetContext("callTimeLeft", strconv.Itoa(config.qpm-response.Integer()))
            if response.Integer() == 1 {
                err := config.client.Expire(timeStamp, 60, func(response resp.Value) {
                    if response.Error() != nil {
                        log.Errorf("call redis error: %v", response.Error())
                    }
                    proxywasm.ResumeHttpRequest()
                })
                if err != nil {
                    log.Errorf("Error occured while calling redis, it seems cannot find the redis cluster.")
                    proxywasm.ResumeHttpRequest()
                }
            } else {
                if response.Integer() > config.qpm {
                    proxywasm.SendHttpResponse(429, [][2]string{{"timeStamp", timeStamp}, {"callTimeLeft", "0"}}, []byte("Too many requests\n"), -1)
                } else {
                    proxywasm.ResumeHttpRequest()
                }
            }
        }
    })
    if err != nil {
        // Since the calling of the ApsaraDB for Redis service fails, allow the request to pass and log this event.
        log.Errorf("Error occured while calling redis, it seems cannot find the redis cluster.")
        return types.ActionContinue
    } else {
        // Hold the request to wait for completing the calling of the ApsaraDB for Redis service.
        return types.ActionPause
    }
}

The following figure shows the plug-in configuration.

Test results:

Implement Token Throttling by Combining Qwen

For developers who provide AI application services, user token quota management is a very critical feature. The following example shows how to use the gateway plug-in to implement token throttling for the backend services of Tongyi Qianwen (Qwen).

First of all, you need to apply for Qwen API access. For more information, see the reference [3]. Then, configure the corresponding services and routes on the MSE gateway as follows:

Write the plug-in code. In the plug-in, write the token quota used by the request during the response to the body stage, and read ApsaraDB for Redis to check the current remaining token quota during the processing of requests stage. If there is no token quota, the response is directly returned and the request is terminated.

func onHttpRequestBody(ctx wrapper.HttpContext, config TokenLimiterConfig, body []byte, log wrapper.Log) types.Action {
  now := time.Now()
  minuteAligned := now.Truncate(time.Minute)
  timeStamp := strconv.FormatInt(minuteAligned.Unix(), 10)
  config.client.Get(timeStamp, func(response resp.Value) {
    if response.Error() != nil {
      defer proxywasm.ResumeHttpRequest()
      log.Errorf("Error occured while calling redis")
    } else {
      tokenUsed := response.Integer()
      if config.tpm < tokenUsed {
        proxywasm.SendHttpResponse(429, [][2]string{{"timeStamp", timeStamp}, {"TokenLeft", fmt.Sprint(config.tpm - tokenUsed)}}, []byte("No token left\n"), -1)
      } else {
        proxywasm.ResumeHttpRequest()
      }
    }
  })

  return types.ActionPause
}

func onHttpResponseBody(ctx wrapper.HttpContext, config TokenLimiterConfig, body []byte, log wrapper.Log) types.Action {
  now := time.Now()
  minuteAligned := now.Truncate(time.Minute)
  timeStamp := strconv.FormatInt(minuteAligned.Unix(), 10)
  tokens := int(gjson.ParseBytes(body).Get("usage").Get("total_tokens").Int())
  config.client.IncrBy(timeStamp, tokens, func(response resp.Value) {
    if response.Error() != nil {
      defer proxywasm.ResumeHttpResponse()
      log.Errorf("Error occured while calling redis")
    } else {
      if response.Integer() == tokens {
        config.client.Expire(timeStamp, 60, func(response resp.Value) {
          defer proxywasm.ResumeHttpResponse()
          if response.Error() != nil {
            log.Errorf("Error occured while calling redis")
          }
        })
      }
    }
  })
  return types.ActionPause
}

Test results:

Cookie-based Caching, Disaster Recovery, and Session Management

In addition to the preceding two throttling examples, more plug-ins can be implemented based on ApsaraDB for Redis to extend the gateway. For example, cookie-based caching, disaster recovery, and session management can be implemented.

• Caching and disaster recovery: caching request responses based on user cookie information can reduce the pressure on backend services and implement disaster recovery when backend services are unavailable.

• Session management: ApsaraDB for Redis is used to store user authentication information. When a request arrives, ApsaraDB for Redis is first accessed to check whether the current user is authorized to access. If the current user is not authorized, the authentication service is then accessed so that the pressure on the authentication service can be reduced.

func onHttpRequestHeaders(ctx wrapper.HttpContext, config HelloWorldConfig, log wrapper.Log) types.Action {
  cookieHeader, err := proxywasm.GetHttpRequestHeader("cookie")
  if err != nil {
    proxywasm.LogErrorf("error getting cookie header: %v", err)
    // Implement your business logic.
  }
    // Process the cookie as needed.
  cookie := CookieHandler(cookieHeader)
  config.client.Get(cookie, func(response resp.Value) {
    if response.Error() != nil {
      log.Errorf("Error occured while calling redis")
      proxywasm.ResumeHttpRequest()
    } else {
      // Implement your business logic.
      proxywasm.ResumeHttpRequest()
    }
  })
  return types.ActionPause
}

Summary

Higress has significantly expanded the capabilities of its plug-ins by enabling ApsaraDB for Redis calls, providing more flexibility to cater to the diverse personalized needs of developers. If you have further ideas or suggestions for Higress, feel free to reach out to us!

Reference

[1] Use a Plug-in to Call ApsaraDB for Redis Services
[2] Sentinal Throttling
[3] Link (Currently available in Chinese only)

Community

Access Redis by Using Higress Based on Custom Plug-ins

Introduction

Global Throttling for Multiple Gateways

Implement Token Throttling by Combining Qwen

Cookie-based Caching, Disaster Recovery, and Session Management

Summary

Reference

Read previous post:

Read next post:

Alibaba Cloud Native Community

You may also like

Comments

Alibaba Cloud Native Community

Related Products

ApsaraDB for Redis

Microservices Engine (MSE)

ApsaraDB RDS for MariaDB

ApsaraDB for MyBase