Reliable Traffic Management for Kubernetes Applications on Alibaba Cloud

By Sai Joshitha Kathari

A user sends a request to an application running in Kubernetes. Before that request reaches a pod, it may pass through a load balancer, an Ingress rule, a Kubernetes Service, and one or more backend health decisions. When the request fails, any of those layers can be involved.

That request path is why I treat traffic management as part of reliability engineering. An application can be running while users still see errors because the route is wrong, the backend was removed from service, or the load balancer considers an unhealthy instance ready to receive traffic. ACK gives teams several ways to expose workloads, including LoadBalancer Services and ALB Ingress for HTTP and HTTPS routing. The operational question is how those mechanisms behave when something changes.

Ingress configuration is production configuration

An Ingress manifest looks like application configuration, but it can directly change where real requests go. A path rule can redirect traffic to another Service. A health-check change can decide whether a backend stays in rotation. A canary rule can expose users to a new application version.

I would review those changes with the same attention as application code. The YAML may be small, but the effect is not. When several applications share traffic infrastructure, one configuration mistake can also have a wider blast radius than the team making the change expects.

A useful health check has to represent useful work

Health checks are simple until the definition of health is wrong. An endpoint that returns HTTP 200 whenever the process is alive proves that the process exists. It does not prove that the application can complete the work users expect.

For example, the application may be running but unable to reach a critical dependency. It may still be initializing. An internal queue may be blocked. If the health endpoint ignores those conditions, the load balancer can continue sending traffic to a backend that is technically alive but practically unusable.

The opposite extreme is also risky. If the health check depends on too many downstream systems, a temporary dependency issue can make every backend remove itself at the same time. I usually think about the question more narrowly: is this instance safe to receive requests right now? The answer should be specific enough to protect users without turning every transient dependency problem into a full traffic outage.

Timing changes the way failures look

Health-check intervals, timeouts, and thresholds determine how quickly a backend is taken out of service and how quickly it returns. Aggressive settings remove bad targets faster, but they can also react to short-lived failures. Conservative settings reduce unnecessary churn but may leave an unhealthy backend serving traffic longer.

There is no useful universal setting here. The right values depend on the application, how quickly it normally responds, and what kind of temporary failure it can tolerate. A low-latency API and a slower internal service may need very different behavior.

Canary routing becomes valuable when it is tied to validation

A deployment is one of the easiest times to introduce a production regression. Moving all traffic to a new version immediately gives the team very little room to observe the new behavior before every user is exposed.

ALB Ingress can support progressive routing, including weighted canary traffic. That allows a stable version to continue receiving most requests while a smaller percentage goes to the new release. The routing mechanism is useful, but the real safety comes from what the team measures during that window.

Suppose 10% of traffic is going to the canary. I would compare the new version with the stable one on a small set of signals: error rate, latency, pod restarts, resource behavior, and any application-specific success measure that matters for the service. If those remain normal for a reasonable period, the team has evidence to increase exposure. If they do not, rollback happens while most traffic is still on the stable version.

Define the rollback line before the release starts

Teams make worse rollout decisions when they decide success criteria in the middle of a bad deployment. The pressure is already high and there is a natural tendency to keep waiting for the new version to recover.

Before traffic moves, I would write down a few conditions that stop the rollout. That might be an unacceptable increase in errors, a sustained latency regression, repeated restarts, or an application-specific failure signal. The exact thresholds depend on the service. What matters is agreeing on them before the canary begins.

Shared traffic infrastructure needs clear ownership

Load balancers and Ingress infrastructure are often shared because it is efficient, but shared infrastructure also creates shared risk. If several applications depend on the same ALB or routing configuration, a change made for one workload may affect another.

During normal operation, that dependency can be easy to forget. During an incident, it becomes important very quickly. Teams should know which applications share the same traffic layer, who can change it, and where to look for recent routing changes. That information saves time when an outage crosses namespace or team boundaries.

Observe the request path, not only the application

Application dashboards usually tell you what happened after a request reached the service. They do not always tell you whether the request reached a healthy backend in the first place.

When troubleshooting traffic issues, I want to separate a few cases: the load balancer cannot reach the backend, health checks removed targets, routing sent traffic somewhere unexpected, or the request reached the application and failed there. Those cases may look similar to a user, but they lead to very different investigations.

This is where traffic-layer observability and application observability should meet. If a release changed Ingress rules at the same time that errors began, that context matters. If backends are being marked unhealthy while the application itself looks normal, that points the investigation somewhere else.

Reliability is the behavior between the layers

ACK and Alibaba Cloud load-balancing services provide the mechanics for routing, health checks, and progressive traffic control. Reliable operation depends on how those mechanics are configured and how clearly teams understand the request path.

A good design should make normal traffic easy to explain and failure traffic possible to diagnose. That means meaningful health checks, deliberate canary criteria, clear ownership of shared routing, and monitoring that can distinguish a traffic problem from an application problem. Those details are what turn a working route into an operable one.

Disclaimer: The views expressed herein are for reference only and don't necessarily represent the official views of Alibaba Cloud.

Community

Reliable Traffic Management for Kubernetes Applications on Alibaba Cloud

Ingress configuration is production configuration

A useful health check has to represent useful work

Timing changes the way failures look

Canary routing becomes valuable when it is tied to validation

Define the rollback line before the release starts

Shared traffic infrastructure needs clear ownership

Observe the request path, not only the application

Reliability is the behavior between the layers

Read previous post:

Read next post:

Sai Joshitha Kathari

You may also like

Comments

Sai Joshitha Kathari

Related Products

Container Service for Kubernetes

Container Compute Service (ACS)

ACK One

Cloud-Native Applications Management Solution