Container Service for Kubernetes:Use spot instances in Knative

Prerequisites

Before you begin, ensure that you have:

• Knative deployed in your ACK cluster. For more information, see Deploy and manage Knative components.

• ack-virtual-node 2.11.0 or later, if you want to use the Eviction API for graceful shutdown. For more information, see ack-virtual-node.

Key concepts

• Spot instance

  A cost-effective, preemptible compute instance. You can bid on idle compute capacity from Alibaba Cloud to run your containers. The instance runs until your bid falls below the current market price or the resource is reclaimed due to insufficient inventory. For more information, see What is a spot instance?

• Knative

  Knative aims to establish a standard for cloud-native, cross-platform serverless application orchestration. Its key features include request-based autoscaling, scaling to zero, multi-version management, traffic-based canary releases, function deployment, and an event-driven architecture. For more information, see ACK Knative overview and ACK Serverless Knative overview.

How it works

Knative Services automatically scale Pods in and out based on incoming requests. Each Pod runs on a virtual node, which provisions the corresponding Elastic Container Instance (ECI) resource based on Pod annotations. To use spot instances, add spot-specific annotations to the Pod template of your Knative Service — the virtual node handles provisioning and automatic replacement.

Knative is well-suited for spot instances for the following reasons:

• Request-scoped workloads: Knative Pods handle individual requests and release resources immediately after completion, so interruptions rarely affect in-progress work.

• Built-in graceful shutdown: Knative injects a queue-proxy sidecar container into every Pod. When a Pod is terminated, queue-proxy waits for all in-flight requests to complete before the application container shuts down.

• Cost-effective: For cost-conscious users of Knative, using spot instances provides significant savings.

Limitations

Configure spot instances

Add the following annotations to the Pod template of your Knative Service.

Choose a bidding strategy

Two bidding strategies are available. Choose based on your tolerance for cost variance and creation failure.

Examples

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: helloworld-go
spec:
  template:
    metadata:
      labels:
        alibabacloud.com/eci: "true"
      annotations:
        k8s.aliyun.com/eci-use-specs : "ecs.c6.large"           # Specify the ECS instance type.
        k8s.aliyun.com/eci-spot-strategy: "SpotWithPriceLimit"  # Use the strategy that sets a custom price limit.
        k8s.aliyun.com/eci-spot-price-limit: "0.25"            # Set the maximum hourly price.
    spec:
      containers:
      - env:
        - name: TARGET
          value: "Knative"
        image: registry.cn-hangzhou.aliyuncs.com/knative-sample/helloworld-go:73fbdd56

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: helloworld-go
spec:
  template:
    metadata:
      labels:
        alibabacloud.com/eci: "true"
      annotations:
        k8s.aliyun.com/eci-use-specs : "ecs.c6.large"           # Specify the ECS instance type.
        k8s.aliyun.com/eci-spot-strategy: "SpotWithPriceLimit"  # Use the strategy that sets a custom price limit.
        k8s.aliyun.com/eci-spot-price-limit: "0.05"             # Set the maximum hourly price.
        k8s.aliyun.com/eci-spot-fallback: "true"                # Automatically fall back to an on-demand instance if spot inventory is unavailable.
    spec:
      containers:
      - env:
        - name: TARGET
          value: "Knative"
        image: registry.cn-hangzhou.aliyuncs.com/knative-sample/helloworld-go:73fbdd56

Create a Knative Service with spot instances

The following procedure creates a sample Knative Service that uses the SpotAsPriceGo strategy with on-demand fallback enabled.

1. Log on to the ACK console. In the left navigation pane, click ACK consoleClusters.

2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Applications > Knative.

3. On the Knative page, click the Services tab. Select default from the Namespace drop-down list and click Create from Template. In the Sample Template drop-down list, select custom. Paste the following YAML into the editor and click Create.

   apiVersion: serving.knative.dev/v1
   kind: Service
   metadata:
     name: helloworld-go
   spec:
     template:
       metadata:
         labels:
           alibabacloud.com/eci: "true"
         annotations:
           k8s.aliyun.com/eci-use-specs : "ecs.c6.large"           # Specify the ECS instance type.
           k8s.aliyun.com/eci-spot-strategy: "SpotAsPriceGo"       # Use the strategy that follows the market price.
           k8s.aliyun.com/eci-spot-duration: "1"                   # Protection period in hours.
           k8s.aliyun.com/eci-spot-fallback: "true"                # Fall back to on-demand if spot inventory is unavailable.
       spec:
         containers:
         - env:
           - name: TARGET
             value: "Knative"
           image: registry-vpc.{REGION-ID}.aliyuncs.com/knative-samples/helloworld-go:160e4dc8 # Replace {REGION-ID} with your cluster's region ID, such as cn-hangzhou.

   This configuration creates an ecs.c6.large spot instance with the following behavior:

   • If spot inventory is unavailable at creation, an on-demand instance is created instead.

   • After creation, the instance runs for a guaranteed 1-hour protection period.

   • After the protection period, the spot instance may be reclaimed if the market price exceeds the current bid or if inventory becomes insufficient. The eci-spot-fallback: "true" setting ensures an on-demand instance is created if spot inventory is unavailable when the virtual node replaces the Pod.

Handle spot instance interruptions

Alibaba Cloud sends a SpotToBeReleased event and sets the ContainerInstanceExpired pod condition to true approximately 3 minutes before a spot instance is reclaimed.

The following figure shows an example of the notification.

During this 3-minute window, you can configure graceful handling so that in-flight requests complete before the Pod terminates.

Monitor spot instance events

Events:
  Type     Reason            Age    From          Message
  ----     ------            ----   ----          -------
  Warning  SpotToBeReleased  3m32s  kubelet, eci  Spot ECI will be released in 3 minutes

LAST SEEN   TYPE      REASON             OBJECT         MESSAGE
3m39s       Warning   SpotToBeReleased   pod/pi-frmr8   Spot ECI will be released in 3 minutes

NAME       READY   STATUS      RESTARTS   AGE
pi-frmr8   1/1     BidFailed   0          3h5m

Status:             Failed
Reason:             BidFailed
Message:            The pod is spot instance, and have been released at 2020-04-08T12:36Z

What's next

If your application is sensitive to cold start latency, consider using reserved instances to keep a low-spec burstable instance running at all times. This reduces startup time while keeping costs low. For more information, see Configure reserved instances.