Container Service for Kubernetes:Use event-driven scaling based on a MongoDB event source

ScaledJob vs. ScaledObject

Use ScaledJob when each event maps to a discrete, long-running task that should run to completion in its own pod. For each detected event, ack-keda schedules a single Job: it initializes, processes one event, and terminates. This isolation prevents a slow task from blocking others and lets you control concurrency precisely with maxReplicaCount.

Use ScaledObject when events drive continuous throughput and your workload is best handled by a pool of long-running Deployment replicas.

The tutorial below uses ScaledJob for a video transcoding scenario. When a record with "state":"waiting" is inserted into MongoDB, ack-keda creates a Job pod to execute the transcoding task and updates the record to "state":"finished" after completion. Once the Job finishes, its metadata is automatically cleaned up.

Create example namespaces

This example uses the mongodb namespace for the database and mongodb-test for autoscaling configurations.

kubectl create ns mongodb
kubectl create ns mongodb-test

Deploy MongoDB

If you already have a MongoDB service, skip this step.

1. Create mongoDB.yaml.

   Important

   This MongoDB service is for demonstration only and does not provide high availability. Do not use it in a production environment.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: mongodb
     namespace: mongodb
   spec:
     replicas: 1
     selector:
       matchLabels:
         name: mongodb
     template:
       metadata:
         labels:
           name: mongodb
       spec:
         containers:
         - name: mongodb
           image: registry-cn-shanghai.ack.aliyuncs.com/acs/mongo:v5.0.0
           imagePullPolicy: IfNotPresent
           ports:
           - containerPort: 27017
             name: mongodb
             protocol: TCP
   ---
   kind: Service
   apiVersion: v1
   metadata:
     name: mongodb-svc
     namespace: mongodb
   spec:
     type: ClusterIP
     ports:
     - name: mongodb
       port: 27017
       targetPort: 27017
       protocol: TCP
     selector:
       name: mongodb

2. Deploy MongoDB.

   kubectl apply -f mongoDB.yaml

Initialize the MongoDB database

1. Get the MongoDB pod name.

   MONGO_POD_NAME=$(kubectl get pods -n mongodb -l name=mongodb -o jsonpath='{.items[0].metadata.name}')
   echo "MongoDB pod name: $MONGO_POD_NAME"

2. In the test database, create user test_user and collection test_collection.

   # Create user
   kubectl exec -n mongodb ${MONGO_POD_NAME} -- mongo --eval 'db.createUser({ user:"test_user",pwd:"test_password",roles:[{ role:"readWrite", db: "test"}]})'
   
   # Authenticate user
   kubectl exec -n mongodb ${MONGO_POD_NAME} -- mongo --eval 'db.auth("test_user","test_password")'
   
   # Create collection
   kubectl exec -n mongodb ${MONGO_POD_NAME} -- mongo test --eval 'db.createCollection("test_collection")'

Configure TriggerAuthentication and ScaledJob

ack-keda uses TriggerAuthentication to securely manage credentials for connecting to event sources. ScaledJob defines scaling rules, polling intervals, and the Job template to execute.

The following configuration combines all three required resources—Secret, TriggerAuthentication, and ScaledJob—in a single file.

1. Create keda-mongodb.yaml. The secretTargetRef field in TriggerAuthentication reads the connection string from the specified Secret and passes it to ack-keda to authenticate with MongoDB. The query field in ScaledJob defines the data condition: when ack-keda finds documents in test_collection matching {"type":"mp4","state":"waiting"}, it launches Job resources.

   apiVersion: v1
   kind: Secret
   metadata:
     name: mongodb-secret
     namespace: mongodb-test
   type: Opaque
   data:
     # Base64-encoded value of:
     # mongodb://test_user:test_password@mongodb-svc.mongodb.svc.cluster.local:27017/test
     connect: bW9uZ29kYjovL3Rlc3RfdXNlcjp0ZXN0X3Bhc3N3b3JkQG1vbmdvZGItc3ZjLm1vbmdvZGIuc3ZjLmNsdXN0ZXIubG9jYWw6MjcwMTcvdGVzdA==
   ---
   apiVersion: keda.sh/v1alpha1
   kind: TriggerAuthentication
   metadata:
     name: mongodb-trigger
     namespace: mongodb-test
   spec:
     secretTargetRef:
       - parameter: connectionString
         name: mongodb-secret
         key: connect
   ---
   apiVersion: keda.sh/v1alpha1
   kind: ScaledJob
   metadata:
     name: mongodb-job
     namespace: mongodb-test
   spec:
     jobTargetRef:
       template:
         spec:
           containers:
             - name: mongo-update
               image: registry-cn-shanghai.ack.aliyuncs.com/acs/mongo-update:v6
               args:
                 - --dataBase=test
                 - --collection=test_collection
                 - --operation=updateMany
                 - --update={"$set":{"state":"finished"}}
               env:
                 - name: MONGODB_CONNECTION_STRING
                   value: mongodb://test_user:test_password@mongodb-svc.mongodb.svc.cluster.local:27017/test
               imagePullPolicy: IfNotPresent
           restartPolicy: Never
       backoffLimit: 1
     pollingInterval: 15         # Check MongoDB every 15 seconds
     maxReplicaCount: 5          # Run at most 5 concurrent Jobs
     successfulJobsHistoryLimit: 0  # Delete completed Jobs immediately
     failedJobsHistoryLimit: 10     # Keep the last 10 failed Jobs for debugging
     triggers:
       - type: mongodb
         metadata:
           dbName: test
           collection: test_collection
           query: '{"type":"mp4","state":"waiting"}'  # Launch a Job for each matching document
           queryValue: "1"
         authenticationRef:
           name: mongodb-trigger

2. Deploy the configuration.

   kubectl apply -f keda-mongodb.yaml