Offload Argo workflow status information to ApsaraDB RDS for MySQL - Container Service for Kubernetes

Workflows are stored as Kubernetes resources in workflow clusters. The size of each resource is limited to 1 MB or less. A workflow resource stores the status of each task in the /status/nodes field. When a workflow contains large numbers of tasks, the resource may exceed 1 MB in size. In this case, exceptions occur when the status information is updated. To resolve this issue, you can offload the status information to an external SQL database. This topic describes how to offload the status information of workflows to ApsaraDB RDS for MySQL to support large workflows.

Prerequisites

A workflow is created.
A kubectl client is connected to the workflow cluster.
An ApsaraDB RDS for MySQL instance is created.
Important
The ApsaraDB RDS for MySQL instance must be deployed in the same virtual private cloud (VPC) as the workflow cluster. The VPC CIDR block must be added to the whitelist of the ApsaraDB RDS for MySQL instance.
A database and a database account are created.
After the database is created, record the database name, database username and password, and endpoint of the ApsaraDB RDS for MySQL instance. The account must have read and write permissions on the database. For more information about how to view the endpoint of an ApsaraDB RDS for MySQL instance, see View and manage instance endpoints and ports.

Procedure

Create file named secret.yaml and add the following content to the file to store the username and password of the database:

apiVersion: v1
stringData:
  username: <database-username>  # Replace with the database username. 
  password: <database-password>  # Replace with the database password. 
kind: Secret
metadata:
  name: argo-mysql-config
  namespace: default
type: Opaque

Run the following command to create a Secret:
```
kubectl apply -f secret.yaml
```

Modify the workflow-controller-configmap file. Add nodeStatusOffLoad: true to persistence to enable the workflow offloading feature. For more information about modifying the workflow-controller-configmap file, see Modify the configuration of a workflow cluster.

  persistence: |
    connectionPool:
      maxIdleConns: 100
      maxOpenConns: 0
      connMaxLifetime: 0s # 0 means connections don't have a max lifetime
    archiveTTL: 30d
    nodeStatusOffLoad: true
    mysql:
      host: rm-****.mysql.cn-beijing.rds.aliyuncs.com
      port: 3306
      database: argo-workflow  # Specify the database you created in the Prerequisites section.
      tableName: argo_workflows
      userNameSecret:
        name: argo-mysql-config
        key: username
      passwordSecret:
        name: argo-mysql-config
        key: password

Parameter	Description
nodeStatusOffLoad	Set to `true` to enable the workflow offloading feature.
mysql.host	The endpoint of the ApsaraDB RDS for MySQL instance.
mysql.database	The name of the database.

After you complete the configuration, restart the Argo Server pod and ensure that the pod runs as normal.
```
kubectl delete pod argo-server-xxx -n <cluster-id>
```
Note
If the Argo Server pod repeatedly restarts, check the connection between the workflow cluster and ApsaraDB RDS for MySQL instance. Make sure that they are deployed in the same VPC and the VPC CIDR block is added to the whitelist of the ApsaraDB RDS for MySQL instance.

References

For more information about how to view workflow status and the billing rules, see Create a workflow.
For more information about offloading large workflows, see Offloading Large Workflows.