Configure Argo workflow persistence using ApsaraDB for MySQL - Container Service for Kubernetes

Argo Workflows periodically deletes workflow-related resources from workflow clusters. To retain workflow history for analysis and tracing, persist workflow data to a database. After persistence is configured, workflow logs remain accessible even after workflows or their pods are deleted. This topic uses ApsaraDB RDS for MySQL as an example to describe how to persist workflow data to a database.

Prerequisites

Before you begin, make sure that you have:

An ACK Distributed Cloud Container Platform for Kubernetes workflow cluster
An ApsaraDB RDS for MySQL instance. For more information, see Create an ApsaraDB RDS for MySQL instance
A database and an account for the ApsaraDB RDS for MySQL instance. For more information, see Create accounts and databases
The ApsaraDB RDS for MySQL instance in the same virtual private cloud (VPC) as the workflow cluster
The VPC CIDR block added to the whitelist of the ApsaraDB RDS for MySQL instance

Create a Secret for database credentials

Create a Secret named argo-mysql-config in the workflow cluster to store the database username and password.

Run the following command:

kubectl create secret generic argo-mysql-config \
  --namespace=default \
  --from-literal=username=<database-username> \
  --from-literal=password=<database-password>

Replace <database-username> and <database-password> with the credentials of the database account created in the prerequisites.

The command creates a Secret equivalent to the following YAML manifest:

apiVersion: v1
kind: Secret
metadata:
  name: argo-mysql-config   # Secret name referenced by the persistence configuration
  namespace: default
type: Opaque
stringData:
  username: <database-username>   # Replace with your database username
  password: <database-password>   # Replace with your database password

Configure persistence settings

Add the persistence block to the workflow-controller-configmap ConfigMap.

The workflow-controller-configmap ConfigMap is stored in the namespace that uses the cluster ID as its name.
Set host to the endpoint of the ApsaraDB RDS for MySQL instance.
Set database to the name of the database created in the prerequisites.
Set archive to true to enable workflow archiving.
archiveTTL specifies how long archived workflows are retained. For example, 30d retains workflows for 30 days. Valid values are not limited to a fixed set.

The following table describes the key parameters:

Parameter	Description	Example value
`connectionPool.maxIdleConns`	Maximum number of idle connections in the pool	`100`
`connectionPool.maxOpenConns`	Maximum number of open connections. `0` means unlimited.	`0`
`connectionPool.connMaxLifetime`	Maximum connection lifetime. `0s` means no limit.	`0s`
`archiveTTL`	Retention period for archived workflows	`30d`
`archive`	Whether to enable workflow archiving	`true`
`mysql.host`	ApsaraDB RDS for MySQL endpoint	`rm-xxx.mysql.cn-beijing.rds.aliyuncs.com`
`mysql.port`	MySQL port	`3306`
`mysql.database`	Database name	`argo-workflow`
`mysql.tableName`	Table name for workflow data	`argo_workflows`
`mysql.userNameSecret.name`	Secret name that stores the database username	`argo-mysql-config`
`mysql.userNameSecret.key`	Key in the Secret for the username	`username`
`mysql.passwordSecret.name`	Secret name that stores the database password	`argo-mysql-config`
`mysql.passwordSecret.key`	Key in the Secret for the password	`password`

persistence: |
    connectionPool:
      maxIdleConns: 100
      maxOpenConns: 0
      connMaxLifetime: 0s
    archiveTTL: 30d
    archive: true
    mysql:
      host: rm-xxx.mysql.cn-beijing.rds.aliyuncs.com
      port: 3306
      database: argo-workflow
      tableName: argo_workflows
      userNameSecret:
        name: argo-mysql-config
        key: username
      passwordSecret:
        name: argo-mysql-config
        key: password

What to do next

After workflows are persisted, use the Argo CLI to view workflow logs even after the workflows are deleted. For more information, see Configure Log Service.