Configure Argo Workflows to Persist Data in RDS MySQL - Container Compute Service

Argo Workflows periodically deletes workflow states and related resources. Configure a persistence policy to archive workflows to a MySQL database, so you can view workflow logs even after the workflows or their pods are deleted.

This topic uses ApsaraDB RDS for MySQL as the database backend.

Prerequisites

Before you begin, make sure you have:

An Alibaba Cloud Container Service cluster with Argo Workflows installed
Access to the argo namespace in the cluster
Permissions to create Secrets and modify ConfigMaps in the cluster

Step 1: Set up ApsaraDB RDS for MySQL

Create an ApsaraDB RDS for MySQL instance, create a database on the instance, and configure a database account. The RDS instance must be in the same Virtual Private Cloud (VPC) as the cluster. Configure an IP whitelist on the instance to allow inbound access from the VPC's CIDR block.

For detailed steps, see Create an ApsaraDB RDS for MySQL instance and configure a database.

After the setup is complete, record the database account username and password.

For ApsaraDB RDS for MySQL pricing, see Billable items.

Step 2: Add persistence settings

Add persistence settings to the Argo Workflows ConfigMap. By default, you need to manually restart the Argo Workflows controller and Argo Server for the settings to take effect.

1. Create a Secret for database credentials

Create a Secret named argo-mysql-config in the argo namespace to store the database username and password.

Option A: Using kubectl (recommended)

kubectl create secret generic argo-mysql-config -n argo \
  --from-literal=username=<database-username> \
  --from-literal=password=<database-password>

Option B: Using a YAML manifest

apiVersion: v1
kind: Secret
metadata:
  name: argo-mysql-config
  namespace: argo
type: Opaque
stringData:
  username: <database-username>  # Replace with the actual username.
  password: <database-password>  # Replace with the actual password.

Apply the manifest:

kubectl apply -f secret.yaml

2. Update the ConfigMap

Edit workflow-controller-configmap in the argo namespace to add the persistence block under data:

data:
  persistence: |
    connectionPool:
      maxIdleConns: 100
      maxOpenConns: 0
      connMaxLifetime: 0s     # 0 means connections don't have a max lifetime.
    archiveTTL: 30d
    archive: true
    mysql:
      host: rm-xxx.mysql.cn-beijing.rds.aliyuncs.com
      port: 3306
      database: argo-workflow
      tableName: argo_workflows
      userNameSecret:
        name: argo-mysql-config
        key: username
      passwordSecret:
        name: argo-mysql-config
        key: password

Key parameters:

Parameter	Description
`archive`	Set to `true` to enable workflow persistence.
`archiveTTL`	The retention period for archived workflows. In this example, set to `30d`, which means workflows are persisted for 30 days. The retention period does not have an upper limit.
`mysql.host`	The endpoint of the ApsaraDB RDS for MySQL instance.
`mysql.port`	The database port. Default is `3306`.
`mysql.database`	The name of the database to write workflow data to.
`mysql.tableName`	The name of the primary workflow table.
`connectionPool.maxIdleConns`	Maximum number of idle connections in the pool.
`connectionPool.maxOpenConns`	Maximum number of open connections. `0` means unlimited.
`connectionPool.connMaxLifetime`	Maximum connection lifetime. `0s` means connections don't have a max lifetime.
`userNameSecret`	Reference to the Secret key that holds the database username.
`passwordSecret`	Reference to the Secret key that holds the database password.

3. Restart the Argo Workflows controller and Argo Server

Restart the Argo Workflows controller and Argo Server for the persistence settings to take effect.

What's next

To disable persistence, set archive: false in the persistence block and restart the Argo Workflows controller and Argo Server.
To view archived workflows, open the Argo Server UI and navigate to the Archived Workflows section.