Store Argo Workflows Artifacts in OSS for Pipeline Sharing - ACK

In complex workflows, the output of one step often becomes the input for the next. Artifacts let you pass files and data between workflow steps by storing them in Object Storage Service (OSS). This topic explains how to configure an OSS artifact repository for the Argo Workflows controller.

Prerequisites

Before you begin, ensure that you have:

An activated OSS service and a bucket. To get started, see Activate OSS and Create a bucket
An AccessKey ID and AccessKey secret with access to the bucket
(Optional) If your OSS bucket and ACK cluster are in the same virtual private cloud (VPC), use the OSS internal endpoint to avoid traffic costs. See Billable items

Configure the OSS artifact repository

Step 1: Create a Secret for OSS credentials

Create a Secret named my-oss-secret in the argo namespace to store your OSS credentials.

apiVersion: v1
kind: Secret
metadata:
  name: my-oss-secret
  namespace: argo
type: Opaque
stringData:
  accessKey: <your-access-key-id>      # Replace with your AccessKey ID.
  secretKey: <your-access-key-secret>   # Replace with your AccessKey secret.

Store credentials in a Secret rather than hardcoding them in workflow templates. This limits exposure if templates are shared or stored in version control.

Step 2: Add the artifact repository to the workflow controller ConfigMap

Run the following command to open the workflow controller ConfigMap for editing:

kubectl edit configmap workflow-controller-configmap -n argo

Insert the following configuration into the data field:

data:
  artifactRepository: |
    archiveLogs: true
    oss:
      endpoint: http://oss-cn-zhangjiakou-internal.aliyuncs.com  # Replace with your OSS endpoint.
      bucket: my-bucket                                            # Replace with your bucket name.
      accessKeySecret:
        name: my-oss-secret
        key: accessKey
      secretKeySecret:
        name: my-oss-secret
        key: secretKey

After you save the ConfigMap, the Argo Workflows controller picks up the changes automatically and uses this repository to upload, download, and delete artifact files based on your workflow definitions.