Store Argo Workflows Artifacts in OSS for Pipeline Data Passing - Container Compute Service

In a complex workflow, the output of one step often feeds into the next. Artifacts let you pass files between steps by storing them in Object Storage Service (OSS). This topic explains how to configure OSS as the artifact repository for Argo Workflows by updating the workflow-controller-configmap ConfigMap.

Prerequisites

Before you begin, make sure you have:

An activated OSS service and an existing OSS bucket. See Activate OSS and Create a bucket.
An AccessKey ID and AccessKey secret with permissions to read from and write to the bucket.

If the OSS bucket and your cluster are in the same Virtual Private Cloud (VPC), use the private endpoint of the bucket to access OSS objects without incurring data transfer fees. For billing details, see Billable items.

Configure the artifact repository

Step 1: Create a Secret for OSS credentials

Create a Secret named my-oss-secret in the argo namespace to store the credentials used to access the OSS bucket.

apiVersion: v1
kind: Secret
metadata:
  name: my-oss-secret
  namespace: argo
type: Opaque
stringData:
  accessKey: <your-access-key-id>      # Replace with your AccessKey ID
  secretKey: <your-access-key-secret>  # Replace with your AccessKey secret

Step 2: Add artifact configuration to the ConfigMap

Run the following command to edit the workflow-controller-configmap ConfigMap:

kubectl edit configmap workflow-controller-configmap -n argo

Add the following artifact repository configuration to the data field:

data:
  artifactRepository: |
   archiveLogs: true
   oss:
     endpoint: http://oss-cn-zhangjiakou-internal.aliyuncs.com  # Replace with your bucket's endpoint
     bucket: my-bucket                                           # Replace with your bucket name
     accessKeySecret:
       name: my-oss-secret  # References the Secret created in Step 1
       key: accessKey
     secretKeySecret:
       name: my-oss-secret  # References the Secret created in Step 1
       key: secretKey

The following table describes the key fields:

Field	Description
`archiveLogs`	Set to `true` to archive workflow logs as artifacts in OSS
`oss.endpoint`	The endpoint of your OSS bucket. Use the private endpoint if the bucket and cluster are in the same VPC
`oss.bucket`	The name of your OSS bucket
`accessKeySecret` / `secretKeySecret`	References to the Secret that holds your OSS credentials

After you save the ConfigMap, the Argo Workflows controller detects the change and starts uploading, downloading, and deleting artifact files based on your workflow definitions.