Object Storage Service:Process billions of objects using Batch Operations

Availability

The Batch Operations feature is available in the China (Hangzhou), China (Shanghai), China (Shenzhen), China (Beijing), China (Zhangjiakou), China (Ulanqab), and Singapore regions.

Overview

1. Create a RAM role: Create a RAM role to allow the OSS service to read the manifest file, execute batch operations, and generate report files.

1. Prepare an input manifest: Specify the objects to process using a bucket inventory file (manifest.json), a prefix, or a CSV manifest file (manifest.csv).

2. Create a job: Configure a specific operation, such as tagging, setting permissions, or restoring objects, and then submit the job.

3. Manage and maintain jobs: Monitor job status, confirm jobs that are ready to run, view completion reports, and adjust job priority or cancel jobs as needed.

Step 1: Create a RAM role

1. Go to the RAM console to create a role. Set the trusted entity type to Alibaba Cloud Service. Select Object Storage Service as the trusted service.

2. Create and attach a custom permission policy to the role. Use the following policy. Replace your-bucket with the name of your bucket.

   {
     "Version": "1",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "oss:GetBucketInfo",
           "oss:ListObjects",
           "oss:GetObject",
           "oss:GetObjectTagging",
           "oss:PutObjectTagging",
           "oss:DeleteObjectTagging",
           "oss:PutObjectAcl",
           "oss:RestoreObject",
           "oss:PutObject"
         ],
         "Resource": [
           "acs:oss:*:*:your-bucket",
           "acs:oss:*:*:your-bucket/*"
         ]
       }
     ]
   }

   Permission details

   • Required permissions

     • oss:GetBucketInfo: Retrieves bucket information.

   • Input manifest permissions

     • oss:ListObjects: Required to generate an object list from a prefix.

     • oss:GetObject: Required to read a bucket inventory file or a CSV manifest file.

   • Batch operation permissions

     • oss:GetObjectTagging: Required to read existing tags when you batch replace tags.

     • oss:PutObjectTagging: Required to batch add or replace tags.

     • oss:DeleteObjectTagging: Required to batch delete tags.

     • oss:PutObjectAcl: Required to batch modify permissions.

     • oss:RestoreObject: Required to batch restore archived objects.

   • Report output permissions

     • oss:PutObject: Required to upload the batch operation report file.

3. Record the ARN of the role, such as acs:ram::<your-account-uid>:role/BatchRole. You will need the ARN in subsequent steps.

Step 2: Prepare an input manifest

Batch Operations requires you to specify which objects to process. You can choose one of the following three methods depending on your needs:

• Bucket inventory file: Suitable for large-scale, recurring batch operations.

• Prefix: Process all objects in a specific directory.

• CSV manifest file: A CSV file that lists the objects to be processed.

Bucket inventory file

Use a manifest file automatically generated by Bucket Inventory.

<Manifest>
    <Location>
        <ETag>495B279A99C2BA72A66CA1E23B5FEA13</ETag>
        <Bucket>my-bucket</Bucket>
        <Object>inventory-reports/my-bucket/daily-inventory/2025-10-29T12-49Z/manifest.json</Object>
    </Location>
    <Spec>
        <Format>OSS_InventoryReport_CSV_20250611</Format>
    </Spec>
</Manifest>

Configuration details

• ETag: Use the ETag value of the manifest.json file.

• Format: OSS_InventoryReport_CSV_20250611

Prefix

Generates an object list based on a specified prefix.

<KeyPrefixManifestGenerator>
    <SourceBucket>my-bucket</SourceBucket>
    <Prefix>documents/2024/</Prefix>
</KeyPrefixManifestGenerator>

CSV manifest file

Use a CSV file to specify the objects. The CSV manifest file must be uploaded to an OSS bucket in the same region as the batch job.

<Manifest>
    <Location>
        <Bucket>my-bucket</Bucket>
        <Object>manifest.csv</Object>
        <ETag>1466CA50719515AAE6F92101D83B6A20</ETag>
    </Location>
    <Spec>
        <Format>OSS_BatchOperations_CSV_20250611</Format>
        <Fields>Bucket,Key</Fields>
    </Spec>
</Manifest>

Configuration details

• ETag: Use the ETag value of the manifest.csv file.

• Fields: Specify the fields that define the columns in the CSV file.

  • Versioning disabled: Bucket,Key

  • Versioning enabled: Bucket,Key,VersionId

• Format: OSS_BatchOperations_CSV_20250611

CSV file format requirements

• File encoding: Must be UTF-8.

• Field separator: Use a comma (,).

• Object Key encoding: Must be URL-encoded, such as documents%2Freport1.pdf.

• No comments: The CSV file cannot contain comments.

CSV file examples

• Versioning disabled

  my-bucket,documents%2Freport1.pdf
  my-bucket,documents%2Freport2.pdf
  my-bucket,images%2Fphoto1.jpg

• Versioning enabled

  my-bucket,important%2Fdata.json,3/L4kqtJlcpXroDTDmJ+rmSpXd3dIbrHY+MTRCxf3vjVBH40Nr8X8gdRQBpUMLUo
  my-bucket,config%2Fsettings.xml,null

Step 3: Create a job

ossutil api invoke-operation --op-name create-job --method POST --parameters batchJob --body file:///path/to/config.xml --endpoint oss-cn-hangzhou.aliyuncs.com

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <PutObjectTagging>
            <TagSet>
                <Tag><Key>Environment</Key><Value>Production</Value></Tag>
                <Tag><Key>Team</Key><Value>DataOps</Value></Tag>
            </TagSet>
        </PutObjectTagging>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>put-tag-inventory-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>495B279A99C2BA72A66CA1E23B5FEA13</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>inventory-reports/my-bucket/daily-inventory/2025-10-29T12-49Z/manifest.json</Object>
        </Location>
        <Spec>
            <Format>OSS_InventoryReport_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch replace tags using Inventory manifest</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <PutObjectTagging>
            <TagSet>
                <Tag><Key>Environment</Key><Value>Production</Value></Tag>
                <Tag><Key>Team</Key><Value>DataOps</Value></Tag>
            </TagSet>
        </PutObjectTagging>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>put-tag-job-001</ClientRequestToken>
    <KeyPrefixManifestGenerator>
        <SourceBucket>my-bucket</SourceBucket>
        <Prefix>documents/2024/</Prefix>
    </KeyPrefixManifestGenerator>
    <Description>Batch replace tags for documents</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <PutObjectTagging>
            <TagSet>
                <Tag><Key>Environment</Key><Value>Production</Value></Tag>
                <Tag><Key>Team</Key><Value>DataOps</Value></Tag>
            </TagSet>
        </PutObjectTagging>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>put-tag-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>1466CA50719515AAE6F92101D83B6A20</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>manifest.csv</Object>
        </Location>
        <Spec>
            <Fields>Bucket,Key</Fields>
            <Format>OSS_BatchOperations_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch replace tags for specified objects</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<?xml version="1.0" encoding="UTF-8"?>
<CreateJobResult>
  <JobId>ODRiZWQ5Njc5ZTE0NDE5NTlmNjJmZDlkMTk2ODI2NDU=</JobId>
</CreateJobResult>

ossutil api invoke-operation --op-name create-job --method POST --parameters batchJob --body file:///path/to/config.xml --endpoint oss-cn-hangzhou.aliyuncs.com

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <AddObjectTagging>
            <TagSet>
                <Tag><Key>Department</Key><Value>Engineering</Value></Tag>
                <Tag><Key>Project</Key><Value>BatchOps</Value></Tag>
            </TagSet>
        </AddObjectTagging>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>add-tag-inventory-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>495B279A99C2BA72A66CA1E23B5FEA13</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>inventory-reports/my-bucket/daily-inventory/2025-10-29T12-49Z/manifest.json</Object>
        </Location>
        <Spec>
            <Format>OSS_InventoryReport_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch add tags using Inventory manifest</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <AddObjectTagging>
            <TagSet>
                <Tag><Key>Department</Key><Value>Engineering</Value></Tag>
                <Tag><Key>Project</Key><Value>BatchOps</Value></Tag>
            </TagSet>
        </AddObjectTagging>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>add-tag-job-001</ClientRequestToken>
    <KeyPrefixManifestGenerator>
        <SourceBucket>my-bucket</SourceBucket>
        <Prefix>documents/2024/</Prefix>
    </KeyPrefixManifestGenerator>
    <Description>Batch add tags to documents</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <AddObjectTagging>
            <TagSet>
                <Tag><Key>Department</Key><Value>Engineering</Value></Tag>
                <Tag><Key>Project</Key><Value>BatchOps</Value></Tag>
            </TagSet>
        </AddObjectTagging>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>add-tag-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>1466CA50719515AAE6F92101D83B6A20</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>manifest.csv</Object>
        </Location>
        <Spec>
            <Fields>Bucket,Key</Fields>
            <Format>OSS_BatchOperations_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch add tags to specified objects</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<?xml version="1.0" encoding="UTF-8"?>
<CreateJobResult>
  <JobId>Nzg5ZjE3ZTU5OGMxNDZjZDk0YjJiODMzZDZiZjA0ZmU=</JobId>
</CreateJobResult>

ossutil api invoke-operation --op-name create-job --method POST --parameters batchJob --body file:///path/to/config.xml --endpoint oss-cn-hangzhou.aliyuncs.com

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <DeleteObjectTagging />
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>delete-tag-inventory-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>495B279A99C2BA72A66CA1E23B5FEA13</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>inventory-reports/my-bucket/daily-inventory/2025-10-29T12-49Z/manifest.json</Object>
        </Location>
        <Spec>
            <Format>OSS_InventoryReport_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch delete tags using Inventory manifest</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <DeleteObjectTagging />
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>delete-tag-job-001</ClientRequestToken>
    <KeyPrefixManifestGenerator>
        <SourceBucket>my-bucket</SourceBucket>
        <Prefix>documents/2024/</Prefix>
    </KeyPrefixManifestGenerator>
    <Description>Batch delete tags for documents</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <DeleteObjectTagging />
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>delete-tag-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>1466CA50719515AAE6F92101D83B6A20</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>manifest.csv</Object>
        </Location>
        <Spec>
            <Fields>Bucket,Key</Fields>
            <Format>OSS_BatchOperations_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch delete tags for specified objects</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<?xml version="1.0" encoding="UTF-8"?>
<CreateJobResult>
  <JobId>NTVlYmQ4ZmQyOTM0NDJmY2E2NmNjYTUwNjU1ZGRkMGM=</JobId>
</CreateJobResult>

Batch modify permissions

Batch permission modification supports four Access Control List (ACL) types: `private` (only the owner can read and write), `public-read` (all users can read, but only the owner can write), `public-read-write` (all users can read and write), and `default` (inherits the bucket's permission settings). Permission changes take effect immediately and directly affect object access permissions. Choose the appropriate permission type carefully based on your requirements.

ossutil api invoke-operation --op-name create-job --method POST --parameters batchJob --body file:///path/to/config.xml --endpoint oss-cn-hangzhou.aliyuncs.com

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <PutObjectAcl>
            <ObjectAcl>private</ObjectAcl>
        </PutObjectAcl>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>acl-inventory-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>495B279A99C2BA72A66CA1E23B5FEA13</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>inventory-reports/my-bucket/daily-inventory/2025-10-29T12-49Z/manifest.json</Object>
        </Location>
        <Spec>
            <Format>OSS_InventoryReport_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch change ACL using Inventory manifest</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <PutObjectAcl>
            <ObjectAcl>private</ObjectAcl>
        </PutObjectAcl>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>acl-job-001</ClientRequestToken>
    <KeyPrefixManifestGenerator>
        <SourceBucket>my-bucket</SourceBucket>
        <Prefix>documents/2024/</Prefix>
    </KeyPrefixManifestGenerator>
    <Description>Batch change ACL to private for documents</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <PutObjectAcl>
            <ObjectAcl>private</ObjectAcl>
        </PutObjectAcl>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>acl-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>1466CA50719515AAE6F92101D83B6A20</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>manifest.csv</Object>
        </Location>
        <Spec>
            <Fields>Bucket,Key</Fields>
            <Format>OSS_BatchOperations_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch change ACL to private for specified objects</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<?xml version="1.0" encoding="UTF-8"?>
<CreateJobResult>
  <JobId>YWNsR29iMTIzNDU2Nzg5MA==</JobId>
</CreateJobResult>

Batch data restoration

The batch restore operation supports objects in the Archive, ColdArchive, and DeepColdArchive storage classes. Restoring Archive objects typically takes about one hour, and the restored objects are accessible for 1 to 7 days. The ColdArchive storage class offers two restoration tiers: Standard (2 to 5 hours) and Bulk (5 to 12 hours). The DeepColdArchive storage class supports only the Standard tier, which takes up to 48 hours to complete. You can set the accessibility period of a restored object to a value from 1 to 365 days.

ossutil api invoke-operation --op-name create-job --method POST --parameters batchJob --body file:///path/to/config.xml --endpoint oss-cn-hangzhou.aliyuncs.com

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <RestoreObject>
            <Days>7</Days>
            <Tier>Standard</Tier>
        </RestoreObject>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>restore-inventory-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>495B279A99C2BA72A66CA1E23B5FEA13</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>inventory-reports/my-bucket/daily-inventory/2025-10-29T12-49Z/manifest.json</Object>
        </Location>
        <Spec>
            <Format>OSS_InventoryReport_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch restore using Inventory manifest for 7 days</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <RestoreObject>
            <Days>7</Days>
            <Tier>Standard</Tier>
        </RestoreObject>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>restore-job-001</ClientRequestToken>
    <KeyPrefixManifestGenerator>
        <SourceBucket>my-bucket</SourceBucket>
        <Prefix>archive/2024/</Prefix>
    </KeyPrefixManifestGenerator>
    <Description>Batch restore archive objects for 7 days</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<CreateJobRequest>
    <ConfirmationRequired>true</ConfirmationRequired>
    <Operation>
        <RestoreObject>
            <Days>7</Days>
            <Tier>Standard</Tier>
        </RestoreObject>
    </Operation>
    <Report>
        <Bucket>my-bucket</Bucket>
        <Enabled>true</Enabled>
        <Prefix>batch-reports/</Prefix>
        <ReportScope>AllTasks</ReportScope>
    </Report>
    <ClientRequestToken>restore-job-001</ClientRequestToken>
    <Manifest>
        <Location>
            <ETag>1466CA50719515AAE6F92101D83B6A20</ETag>
            <Bucket>my-bucket</Bucket>
            <Object>manifest.csv</Object>
        </Location>
        <Spec>
            <Fields>Bucket,Key</Fields>
            <Format>OSS_BatchOperations_CSV_20250611</Format>
        </Spec>
    </Manifest>
    <Description>Batch restore specified archive objects for 7 days</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
</CreateJobRequest>

<?xml version="1.0" encoding="UTF-8"?>
<CreateJobResult>
  <JobId>cmVzdG9yZUpvYjEyMzQ1Njc4OTA=</JobId>
</CreateJobResult>

Step 4: Job management and O&M

View job status

After you create a job, you can monitor its status by querying the job details. This provides complete status information for a specific job, including its execution progress and configuration parameters.

ossutil api invoke-operation --op-name describe-job --method GET --parameters batchJob --parameters batchJobId=<JobId> --endpoint oss-cn-hangzhou.aliyuncs.com

<?xml version="1.0" encoding="UTF-8"?>
<DescribeJobResult>
  <Job>
    <ConfirmationRequired>true</ConfirmationRequired>
    <CreationTime>1761723644</CreationTime>
    <FailureReasons>
      <JobFailure />
    </FailureReasons>
    <JobId>M2E0ODMyNmJjYmYzNGY0ZThiYjMyMmI1ZDE1YWE0OGE=</JobId>
    <Operation>
      <PutObjectTagging>
        <TagSet>
          <Tag>
            <Key>Environment</Key>
            <Value>Production</Value>
          </Tag>
          <Tag>
            <Key>Team</Key>
            <Value>DataOps</Value>
          </Tag>
        </TagSet>
      </PutObjectTagging>
    </Operation>
    <Report>
      <Enabled>true</Enabled>
      <Bucket>my-bucket</Bucket>
      <Prefix>batch-reports/</Prefix>
      <ReportScope>AllTasks</ReportScope>
    </Report>
    <Manifest>
      <Location>
        <ETag>1466CA50719515AAE6F92101D83B6A20</ETag>
        <Bucket>my-bucket</Bucket>
        <Object>manifest.csv</Object>
      </Location>
      <Spec>
        <Format>OSS_BatchOperations_CSV_20250611</Format>
        <Fields>Bucket,Key</Fields>
      </Spec>
    </Manifest>
    <Description>Test task for Suspended status</Description>
    <Priority>10</Priority>
    <RoleArn>acs:ram::<your-account-uid>:role/BatchRole</RoleArn>
    <ProgressSummary>
      <NumberOfTasksFailed>0</NumberOfTasksFailed>
      <NumberOfTasksSucceeded>0</NumberOfTasksSucceeded>
      <TotalNumberOfTasks>3</TotalNumberOfTasks>
      <Timers>
        <ElapsedTimeInActiveSeconds>-1</ElapsedTimeInActiveSeconds>
      </Timers>
    </ProgressSummary>
    <Status>Suspended</Status>
    <TerminationDate>-1</TerminationDate>
  </Job>
</DescribeJobResult>

Confirm a pending job

Change the job status from Suspended to Ready to queue the job for execution. If a job is not confirmed within 14 days, its status automatically changes to Failed.

ossutil api invoke-operation --op-name update-job-status --method POST --parameters batchJob --parameters batchJobId=<JobId> --parameters requestedJobStatus=Ready --endpoint oss-cn-hangzhou.aliyuncs.com

<?xml version="1.0" encoding="UTF-8"?>
<UpdateJobStatusResult>
  <JobId>NGFhNmFmZDRlNTcxNDE2OGFhY2FjYWQyNDcxYjc3YzM=</JobId>
  <Status>Ready</Status>
</UpdateJobStatusResult>

Query a list of jobs in a batch

You can query a list of all your jobs. This feature lets you filter the job list by job status and creation time range. You can also set a limit on the number of returned results and use pagination to manage large numbers of jobs. By default, the query results are sorted by creation time in descending order, with the most recently created jobs displayed first.

ossutil api invoke-operation --op-name list-jobs --method GET --parameters batchJob --parameters maxKeys=100 --endpoint oss-cn-hangzhou.aliyuncs.com

<?xml version="1.0" encoding="UTF-8"?>
<ListJobsResult>
  <Jobs>
    <Job>
      <JobId>OTE5ZGU1NWMxNjkyNGFkMjhkYTNmOGEzNTU2YmJiOWE=</JobId>
      <Status>Complete</Status>
      <CreationTime>1761666548</CreationTime>
    </Job>
  </Jobs>
</ListJobsResult>

Adjust job priority

The priority value ranges from 0 to 2147483647. A larger value indicates a higher priority. A high-priority job can preempt resources from a low-priority job. Job priority does not guarantee a strict execution order. OSS attempts to allocate more resources to higher-priority jobs.

ossutil api invoke-operation --op-name update-job-priority --method POST --parameters batchJob --parameters batchJobId=<JobId> --parameters priority=1000 --endpoint oss-cn-hangzhou.aliyuncs.com

<?xml version="1.0" encoding="UTF-8"?>
<UpdateJobPriorityResult>
  <JobId>OTE5ZGU1NWMxNjkyNGFkMjhkYTNmOGEzNTU2YmJiOWE=</JobId>
  <Priority>1000</Priority>
</UpdateJobPriorityResult>

Cancel a job

Terminate a job that is running or waiting to run. Operations on objects that have already been processed cannot be rolled back.

ossutil api invoke-operation --op-name update-job-status --method POST --parameters batchJob --parameters batchJobId=<JobId> --parameters requestedJobStatus=Cancelled --endpoint oss-cn-hangzhou.aliyuncs.com

<?xml version="1.0" encoding="UTF-8"?>
<UpdateJobStatusResult>
  <JobId>OTE5ZGU1NWMxNjkyNGFkMjhkYTNmOGEzNTU2YmJiOWE=</JobId>
  <Status>Cancelled</Status>
</UpdateJobStatusResult>

View a job report

After a job is complete, the system generates a detailed execution report in the path you specify. The report contains two key types of files: a manifest.json file (the report manifest file) and a detailed report file in CSV format. These reports provide the processing status, HTTP status code, and failure reason for each object. This information is useful for troubleshooting and result verification.

<Bucket>/<Prefix>/<JobId>/results/

<Report>
    <Bucket>my-bucket</Bucket>
    <Prefix>batch-reports/</Prefix>
    <ReportScope>AllTasks</ReportScope>
</Report>

{
  "Format": "Report_CSV_20250901",
  "ReportCreationDate": "2025-10-31T07:18:56Z",
  "Results": [
    {
      "TaskExecutionStatus": "succeeded",
      "Bucket": "test-bucket",
      "MD5Checksum": "\"424356315f4a82c1b6a4c190dbed15d9\"",
      "Key": "reports/prefix-job-0/job-MTRiZGI1NDFiMTI2NDFiNDg0YWQyYTM2YmY1NzNjNDM=/results/report-0.csv"
    }
  ],
  "ReportSchema": "Bucket,Key,VersionId,HTTPStatus,Error"
}

{
  "Format": "Report_CSV_20250901",
  "ReportCreationDate": "2025-10-31T07:37:56Z",
  "Results": [
    {
      "TaskExecutionStatus": "failed",
      "Bucket": "test-bucket",
      "MD5Checksum": "\"85749f22b5b62b161daf00b4adcf6da7\"",
      "Key": "reports/prefix-job-0/job-M2E4MDkwNGJmNmQzNDgwYmE4ODI1YjdjNDgyNjk5NmM=/results/report-0.csv"
    }
  ],
  "ReportSchema": "Bucket,Key,VersionId,HTTPStatus,Error"
}

test-bucket,batchOperation%2F1761895099%2Fobject-1.txt,,200,
test-bucket,batchOperation%2F1761895099%2Fobject-9.txt,,200,
test-bucket,batchOperation%2F1761895099%2Fobject-4.txt,,200,
test-bucket,batchOperation%2F1761895099%2Fobject-8.txt,,200,
test-bucket,batchOperation%2F1761895099%2Fobject-3.txt,,200,

test-bucket,batchOperation%2F1761896253%2Fobject-0.txt,,403,"Error returned by Service. {""EC"":""0003-00000001"",""Error Code"":""AccessDenied"",""Http Status Code"":403,""Message"":""You have no right to access this object."",""Request Endpoint"":""PUT http://test-bucket.oss-cn-hangzhou-internal.aliyuncs.com/batchOperation/1761896253/object-0.txt?tagging"",""Request Id"":""690467520D9F2E3232004FFD"",""Timestamp"":""2025-10-31T07:37:54Z""}"
test-bucket,batchOperation%2F1761896253%2Fobject-7.txt,,403,"Error returned by Service. {""EC"":""0003-00000001"",""Error Code"":""AccessDenied"",""Http Status Code"":403,""Message"":""You have no right to access this object."",""Request Endpoint"":""PUT http://test-bucket.oss-cn-hangzhou-internal.aliyuncs.com/batchOperation/1761896253/object-7.txt?tagging"",""Request Id"":""69046752A016113937EB7029"",""Timestamp"":""2025-10-31T07:37:54Z""}"

# Download the report manifest file
ossutil cp oss://my-bucket/batch-reports/<JobId>/results/manifest.json ./

# Download the detailed CSV report
ossutil cp oss://my-bucket/batch-reports/<JobId>/results/report-0.csv ./

Using in production environments

Best practices

• Job planning strategy: To ensure stability, limit the number of objects in a single job to 100 million. Split large datasets into multiple jobs to reduce the risk of a single point of failure. Set appropriate job priorities to prevent important jobs from being blocked. Estimate the running time to schedule maintenance windows. When multiple jobs operate on the same objects concurrently, the execution order is not guaranteed. Avoid performing concurrent operations on the same object.

• Performance optimization measures: To prevent slow parsing, organize your manifest file structure and avoid using excessively large files. Use the independent QPS quota to ensure that batch jobs do not affect your regular API calls. Run a maximum of three jobs concurrently to avoid resource competition.

• Error handling mechanism: Enable detailed reports (`ReportScope=AllTasks`) to obtain complete results for troubleshooting.

• Result confirmation: After a job runs, some objects may be processed successfully while others fail. This can happen regardless of the final job status, which can be `Running`, `Complete`, `Failed`, `Paused`, or `Canceled`. Always check the number of successful and failed operations in the report to ensure that the outcome meets your expectations.

• Failure handling: A `Complete` status means the job has finished, but it does not guarantee that all objects were processed successfully. Always check the execution report for details. For failed tasks, analyze the cause of failure and retry the operation. For jobs that partially fail, you can create a new job based on the failure report to retry the failed operations. For jobs with a `Cancelled`, `Paused`, or `Failed` status, check the report because some objects may have been processed successfully.

Resource scheduling mechanism

• Independent QPS quota: Jobs use an independent quota of 10,000 QPS and do not consume your application's QPS.

• Priority scheduling: Supports priorities from 0 to 2,147,483,647. High-priority jobs preempt resources from low-priority jobs.

• Concurrency control: A single account can run a maximum of six concurrent jobs. Additional jobs are placed in a waiting queue.

Fault tolerance strategy

• Permission verification: Before creating a job, verify access permissions for the source path to prevent execution failures. Ensure the RAM role has all the necessary permissions for the target operation. Rotate the AccessKey for the RAM role regularly to improve security.

• Data protection: Create snapshots or backups before performing batch delete or overwrite operations. Enable versioning for important data to protect it with multiple versions. Verify the operation logic in a test environment before you run it in production.

• Monitoring and alerts: Configure event notifications for job status changes to stay informed about job progress. Monitor job running times to detect abnormally long-running jobs. Set threshold alerts for failure rates to respond quickly to issues.

Cost control

• Billing method: Billing is based on the number of objects processed, at a rate of CNY 1.00 per 1 million objects. No fees are charged for creating jobs. You are billed only for the number of objects processed. This method can save over 90% in operational costs compared to making individual API calls for each object.

• Optimization suggestions: To avoid idle resources, use the full limit of six concurrent jobs. Regularly delete old job reports to reduce storage costs. Promptly delete test jobs to avoid unnecessary charges.

Quotas and limits

Billing

OSS Batch Operations uses a pay-as-you-go billing model. The service is billed at CNY 1 for every 1 million objects processed. No fees are charged for job creation or management. No additional data transfer fees are incurred.

Billing examples:

• Batch tagging 5 million objects: 5 million ÷ 1 million × CNY 1 = CNY 5

• Large-scale data processing for 5 billion objects: 5 billion ÷ 1 million × CNY 1 = CNY 5,000

Cost optimization suggestions: Merge related batch operations into a single job to reduce management overhead. Use precise prefixes or manifest files to avoid processing unnecessary objects. Schedule non-urgent batch operations to run during off-peak hours.

FAQ

Why does a job remain in the Preparing state for a long time after submission?

This can happen for several reasons:

• The manifest file is too large, which increases parsing time.

• The prefix scan must process many objects.

• The system load is high, causing the processing queue to be busy.

To resolve this issue, you can split large jobs into smaller ones or use a pre-generated manifest file.

What should I do if some objects fail to process?

Follow these steps:

1. Download the job execution report and analyze the cause of each failure.

2. Create a new job to retry the operations on the failed objects.

3. Verify your permission configuration to ensure that the RAM role has all the required permissions.

How do I cancel a running job?

Use the following command:

POST /?batchJobStatus&batchJobId=<JobId>&requestedJobStatus=Cancelled&statusUpdateReason=Cancelled by user HTTP/1.1
Host: oss-cn-hangzhou.aliyuncs.com
Authorization: <SignatureInformation>

Why do I get a permission denied error?

Check your permission configuration:

1. Ensure the Resource Access Management (RAM) user has the oss:CreateBatchJob permission.

2. Verify that the RAM role includes the specific permissions required for the operation.

3. Verify the permission settings of the destination bucket for cross-bucket operations.

What are the configuration requirements for a RAM role?

The IAM role must include the following:

• Read and write permissions for OSS resources.

• Permission to call Batch Operations APIs.

• Write permission for the bucket where reports are stored.

Example trust policy:

{
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": ["oss.aliyuncs.com"]
      }
    }
  ]
}

How can I improve the execution speed of batch operations?

Follow these optimization suggestions:

1. Properly organize manifest files and avoid using single, excessively large files.

2. Avoid running large jobs during peak business hours.

3. Assign job priorities to ensure important jobs are executed first.

4. To prevent long running times, limit the number of objects in a single job to under 100 million.

5. Avoid splitting a large job into an excessive number of small jobs. For example, do not split a job with 1 million objects into 1,000 separate jobs. This approach reduces overhead from task scheduling.

Why is the job running time longer than expected?

This can happen for several reasons:

• The objects are unevenly distributed, which causes some shards to take longer to process.

• Hotspots in the source or destination objects are affecting the processing speed.

• Too many jobs are running at the same time, which leads to intense resource competition.

To resolve this issue, you can adjust the job scheduling policy or reduce the number of concurrent jobs.