Manually back up and restore PolarSearch data - PolarDB - Alibaba Cloud Documentation Center

PolarSearch supports backing up index data from your cluster to an Object Storage Service (OSS) bucket and restoring data from OSS using snapshots. This feature enables cross-cluster data migration and custom backup and recovery for PolarSearch clusters, which provides a flexible and cost-effective solution for data protection and data transfer.

Note

This feature is in canary release. To use this feature, submit a ticket to request access.

Applicability

Your PolarSearch cluster is version 2.19.3 or later.
Note
You can access PolarSearch using the endpoint and check the version in the version.number field.
You have activated Alibaba Cloud Object Storage Service (OSS) and created a bucket to store snapshots.
You have created a Resource Access Management (RAM) user and granted the AliyunOSSFullAccess access policy to the user. For more information, see Grant permissions to a RAM user.

Billing information

The snapshot feature is free of charge. However, snapshot files that are stored in your OSS bucket incur storage and request fees. For more information about billing, see OSS Billing overview.

Register a snapshot repository

Before you use the snapshot feature, you must register a snapshot repository and associate the repository with your OSS bucket. You can use the following API to create a snapshot repository:

PUT /_snapshot/{repo-name}
{
    "type": "oss",
    "settings": {
      "endpoint": "{endpoint}",
      "bucket": "{bucket-name}",
      "base_path": "{path-name}",
      "region": "{region}",
      "access_key": "{your-AccessKey-ID}", 
      "secret_key": "{your-AccessKey-secret}",
      "session_token": "{your-STS-Token}"
    }
}

Parameters

Parameter	Description
`{repo-name}`	Repository name. Define your own.
`type`	Repository type. Set this to `oss`.
`endpoint`	The endpoint of your OSS bucket. For details, see Regions and endpoints.
`bucket`	Your OSS bucket name.
`base_path`	(Optional) The root directory path in the OSS bucket where snapshot files are stored.
`region`	The region where the bucket resides.
`access_key`	Your AccessKey ID.
`secret_key`	Your AccessKey secret.
`session_token`	(Optional) The temporary identity credentials (Security Token Service token) for your RAM role. Important If you use an STS token, set both `access_key` and `secret_key` to the values provided by the STS token. If you do not use temporary credentials, use your long-term AccessKey ID and AccessKey secret directly.

Example

Replace the following parameters with your actual values.

curl -X PUT "https://{pc-endpoint}:3001/_snapshot/{repo-name}" \
  -u "{username}:{passwd}" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "oss",
    "settings": {
      "endpoint": "{endpoint}",
      "bucket": "{bucket-name}",
      "base_path": "{path-name}",
      "region": "{region}",
      "access_key": "{your-AccessKey-ID}", 
      "secret_key": "{your-AccessKey-secret}"
    }
  }'

Create a snapshot: Back up data to OSS

After you register a snapshot repository, you can run the following command to create a snapshot for a specific index:

PUT /_snapshot/{repo-name}/{snapshot-name}?wait_for_completion=true
{
    "indices": "{index-name}",
    "ignore_unavailable":false
}

Parameter description

Parameter Categories	Parameter name	Description
Request parameter	`wait_for_completion`	Whether to wait synchronously for the snapshot operation to complete. Default is `false`. Synchronous execution: Set `wait_for_completion=true`. The command waits until the snapshot finishes before returning a result. Asynchronous execution: Set `wait_for_completion=false`. The command returns immediately, and the snapshot runs in the background. Use the following command to check the snapshot status. When the `state` field shows `SUCCESS`, the snapshot is complete. `GET /_snapshot/{repo-name}/{snapshot-name}/_status`
Request body parameters	`indices`	Specifies the indices to back up. Supports wildcard character ``. Separate multiple indices with commas `,`. Default backs up all indices. Note* Using the wildcard `*` includes system index tables. To exclude them, specify only the indices you want to back up and restore, or use `-` to exclude system index tables.
	`ignore_unavailable`	Whether to ignore missing indices and continue creating the snapshot. Default is `false` (fails if an index is missing).
	`partial`	Whether to allow partial snapshots. If set to `true`, successful shards are saved even if some shards fail. Default is `false`.

Example

Replace the following parameters with your actual values.

curl -X PUT "https://{pc-endpoint}:3001/_snapshot/{repo-name}/{snapshot-name}?wait_for_completion=true" \
  -u "{username}:{passwd}" \
  -H "Content-Type: application/json" \
  -d '{
    "indices": "{index-name}",
    "ignore_unavailable": false
  }'

View snapshots

You can list all snapshots in your OSS snapshot repository using the following API:

GET /_snapshot/{repo-name}/_all?pretty

Example

Replace the following parameters with your actual values.

curl -X GET "https://{pc-endpoint}:3001/_snapshot/{repo-name}/_all?pretty" -u "{username}:{passwd}"

Restore data

You can run the following command to restore index data from a specific snapshot using the following API:

Note

To restore a snapshot from one PolarSearch cluster to another PolarSearch cluster, the destination PolarSearch cluster must register the same snapshot repository as the source PolarSearch cluster. If you restore within the same PolarSearch cluster, you can skip registration and restore the data directly.

POST /_snapshot/{repo-name}/{snapshot-name}/_restore?wait_for_completion=true
{
  "indices": "{index-name}",
  "ignore_unavailable": true
}

Parameters

Parameter Classification	Parameter Name	Description
Request parameter	`wait_for_completion`	Whether to wait synchronously for the restore operation to complete. Default is `false`. Synchronous execution: Set `wait_for_completion=true`. The command waits until the restore finishes before returning. Asynchronous execution: Set `wait_for_completion=false`. The command returns immediately, and the restore job runs in the background. Use the following command to check restore progress. When the `stage` field shows `DONE`, the restore is complete. `GET /{index-name}/_recovery`
Request body parameters	`indices`	Specifies the indices to restore. Supports wildcard character `*`. Separate multiple indices with commas `,`. Default restores all indices.
	`ignore_unavailable`	Specifies whether to ignore the index and continue creating the snapshot if the specified index does not exist. The default value is `false`, which causes the operation to fail.
	`partial`	Specifies whether to create a partial snapshot. If set to `true`, data from successful shards is saved even if some shards fail. The default is `false`.
	`index_settings`	Overrides index settings from the snapshot during restore. For example, adjust the number of replicas to match the destination cluster configuration.
	`ignore_index_settings`	Specifies a list of index settings to ignore during restore, typically used to skip source-specific configurations.

Example

Replace the following parameters with your actual values.

curl -X POST "https://{pc-endpoint}:3001/_snapshot/{repo-name}/{snapshot-name}/_restore?wait_for_completion=true" \
  -u "{username}:{passwd}" \
  -H "Content-Type: application/json" \
  -d '{
    "indices": "{index-name}",
    "ignore_unavailable": true
  }'