manually back up and restore polarsearch data - PolarDB - Alibaba Cloud Documentation Center

PolarSearch supports backing up index data from your cluster to an Object Storage Service (OSS) bucket using the snapshot feature, or restoring data from OSS. This feature enables cross-cluster data migration and custom backup and restore for PolarSearch clusters, providing a flexible, low-cost solution for data protection and transfer.

Note

This feature is currently in a canary release. To use it, you can submit a ticket to contact Alibaba Cloud support and request that the feature be enabled for your account.

Scope

Your PolarSearch version must be 2.19.3 or later.
Note
You can access PolarSearch using the search endpoint to view the PolarSearch version in version.number.
You have enabled Alibaba Cloud Object Storage Service (OSS) and created a bucket to store snapshots.
Prepare a Resource Access Management (RAM) user and grant the AliyunOSSFullAccess access policy to the RAM user. For details, see Grant permissions to a RAM user.

Billing Details

The snapshot feature is free. However, snapshot files stored in your OSS bucket incur OSS storage and request fees. For billing details, see OSS Billing overview.

Register a Snapshot Repository

Before you use the snapshot feature, register a snapshot repository to associate it with your OSS bucket. Use the following API to create the repository:

PUT /_snapshot/{repo-name}
{
    "type": "oss",
    "settings": {
      "endpoint": "{endpoint}",
      "bucket": "{bucket-name}",
      "base_path": "{path-name}",
      "region": "{region}",
      "access_key": "{your-AccessKey-ID}", 
      "secret_key": "{your-AccessKey-Secret}",
      "session_token": "{your-STS-Token}"
    }
}

Parameter description

Parameter	Description
`{repo-name}`	Repository name. Customize it.
`type`	Repository type. Set to `oss`.
`endpoint`	The endpoint of your OSS bucket. For details, see Regions and endpoints.
`bucket`	The name of your OSS bucket.
`base_path`	(Optional) Specify the root directory path in the OSS bucket to store snapshot files.
`region`	Specify the region where the bucket is located.
`access_key`	Your AccessKey ID.
`secret_key`	Your AccessKey Secret.
`session_token`	(Optional) The Security Token Service (STS) token of your RAM role. Important If you use an STS token, set `access_key` and `secret_key` to the STS token values. If you do not use a temporary STS token, use your long-term AccessKey ID and AccessKey Secret.

Example

Replace the following parameters with your actual information.

curl -X PUT "https://{pc-endpoint}:3001/_snapshot/{repo-name}" \
  -u "{username}:{passwd}" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "oss",
    "settings": {
      "endpoint": "{endpoint}",
      "bucket": "{bucket-name}",
      "base_path": "{path-name}",
      "region": "{region}",
      "access_key": "{your-AccessKey-ID}", 
      "secret_key": "{your-AccessKey-Secret}"
    }
  }'

Create a Snapshot: Back Up Data to OSS

After you register a snapshot repository, run the following command to create a snapshot for the specified index. Use the following API to create the snapshot:

PUT /_snapshot/{repo-name}/{snapshot-name}?wait_for_completion=true
{
    "indices": "{index-name}",
    "ignore_unavailable":false
}

Parameter description

Parameter category	Parameter name	Description
Request parameters	`wait_for_completion`	Whether to synchronously wait for the snapshot operation to complete. Default is `false`. Synchronous execution: Set `wait_for_completion=true`. The command waits for the snapshot creation to complete before returning a result. Asynchronous execution: Set `wait_for_completion=false`. The command returns immediately, and the snapshot is created in the background. Check the snapshot status using the following command. When the `state` field in the returned result is `SUCCESS`, creation is complete. `GET /_snapshot/{repo-name}/{snapshot-name}/_status`
Request body parameters	`indices`	Specify the indexes to back up. Wildcard characters (``) are supported. Separate multiple indexes with commas (`,`). Default is all indexes. Note* Using a wildcard character (`*`) backs up system tables. To avoid backing up or restoring system tables, specify only the indexes to back up and restore, or exclude system tables using `-`.
	`ignore_unavailable`	Whether to ignore the specified index if it does not exist and continue creating the snapshot. Default is `false` (meaning failure).
	`partial`	Whether to allow creating partial snapshots. If set to `true`, data from successful shards is saved even if some shards fail. Default is `false`.

Example

Replace the following parameters with your actual information.

curl -X PUT "https://{pc-endpoint}:3001/_snapshot/{repo-name}/{snapshot-name}?wait_for_completion=true" \
  -u "{username}:{passwd}" \
  -H "Content-Type: application/json" \
  -d '{
    "indices": "{index-name}",
    "ignore_unavailable": false
  }'

View Snapshots

View all snapshot information in your OSS snapshot repository. Use the following API to retrieve the list:

GET /_snapshot/{repo-name}/_all?pretty

Example

Replace the following parameters with your actual information.

curl -X GET "https://{pc-endpoint}:3001/_snapshot/{repo-name}/_all?pretty" -u "{username}:{passwd}"

Restore Data

Run the following command to restore index data from the specified snapshot. Use the following API to perform the restore operation:

Note

To restore a snapshot from one PolarSearch cluster to another PolarSearch cluster, the destination PolarSearch cluster must register the same snapshot repository as the source PolarSearch cluster. If you perform the restore operation within the same PolarSearch cluster, re-registration is not required, and you can directly restore the data.

POST /_snapshot/{repo-name}/{snapshot-name}/_restore?wait_for_completion=true
{
  "indices": "{index-name}",
  "ignore_unavailable": true
}

Parameter description

Parameter category	Parameter name	Description
Request parameters	`wait_for_completion`	Whether to synchronously wait for the snapshot restoration to complete. Default is `false`. Synchronous execution: Set `wait_for_completion=true`. The command waits for the restoration to complete before returning. Asynchronous execution: Set `wait_for_completion=false`. The command returns immediately, and the restore job runs in the background. Check the index restoration progress using the following command. When the `stage` field is `DONE`, restoration is complete. `GET /{index-name}/_recovery`
Request body parameters	`indices`	Specify the indexes to restore. Wildcard characters (`*`) are supported. Separate multiple indexes with commas (`,`). Default is all indexes.
	`ignore_unavailable`	Whether to ignore the specified index if it does not exist and continue creating the snapshot. Default is `false` (meaning failure).
	`partial`	Whether to allow creating partial snapshots. If set to `true`, data from successful shards is saved even if some shards fail. Default is `false`.
	`index_settings`	Overwrite index settings in the snapshot during restoration. For example, modify the number of replicas to suit the destination cluster's configuration.
	`ignore_index_settings`	Specify a list of index settings to ignore during restoration, typically used to ignore original unique configurations.

Example

Replace the following parameters with your actual information.

curl -X POST "https://{pc-endpoint}:3001/_snapshot/{repo-name}/{snapshot-name}/_restore?wait_for_completion=true" \
  -u "{username}:{passwd}" \
  -H "Content-Type: application/json" \
  -d '{
    "indices": "{index-name}",
    "ignore_unavailable": true
  }'