Back Up and Restore PolarSearch Data to OSS for Disaster Recovery - PolarDB

PolarSearch lets you back up index data to an Object Storage Service (OSS) bucket and restore it using snapshots. Use this feature to migrate data across clusters or implement custom backup and recovery workflows.

This feature is in canary release. Submit a ticket to request access.

This guide covers:

Registering a snapshot repository backed by OSS
Creating a snapshot to back up index data
Viewing snapshots in a repository
Restoring index data from a snapshot

Prerequisites

Before you begin, make sure you have:

A PolarSearch cluster running version 2.19.3 or later. To check the version, access PolarSearch using the endpoint and look up the version.number field.
An activated Alibaba Cloud Object Storage Service (OSS) account and a bucket created to store snapshots.
A Resource Access Management (RAM) user with the AliyunOSSFullAccess access policy. See Grant permissions to a RAM user.

Billing

Snapshots are free. Storing snapshot files in OSS incurs storage and request fees. See OSS billing overview for details.

Register a snapshot repository

Before creating snapshots, register a snapshot repository and associate it with your OSS bucket.

PUT /_snapshot/{repo-name}
{
    "type": "oss",
    "settings": {
      "endpoint": "{endpoint}",
      "bucket": "{bucket-name}",
      "base_path": "{path-name}",
      "region": "{region}",
      "access_key": "{your-AccessKey-ID}",
      "secret_key": "{your-AccessKey-secret}",
      "session_token": "{your-STS-token}"
    }
}

Parameters

Parameter	Required	Description
`{repo-name}`	Yes	Repository name. Define your own.
`type`	Yes	Repository type. Must be `oss`.
`endpoint`	Yes	The endpoint of your OSS bucket. See Regions and endpoints.
`bucket`	Yes	Your OSS bucket name.
`region`	Yes	The region where the bucket resides.
`access_key`	Yes	Your AccessKey ID.
`secret_key`	Yes	Your AccessKey secret.
`base_path`	No	The root directory path in the OSS bucket where snapshot files are stored.
`session_token`	No	A Security Token Service (STS) token for temporary identity credentials. If you use an STS token, set both `access_key` and `secret_key` to the values provided by STS. Without temporary credentials, use your long-term AccessKey ID and secret and omit `session_token`.

Example

curl -X PUT "https://{pc-endpoint}:3001/_snapshot/{repo-name}" \
  -u "{username}:{passwd}" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "oss",
    "settings": {
      "endpoint": "{endpoint}",
      "bucket": "{bucket-name}",
      "base_path": "{path-name}",
      "region": "{region}",
      "access_key": "{your-AccessKey-ID}",
      "secret_key": "{your-AccessKey-secret}"
    }
  }'

Create a snapshot

After registering a repository, create a snapshot to back up specific indices.

PUT /_snapshot/{repo-name}/{snapshot-name}?wait_for_completion=true
{
    "indices": "{index-name}",
    "ignore_unavailable": false
}

Parameters

Parameter	Type	Description
`wait_for_completion`	Request parameter	Controls synchronous versus asynchronous execution. Default is `false` (asynchronous). Set to `true` to wait for the snapshot to finish before the command returns.
`indices`	Request body	Indices to back up. Supports the wildcard character `` and comma-separated lists. Defaults to all indices. Note* Using `*` includes system indices. Specify index names explicitly, or use `-` to exclude system indices.
`ignore_unavailable`	Request body	Whether to skip missing indices and continue. Default is `false`, which causes the operation to fail if an index is missing.
`partial`	Request body	Whether to save a partial snapshot when some shards fail. Default is `false`. Set to `true` to save data from successful shards even if others fail.

To check the status of an asynchronous snapshot, run the following command. When state shows SUCCESS, the snapshot is complete.

GET /_snapshot/{repo-name}/{snapshot-name}/_status

Example

curl -X PUT "https://{pc-endpoint}:3001/_snapshot/{repo-name}/{snapshot-name}?wait_for_completion=true" \
  -u "{username}:{passwd}" \
  -H "Content-Type: application/json" \
  -d '{
    "indices": "{index-name}",
    "ignore_unavailable": false
  }'

View snapshots

List all snapshots in a repository.

GET /_snapshot/{repo-name}/_all?pretty

Example

curl -X GET "https://{pc-endpoint}:3001/_snapshot/{repo-name}/_all?pretty" -u "{username}:{passwd}"

Restore data

Restore index data from a snapshot using the following command.

To restore a snapshot to a different PolarSearch cluster, the destination cluster must register the same snapshot repository as the source cluster. For same-cluster restores, skip this step and restore directly.

POST /_snapshot/{repo-name}/{snapshot-name}/_restore?wait_for_completion=true
{
  "indices": "{index-name}",
  "ignore_unavailable": true
}

Parameters

Parameter	Type	Description
`wait_for_completion`	Request parameter	Controls synchronous versus asynchronous execution. Default is `false` (asynchronous). Set to `true` to wait for the restore to finish before the command returns.
`indices`	Request body	Indices to restore. Supports the wildcard character `*` and comma-separated lists. Defaults to all indices.
`ignore_unavailable`	Request body	Whether to skip missing indices and continue. Default is `false`.
`partial`	Request body	Whether to restore from a partial snapshot when some shards failed. Default is `false`.
`index_settings`	Request body	Overrides index settings from the snapshot during restore. Use this to adjust replica count to match the destination cluster.
`ignore_index_settings`	Request body	A list of index settings to ignore during restore, typically used to skip source-specific configurations.

To check the progress of an asynchronous restore, run the following command. When stage shows DONE, the restore is complete.

GET /{index-name}/_recovery

Example

curl -X POST "https://{pc-endpoint}:3001/_snapshot/{repo-name}/{snapshot-name}/_restore?wait_for_completion=true" \
  -u "{username}:{passwd}" \
  -H "Content-Type: application/json" \
  -d '{
    "indices": "{index-name}",
    "ignore_unavailable": true
  }'