How to schedule automatic snapshot backups on Alibaba Cloud Elasticsearch - Elasticsearch

Alibaba Cloud Elasticsearch offers two methods to automate snapshot backups. For clusters running version 7.6 or later, you can use snapshot lifecycle management (SLM) to schedule snapshots and manage their retention. For versions earlier than 7.6, you must use a client-side crontab job to trigger snapshot creation. Both methods store snapshots in an OSS repository.

Background

For more information about snapshot lifecycle management, see Snapshot lifecycle management.
Data backup and restoration in Elasticsearch depend on the elasticsearch-repository-oss plugin. This plugin comes pre-installed on Alibaba Cloud Elasticsearch clusters and cannot be removed. For more information about the plugin, see elasticsearch-repository-oss.

Versions 7.6 and later (Recommended)

Before you begin, complete the following prerequisites:

Create an OSS bucket of the Standard storage class. Archive-class buckets are not supported. The bucket must be in the same region as your Elasticsearch cluster. For instructions, see Create a bucket.
Create a repository to associate the OSS bucket with your Elasticsearch cluster. For instructions, see Create a repository.

In the Kibana console of your cluster, click Dev Tools and follow these steps.

Create an SLM policy.

PUT _slm/policy/auto-snapshots
{
  "schedule": "0 0 0/12 * * ?",
  "name": "<auto-snap-{now/d}>",
  "repository": "my_auto_backup",
  "config": {
    "indices": "*",
    "include_global_state": true
  },
  "retention": {
    "expire_after": "30d",
    "min_count": 5,
    "max_count": 50
  }
}

Parameter	Description
schedule	The format of a Cron expression is: Second Minute Hour Day of month Month Day of week Year (optional). `"0 0 0/12 * * ?"` creates a snapshot every 12 hours. For more information, see Cron.
name	The format for the snapshot name. You can use date math expressions, such as `<auto-snap-{now/d}>`, for date-based naming.
repository	The name of the snapshot repository. To find the repository name, see Get repository information.
config.indices	The indexes included in the snapshot. `*` indicates all indexes.
config.include_global_state	Specifies whether to include the cluster status and feature status. `true` includes them, and `false` does not.
retention.expire_after	The retention period for snapshots. The example sets this to 30 days.
retention.min_count	The minimum number of snapshots to retain. This minimum count is preserved even if the snapshots are older than the `expire_after` period.
retention.max_count	The maximum number of snapshots to retain. If this limit is exceeded, the oldest snapshots are deleted, regardless of their age.

Execute the SLM policy to create a snapshot immediately.
```
POST _slm/policy/auto-snapshots/_execute
```
After the command runs, Elasticsearch automatically creates snapshots based on the schedule parameter. To run the retention policy immediately, execute the following command:
```
POST _slm/_execute_retention
```

Versions earlier than 7.6

Versions earlier than 7.6 do not support SLM. You must use a client-side crontab job to automate snapshots.

Before you begin, complete the following prerequisites:

Create an OSS bucket of the Standard storage class. Archive-class buckets are not supported. The bucket must be in the same region as your Elasticsearch cluster. For instructions, see Create a bucket.
Create a repository. For instructions, see Create a repository.
Configure a client that can access the Elasticsearch cluster over the public network or an internal network. For more information, see Use a client to access an Alibaba Cloud Elasticsearch cluster.

On your client server, follow these steps:

Create a snapshot backup script.
```
vi /root/snapshot.sh
```
Add the following content to the script file and save it. Replace elastic:***** with the actual username and password, and es-***** with the actual cluster ID.
```
curl -u elastic:***** -X PUT https://es-*****.public.elasticsearch.aliyuncs.com:9200/_snapshot/my_auto_backup_crontab/snapshot_$(date +%s)
```
Grant execute permissions to the script.
```
chmod +x /root/snapshot.sh
```
Configure a crontab job. The following example sets the job to run daily at 2:00 AM.
```
crontab -e
```
Add the following line:
```
0 2 * * * /bin/bash /root/snapshot.sh
```
After you save the file, crontab automatically loads the new configuration. You do not need to restart the cron service.

After backing up snapshots, you can delete them, restore them, or view their details. For more information, see Manual backup and restoration. To restore a snapshot to a different cluster, you must create a repository in the target cluster that points to the same OSS bucket.