Alibaba Cloud Elasticsearch provides the auto snapshot feature. This feature allows you to specify a snapshot creation interval and specific creation time. After you specify them, the system automatically creates snapshots based on the interval and time to ensure data security. These snapshots are called automatic snapshots. You can then quickly restore data from the snapshots to the Elasticsearch cluster in which the snapshots are created. This topic describes how to enable the auto snapshot feature and restore data from automatic snapshots.

Prerequisites

An Alibaba Cloud Elasticsearch cluster is created. For more information, see Create an Elasticsearch cluster.

Enable the auto snapshot feature

  1. Log on to the Alibaba Cloud Elasticsearch console.
  2. In the top navigation bar, select the region where your cluster resides.
  3. In the left-side navigation pane, click Elasticsearch Clusters. On the page that appears, find the target cluster and click its ID in the Cluster ID/Name column.
  4. In the left-side navigation pane of the page that appears, click Snapshots.
  5. In the Snapshots (Free Trial) section, turn on Auto Snapshot.
  6. Click Modify Configuration on the right side of Snapshots (Free Trial).
    Notice To ensure system security, the value of Frequency for Elasticsearch clusters of the Advanced Edition is generated by the system and cannot be changed.
  7. In the Auto Snapshot Configuration pane, set Frequency.
    Set an interval for automatic snapshot creation
    Frequency Description
    Every 30 Minutes The system creates snapshots at 30-minute intervals.
    Daily The system creates snapshots every day. You can customize the specific creation time.
    Custom The system creates snapshots based on the interval and time you specify.
    Notice The system uses the system time of the region where your Elasticsearch cluster resides to create snapshots.
  8. Click OK.

Restore data from automatic snapshots

If you enable the auto snapshot feature for an Elasticsearch cluster, the system creates snapshots for the cluster based on the snapshot creation interval and time that you specified. You can call the snapshot operation to restore data to the cluster from the created snapshots.
Note
  • The first snapshot is a full copy of the data in an Elasticsearch cluster. Subsequent snapshots store only incremental data. Therefore, it requires longer time to create the first snapshot than a subsequent snapshot.
  • Snapshots store only index data. The following information of your Elasticsearch cluster is not stored in snapshots: monitoring data (such as indexes whose names start with .monitoring or .security_audit), metadata, translogs, configurations, software packages, built-in and custom plug-ins, and logs.
  • You can restore data from automatic snapshots only to the Elasticsearch cluster in which the snapshots are created.
  • An automatic snapshot repository is created when the first snapshot is created.
  • If you restore indexes whose names start with . from snapshots, you may fail to access Kibana. Such indexes are system indexes. We recommend that you do not restore these indexes from snapshots.
  1. Log on to the Kibana console of your Elasticsearch cluster.
    For more information, see Log on to the Kibana console.
  2. In the left-side navigation pane, click Dev Tools. On the Console tab of the page that appears, run a command to perform a specific operation.
    • Query all snapshot repositories

      Run the GET _snapshot command to query all snapshot repositories.

      If the command is successfully executed, the following result is returned:
      {
        "aliyun_auto_snapshot": {
          "type": "oss",
          "settings": {
            "compress": "true",
            "base_path": "xxxx",
            "endpoint": "xxxx"
          }
        }
      }
      Parameter Description
      aliyun_auto_snapshot The name of the repository.
      type The storage where snapshots are stored. The value oss indicates that snapshots are stored in Object Storage Service (OSS).
      compress Indicates whether compression is used. The value true indicates that the metadata of indexes is compressed during snapshot creation.
      base_path The location of snapshots in OSS.
      endpoint The endpoint of the OSS bucket that stores the snapshots.
    • Query all snapshots

      Run the GET _snapshot/aliyun_auto_snapshot/_all command to query all snapshots in the aliyun_auto_snapshot repository.

      If the command is successfully executed, the following result is returned:
      {
        "snapshots": [ 
          {
            "snapshot": "es-cn-abcdefghij****_20180627091600",
            "uuid": "MMRniVLPRAiawSCm8D****",
            "version_id": 5050399,
            "version": "5.5.3",
            "indices": [
              "index_1",
              ".security",
              ".kibana"
            ],
            "state": "SUCCESS",
            "start_time": "2018-06-27T01:16:01.009Z",
            "start_time_in_millis": 1530062161009,
            "end_time": "2018-06-27T01:16:05.632Z",
            "end_time_in_millis": 1530062165632,
            "duration_in_millis": 4623,
            "failures": [],
            "shards": {
              "total": 12,
              "failed": 0,
              "successful": 12
            }
          }
        ]
      }
      Notice The system uses the system time of the current region to create snapshots. However, the time in the returned result is in UTC. A time zone difference exists between the system time and the time in the returned result. You can convert the time in the returned result based on the time zone difference. For example, the time zone difference between the system time of the China (Beijing) region and UTC is 8 hours. In this case, the system time of the China (Beijing) region is UTC+0080.
      The auto snapshot feature also supports the following default parameters that are not displayed.
      Parameter Description
      max_snapshot_bytes_per_sec:40mb The maximum speed for snapshot creation on a single node is 40 MB/s.
      max_restore_bytes_per_sec:40mb The maximum speed for data restoration on a single node is 40 MB/s.
      chunk_size: Max 1Gb During snapshot creation, a large index is divided into multiple parts. The maximum size of each part is 1 GB.
    • Restore indexes from a snapshot
      You can call the _restore operation to restore indexes from snapshots.
      • Run the following command to restore all indexes from a specific snapshot that is stored in the aliyun_auto_snapshot repository. The restoration task is executed at the backend.
        POST _snapshot/aliyun_auto_snapshot/<snapshot>/_restore

        Replace <snapshot> with the name of the specific snapshot, such as es-cn-abcdefghij****_20180627091600.

      • Run the following command to restore all indexes from a specific snapshot that is stored in the aliyun_auto_snapshot repository. Then, wait until the restoration task is completed.
        The restore operation asynchronously runs restoration tasks. An Elasticsearch cluster will immediately return a response if the restore operation is executable. The restoration task is executed at the backend. You can set the wait_for_completion parameter to enable the cluster to return a response only after the restoration task is completed.
        POST _snapshot/aliyun_auto_snapshot/<snapshot>/_restore?wait_for_completion=true

        Replace <snapshot> with the name of the specific snapshot, such as es-cn-abcdefghij****_20180627091600.

      • Run the following command to restore specified indexes from a specific snapshot that is stored in the aliyun_auto_snapshot repository, and rename the restored indexes. The restoration task is executed at the backend.
        POST _snapshot/aliyun_auto_snapshot/<snapshot>/_restore
        {
        "indices": "index_1",
        "rename_pattern": "index_(.+)",
        "rename_replacement": "restored_index_$1"
        }
        Parameter Description
        <snapshot> Replace it with the name of the specific snapshot, such as es-cn-abcdefghijklmn_20180627091600.
        indices The name of the index you want to restore.
        rename_pattern Optional. This parameter specifies the regular expression that is used to match the name of the index you want to restore.
        rename_replacement Optional. This parameter specifies the regular expression that is used to rename a matched index.