All Products
Search
Document Center

Cloud Storage Gateway:Replicate data

Last Updated:Nov 09, 2023

If a file gateway is set to replication mode, you can use the data replication feature to replicate data in the associated Object Storage Service (OSS) bucket to the local client.

Prerequisites

Background information

By default, a file gateway synchronizes metadata in the associated OSS bucket to the local client through remote sync or express sync. In replication mode, two backups are created. One is stored in the OSS bucket, and the other is stored on the local client. Cloud Storage Gateway (CSG) provides the data replication feature to replicate data synchronized through remote sync or express sync from a file gateway to the local client. This feature allows you to replicate data under specific paths or all data to the local client.

Data transmission over the Internet incurs OSS service fees. Use this feature based on your actual needs. For more information about the pricing, see Billing overview.

Create a shared directory and enable data replication

The following procedure shows how to create a shared directory and enable data replication.

  1. Log on to the CSG console.

  2. Select the region where the target file gateway is deployed.

  3. On the Gateway Clusters page, click the target file gateway.

  4. In the left-side navigation pane, click Share, and then click Create.

  5. On the Bucket Setting tab, set the parameters and click Next. For more information, see the descriptions of bucket parameters in Manage a file gateway in the CSG console.

  6. On the Basic Information tab, set the parameters and click Next. For more information, see the descriptions of basic share parameters in Manage a file gateway in the CSG console.

  7. On the Advanced Settings tab, set the parameters and click Next. For more information, see the descriptions of advanced settings in Manage a file gateway in the CSG console.

    Parameter

    Description

    Mode

    Select the Replication Mode.

    Enable Remote Sync

    Select Yes to Enable Remote Sync. This feature synchronizes metadata in the associated OSS bucket to the local client. This feature is suitable for use in gateway disaster recovery, data restoration, and data sharing scenarios.

    Remote sync is triggered only when the shared directory is accessed. To make sure that existing and incremental data in the shared directory can be synchronized to the local client when no user accesses the directory, we recommend that you enable express sync. For more information, see Configure express synchronization.

    Note
    • During remote synchronization, the system scans all objects in the bucket. If the number of objects is large, fees are incurred for calling the OSS API. For more information, see Pricing of OSS.

    • If you have selected the Join Sync Group check box on the Basic Information tab, this parameter is unavailable.

    Remote Sync Time Interval

    Valid values: 15 to 36000. Default value: 36000. Unit: seconds.

    Note

    If the bucket contains a large number of objects, we recommend that you set the interval to longer than 3,600 seconds. Otherwise, repeated scans frequently call the OSS API, incurring a large amount of fees.

    Advanced Settings

    Select the Advanced Settings check box.

  8. On the Advanced Settings tab, set the following parameters and click Next.

    Parameter

    Description

    Configure Directory in Replication Mode

    Allows you to specify the files to which the replication mode applies.

    • If you do not select this check box, the replication mode applies to all data in the share.

    • After you select the check box, click Add Directory to specify a directory that contains the data to which you want to apply the replication mode. The replication mode is applied to data in the specified directory. The rest of the data uses the cache mode.

    Note
    • If you change the mode of a directory from cache to replication, the files in the directory can be synchronized only when the data download feature is also enabled. We recommend that you enable the data download feature in replication mode.

    • You can specify relative directories under the root directory of the share. For example, if the directory to which you want to apply the replication mode is /mnt/myshare/mydir/ and the mount point is /mnt/myshare, enter /mydir/.

    Data Download

    By default, the reverse synchronization and express synchronization features synchronize the metadata between the OSS bucket and the on-premises cache disk. The data download feature allows you to download data from OSS to the local cache disk. After you enable the Reverse Sync or Express Sync feature, you can set Data Download to Yes.

    Note
    • If you download data in replication mode, the capacity of the cache disk must be 10% larger than the size of the files that you want to replicate. Plan the cache capacity based on the expected growth of the bucket usage.

    • When you enable the data download feature for the first time, a full scan is triggered. This process may reduce the performance of the gateway. We recommend that you enable the data download feature during off-peak hours and wait for the system to replicate all data.

    • The data download feature supports only write-once-read-many storage. If multiple users access the bucket at the same time (including but not limited to direct access or access through the gateway), only one user can upload files to the bucket. Other users can only download data. Data loss may occur if multiple users write data to and read data from the bucket at the same time.

    Download Speed Limit

    This parameter is available only if you enable the Data Download feature in replication mode. The download speed must be in the range of 0 MB/s to 1,280 MB/s. If you set this parameter to 0 MB/s, the download speed is unlimited.

    Reverse Sync Interval

    This parameter is available only if you enable the Data Download feature in replication mode. Valid values: 3600 to 36000. Default value: 36000. Unit: seconds.

    Note
    • If the bucket contains a large number of objects, we recommend that you specify an interval that is greater than 3,600 seconds. If the interval is less than 3,600 seconds, repeated scans are performed, which results in frequent OSS API calls and consequently increased API request fees.

    • Reverse synchronization is triggered only when you access the directory. To make sure that the data in other directories can be downloaded and new data can be downloaded in real time, we recommend that you use express synchronization.

  9. On the Summary tab, make sure that the specified information is correct, and then click OK.

Enable data replication for an existing shared directory

The following procedure shows how to enable data replication for an existing shared directory.

  1. Log on to the CSG console.

  2. In the upper-left corner, select the region where the target file gateway is deployed.

  3. On the Gateway Clusters page, click the target gateway.

  4. In the left-side navigation pane, click Share. Find the target share in the share list, and then click Advanced Set in the Actions column.

  5. In the Advanced Setting dialog box that appears, set the following parameters. For more information, see the descriptions of advanced settings in Manage shares.

    Parameter

    Description

    Mode

    Data replication is supported by replication mode only.

    Enable Remote Sync

    This feature synchronizes metadata in the associated OSS bucket to the local client. This feature is suitable for use in disaster recovery, data restoration, and data sharing scenarios.

    Remote sync is triggered only when the shared directory is accessed. To make sure that existing and incremental data in the shared directory can be synchronized to the local client when no user accesses the directory, we recommend that you enable express sync. For more information, see Configure express synchronization.

    Note
    • During remote synchronization, the system scans all objects in the bucket. If the number of objects is large, fees are incurred for calling the OSS API. For more information, see Pricing of OSS.

    • If you have selected the Join Sync Group check box on the Basic Information tab, this parameter is unavailable.

    Data Replication

    By default, the remote sync and the express sync features synchronize the metadata between the OSS bucket and the local client. The data replication feature allows you to replicate data under specific paths or all data to the local client. If Remote Sync or Configure express synchronization is enabled, you can set Data Replication to Yes to enable data replication.

    Note
    • Data replication requires the capacity of the cache disk to be 1.1 times larger than the file size to be replicated. Specify the cache capacity properly based on the expected growth of the bucket usage.

    • If this is your first time enabling data replication, a full scan is triggered. This process may reduce the performance of the gateway. Enable data replication during off-peak hours and wait for the system to replicate all the data.

    • Data replication allows only a single user to write data to the bucket and multiple users to read data from the bucket simultaneously. If multiple users access the bucket at the same time (whether through the gateway or direct access to the OSS bucket), only a single user is allowed to upload files to the bucket, and other users can only download data. Data loss may occur if multiple users write data to the bucket while multiple users read data from the bucket at the same time. Proceed with caution.

    Download speed limit

    This parameter is required if Data Replication is enabled. The download speed must not be lower than 0 MB/s and not be higher than 1,280 MB/s. If you set this parameter to 0 MB/s, it indicates that the download speed is not limited.

  6. Verify that the specified information is correct and click OK.