In replication mode, you can use the data download feature of a file gateway to download data from an OSS bucket to a local client.
Prerequisites
A file gateway is created and a cache is added.
An OSS Bucket is created.
ImportantA file gateway supports only OSS Buckets of the Standard, Infrequent Access (IA), and Archive storage classes. OSS Buckets with the back-to-source feature enabled are not supported.
Avoid connecting a gateway directly to an Archive OSS Bucket. If files will not be modified for a long time after being written, consider using a Standard or IA bucket. You can then use lifecycle rules to transition the storage class of the objects to Archive or Cold Archive. This approach is more cost-effective and efficient because it avoids unnecessary restore operations.
When a client writes a file to the gateway, the gateway records at least two actions: writing the file and setting the file's modification time. The gateway attempts to merge these actions, but may still initiate multiple operations on the OSS Bucket for a single file. Setting the modification time translates to modifying the object's metadata in the OSS Bucket by using the CopyObject API. For Archive or Cold Archive objects, this requires a time-consuming restore operation, which can slow down uploads and even cause write failures if the cache disk does not have enough space to upload the data in time.
-
The file gateway uses reverse synchronization or express synchronization. For more information, see Express synchronization.
Background information
By default, a file gateway synchronizes metadata from an OSS bucket to a local client by using reverse synchronization or express synchronization. In replication mode, one copy of the data is stored on the local client and another copy is stored in the OSS bucket. The data download feature allows the file gateway to download file data from specified directories or all file data synchronized by reverse synchronization or express synchronization to the local client.
Downloading data over the internet incurs OSS fees. Use this feature as needed. For more information, see Billing overview.
Enable data download for a new share
-
On the Gateway List page of the Cloud Storage Gateway console, find and click the target file gateway, and click .
-
In the Create Share dialog box, configure the parameters in the Bucket Settings and Basic Information sections, and select Advanced Settings. For more information, see Create a share.
-
In the Advanced Settings section, configure the following parameters and click Next.
Parameter
Description
Mode
Select Replication Mode.
Reverse Sync
Turn on Reverse Sync. This feature synchronizes metadata from the OSS bucket to the local client and is suitable for gateway disaster recovery, data restoration, and data sharing scenarios.
Reverse synchronization is triggered only when a directory is accessed. To ensure that data in unaccessed directories and newly added data is downloaded promptly, use express synchronization. For more information, see Express synchronization.
Note-
Reverse synchronization scans all objects in the bucket. If the bucket contains a large number of objects, OSS API request fees are incurred. For more information, see the request fees on the Object Storage Service (OSS) pricing page.
-
If you select Add to Sync Group in the Basic Information section, this option is unavailable.
Reverse Sync Interval
The value must be in the range of 15s to 36000s.
NoteIf the bucket contains a large number of objects, set the reverse synchronization interval to more than 3,600s. Otherwise, frequent scans may generate high OSS API request fees.
Replication Mode Advanced Settings
Select Replication Mode Advanced Settings.
-
-
In the Replication Mode Advanced Settings section, configure the following parameters to enable Data Download.
Parameter
Description
Configure Directory in Replication Mode
This parameter specifies the scope of file data that runs in replication mode.
-
If this parameter is not selected, all data in the share runs in replication mode by default.
-
If this parameter is selected, click Add Directory and enter the path of the target data directory. Data in the specified directory path runs in replication mode, whereas data in unspecified paths runs in cache mode.
Note-
When you change a directory from cache mode to replication mode, the files in the directory are synchronized only if data download is enabled. Enable data download for replication mode at the same time.
-
The specified directory path is a relative path that is based on the root directory of the share. For example, if the absolute path of the directory that you want to enable for replication mode is /mnt/myshare/mydir/ and the mount point is /mnt/myshare, enter /mydir/.
Data Download
When data download is enabled, reverse synchronization or express synchronization downloads both metadata and file data. Otherwise, only metadata is synchronized.
Note-
To download file data in replication mode, the cache disk capacity must be at least 1.1 times the total size of the files to be replicated. Plan your cache disk capacity based on the expected growth in bucket usage.
-
Enabling data download for the first time triggers a full scan, which may affect gateway performance. Make sure that the gateway is idle when you enable this feature and remains idle until all data is downloaded.
-
The data download feature supports only single-writer, multiple-reader scenarios. If the corresponding bucket has multiple accessors (such as gateways and direct OSS access), only one accessor can upload files to the bucket. Other accessors can only download files. Data loss may occur in multiple-writer, multiple-reader scenarios. Use this feature with caution.
Download Speed Limit
Configure this parameter when you enable Data Download. The download speed limit must be between 0 MB/s and 1280 MB/s, inclusive. A value of 0 MB/s means the download speed is unlimited.
Reverse Sync Interval
Configure this parameter when you enable Data Download. The value must be in the range of 3600s to 36000s.
Note-
If the bucket contains a large number of objects, set the reverse synchronization interval to more than 3,600s. Otherwise, frequent scans may generate high OSS API request fees.
-
Reverse synchronization is triggered only when a directory is accessed. To ensure that data in unaccessed directories and newly added data is downloaded, use express synchronization.
-
Enable data download for an existing share
-
In the CSG console, on the Gateways page, find and click the target file gateway.
-
On the Shares tab, find the target share and click Advanced Settings in the Actions column.
-
On the Share Advanced Settings page, configure the following parameters to enable Data Download.
Parameter
Description
Mode
You can use the Data Download feature only in Replication Mode.
Reverse Sync
Turn on Reverse Sync. This feature synchronizes metadata from the OSS bucket to the local client and is suitable for gateway disaster recovery, data restoration, and data sharing scenarios.
Reverse synchronization is triggered only when a directory is accessed. To ensure that data in unaccessed directories and newly added data is downloaded promptly, use express synchronization. For more information, see Express synchronization.
Note-
Reverse synchronization scans all objects in the bucket. If the bucket contains a large number of objects, OSS API request fees are incurred. For more information, see the request fees on the Object Storage Service (OSS) pricing page.
-
If you select Add to Sync Group in the Basic Information section, this option is unavailable.
Data Download
When data download is enabled, reverse synchronization or express synchronization downloads both metadata and file data. Otherwise, only metadata is synchronized.
Note-
To download data, the cache disk capacity must be at least 1.1 times the total size of the files to be replicated. Plan your cache disk capacity based on the expected growth in bucket usage.
-
Enabling data download for the first time triggers a full scan, which may affect gateway performance. Make sure that the gateway is idle when you enable this feature and remains idle until all data is downloaded.
-
The data download feature supports only single-writer, multiple-reader scenarios. If the corresponding bucket has multiple accessors (such as gateways and direct OSS access), only one accessor can upload files to the bucket. Other accessors can only download files. Data loss may occur in multiple-writer, multiple-reader scenarios. Use this feature with caution.
Download Speed Limit
Configure this parameter when you enable Data Download. The download speed limit must be between 0 MB/s and 1280 MB/s, inclusive. A value of 0 MB/s means the download speed is unlimited.
-