In replication mode, you can use the data download feature of a file gateway to download data from an OSS bucket to a local client.
Prerequisites
A file gateway is created, and a cache disk is attached to it. For more information, see Manage file gateways and Add a cache.
An OSS bucket is created. For more information, see Get started by using the OSS console.
ImportantFile gateways support the following storage classes of OSS buckets: Standard, Infrequent Access (IA), and Archive. File gateways do not support OSS buckets for which back-to-origin routing is configured.
We recommend that you do not associate a gateway with an Archive bucket. If files that are written from a file gateway to OSS are infrequently modified, we recommend that you store the files in a Standard or IA bucket first and configure a lifecycle rule that changes the storage class of the files to Archive or Cold Archive. This reduces unnecessary restoration operations and optimizes storage costs and efficiency.
When a client writes a file to a file gateway, the gateway records at least two actions: writing the file and setting the file modification time. The gateway merges the two actions where possible. However, the gateway may still initiate multiple operations on the object to the bucket where the object is stored. The CopyObject operation is called to store the file modification time as a piece of metadata of the object in the bucket. If the object is an Archive or Cold Archive object, this operation requires object restoration, which takes some time to complete. This increases the time required for object uploads and even causes upload failures if not enough time is left to upload data in the cache.
-
The file gateway uses reverse synchronization or express synchronization. For more information, see Express synchronization.
Background information
By default, a file gateway synchronizes metadata from an OSS bucket to a local client by using reverse synchronization or express synchronization. In replication mode, one copy of the data is stored on the local client and another copy is stored in the OSS bucket. The data download feature allows the file gateway to download file data from specified directories or all file data synchronized by reverse synchronization or express synchronization to the local client.
Downloading data over the internet incurs OSS fees. Use this feature as needed. For more information, see Billing overview.
Enable data download for a new share
-
On the Gateway List page of the Cloud Storage Gateway console, find and click the target file gateway, and click .
-
In the Create Share dialog box, configure the parameters in the Bucket Settings and Basic Information sections, and select Advanced Settings. For more information, see Create a share.
-
In the Advanced Settings section, configure the following parameters and click Next.
Parameter
Description
Mode
Select Replication Mode.
Reverse Sync
Turn on Reverse Sync. This feature synchronizes metadata from the OSS bucket to the local client and is suitable for gateway disaster recovery, data restoration, and data sharing scenarios.
Reverse synchronization is triggered only when a directory is accessed. To ensure that data in unaccessed directories and newly added data is downloaded promptly, use express synchronization. For more information, see Express synchronization.
Note-
Reverse synchronization scans all objects in the bucket. If the bucket contains a large number of objects, OSS API request fees are incurred. For more information, see the request fees on the Object Storage Service (OSS) pricing page.
-
If you select Add to Sync Group in the Basic Information section, this option is unavailable.
Reverse Sync Interval
The value must be in the range of 15s to 36000s.
NoteIf the bucket contains a large number of objects, set the reverse synchronization interval to more than 3,600s. Otherwise, frequent scans may generate high OSS API request fees.
Replication Mode Advanced Settings
Select Replication Mode Advanced Settings.
-
-
In the Replication Mode Advanced Settings section, configure the following parameters to enable Data Download.
Parameter
Description
Configure Directory in Replication Mode
This parameter specifies the scope of file data that runs in replication mode.
-
If this parameter is not selected, all data in the share runs in replication mode by default.
-
If this parameter is selected, click Add Directory and enter the path of the target data directory. Data in the specified directory path runs in replication mode, whereas data in unspecified paths runs in cache mode.
Note-
When you change a directory from cache mode to replication mode, the files in the directory are synchronized only if data download is enabled. Enable data download for replication mode at the same time.
-
The specified directory path is a relative path that is based on the root directory of the share. For example, if the absolute path of the directory that you want to enable for replication mode is /mnt/myshare/mydir/ and the mount point is /mnt/myshare, enter /mydir/.
Data Download
When data download is enabled, reverse synchronization or express synchronization downloads both metadata and file data. Otherwise, only metadata is synchronized.
Note-
To download file data in replication mode, the cache disk capacity must be at least 1.1 times the total size of the files to be replicated. Plan your cache disk capacity based on the expected growth in bucket usage.
-
Enabling data download for the first time triggers a full scan, which may affect gateway performance. Make sure that the gateway is idle when you enable this feature and remains idle until all data is downloaded.
-
The data download feature supports only single-writer, multiple-reader scenarios. If the corresponding bucket has multiple accessors (such as gateways and direct OSS access), only one accessor can upload files to the bucket. Other accessors can only download files. Data loss may occur in multiple-writer, multiple-reader scenarios. Use this feature with caution.
Download Speed Limit
Configure this parameter when you enable Data Download. The download speed limit must be between 0 MB/s and 1280 MB/s, inclusive. A value of 0 MB/s means the download speed is unlimited.
Reverse Sync Interval
Configure this parameter when you enable Data Download. The value must be in the range of 3600s to 36000s.
Note-
If the bucket contains a large number of objects, set the reverse synchronization interval to more than 3,600s. Otherwise, frequent scans may generate high OSS API request fees.
-
Reverse synchronization is triggered only when a directory is accessed. To ensure that data in unaccessed directories and newly added data is downloaded, use express synchronization.
-
Enable data download for an existing share
-
In the CSG console, on the Gateways page, find and click the target file gateway.
-
On the Shares tab, find the target share and click Advanced Settings in the Actions column.
-
On the Share Advanced Settings page, configure the following parameters to enable Data Download.
Parameter
Description
Mode
You can use the Data Download feature only in Replication Mode.
Reverse Sync
Turn on Reverse Sync. This feature synchronizes metadata from the OSS bucket to the local client and is suitable for gateway disaster recovery, data restoration, and data sharing scenarios.
Reverse synchronization is triggered only when a directory is accessed. To ensure that data in unaccessed directories and newly added data is downloaded promptly, use express synchronization. For more information, see Express synchronization.
Note-
Reverse synchronization scans all objects in the bucket. If the bucket contains a large number of objects, OSS API request fees are incurred. For more information, see the request fees on the Object Storage Service (OSS) pricing page.
-
If you select Add to Sync Group in the Basic Information section, this option is unavailable.
Data Download
When data download is enabled, reverse synchronization or express synchronization downloads both metadata and file data. Otherwise, only metadata is synchronized.
Note-
To download data, the cache disk capacity must be at least 1.1 times the total size of the files to be replicated. Plan your cache disk capacity based on the expected growth in bucket usage.
-
Enabling data download for the first time triggers a full scan, which may affect gateway performance. Make sure that the gateway is idle when you enable this feature and remains idle until all data is downloaded.
-
The data download feature supports only single-writer, multiple-reader scenarios. If the corresponding bucket has multiple accessors (such as gateways and direct OSS access), only one accessor can upload files to the bucket. Other accessors can only download files. Data loss may occur in multiple-writer, multiple-reader scenarios. Use this feature with caution.
Download Speed Limit
Configure this parameter when you enable Data Download. The download speed limit must be between 0 MB/s and 1280 MB/s, inclusive. A value of 0 MB/s means the download speed is unlimited.
-