All Products
Search
Document Center

Cloud Storage Gateway:Usage notes

Last Updated:Jan 31, 2024

Before you use Cloud Storage Gateway (CSG), we recommend that you read the following usage notes.

File gateways

  • We recommend that you do not frequently interrupt the upload of large files to Network File System (NFS) or Server Message Block (SMB) shares. The system uploads files by using multipart upload. If you interrupt the upload of large files, parts are generated in the associated Object Storage Service (OSS) bucket. These parts occupy the capacity of the OSS bucket. Therefore, the storage usage of the OSS bucket is slightly higher than the total file size. You can configure OSS to automatically delete parts. For more information, see Delete parts.

  • The cache capacity of a share is calculated based on the following formula: Recommended local cache capacity = [Application bandwidth (MB/s) - Backend bandwidth of a gateway (MB/s)] × Write duration (seconds) × 1.2.

    To obtain better performance when you access data from local clients, you can estimate the total amount of hot data. Compare the total amount of hot data with the recommended on-premises cache capacity, and select the higher value as the capacity of the on-premises cache disk.

  • If you want to write a large file by using a file gateway, the size of the file must be smaller than 30% of the cache disk capacity. You cannot write multiple large files at the same time. If you write multiple large files at the same time, the cache disk space may be easily exhausted.

  • A file gateway in version 1.0.37 or earlier supports up to 1.2 TB of a file. Files that are larger than 1.2 TB cannot be uploaded to OSS. A file gateway in version 1.0.38 or later supports up to 30 TB of a file. If you upload a file that is larger than 2 TB, we recommend that you have an Internet bandwidth of 500 MB/s or higher or connect to Alibaba Cloud over an Express Connect circuit. Otherwise, a timeout error may occur.

  • File gateways support sparse files. If a sparse file fails to be uploaded to a file gateway, run the following command to convert the format of the sparse file:

    dd if=<sparse file name> of=<sparse file name> conv=notrunc bs=1M

    The size of the sparse file cannot exceed the available capacity of the cache disk.

  • The names of file gateways and directories must be encoded in UTF-8. File gateways do not support file and directory names that are encoded in formats other than UTF-8. For example, if you mount an NFS share of a file gateway on a Windows client, the files and directories whose names contain Chinese characters cannot be created. In this case, a 0x8007045D error is reported.

  • If the size of a file in a file gateway exceeds 256 MB, we recommend that you disable versioning for the associated OSS bucket. Otherwise, a timeout error may occur when the gateway uploads metadata to the associated bucket. This degrades the performance of the gateway.

  • File gateways implement permission isolation on Windows Active Directory (AD) based on POSIX Access Control Lists (ACLs). File gateways do not allow you to authorize multiple AD users across directories. For example, the AA/BB/CC directory belongs to User 1. If you authorize User 2 to access only the CC directory, User 2 cannot access the data in the CC directory from the AA/BB/CC directory. In this scenario, you must also authorize User 2 to access the AA and BB directories.

  • When the associated bucket stores more than one million files, we recommend that you set the intervals of reverse synchronization to longer than 3,600 seconds.

  • For file gateways version 1.0.36 and later, a Multipurpose Internet Mail Extensions (MIME) type is automatically specified in the OSS metadata based on the file suffix.

  • If reverse synchronization is enabled, empty on-premises directories that are not uploaded to Alibaba Cloud may be deleted by reverse synchronization during a scan cycle. To address this issue, we recommend that you create the directories again.

  • By default, you can rename a directory whose total number of subdirectories and files is no more than 5 million. If you want to rename a directory that contains more than 5 million subdirectories and files, submit a ticket.

  • When reverse synchronization is enabled, some directory rename operations may fail due to incomplete metadata cache on the gateway side.

File gateways deployed on Alibaba Cloud

  • The CSG console uses the HTTPS protocol. Network storage protocols such as NFS and SMB require special ports. Therefore, you must configure a firewall or security group rules for the CSG console to support these ports.

    • CSG supports AD and Lightweight Directory Access Protocol (LDAP) domains. Therefore, you must configure specific ports to support the following protocols: LDAP, AD, Domain Name System (DNS), and Kerberos. A security group rule specifies CIDR blocks and permissions. For more information, see Add a security group rule.

      In a virtual private cloud (VPC) network, if a gateway and a domain server belong to different security groups of an Alibaba Cloud account, you can configure security group rules. For example, if you authorize connections between a security group for a gateway and a security gateway for a domain server, you can include TCP 53/636 and UDP 53/636 in the rule for the domain server.

    • To support NFS and SMB, configure the corresponding service ports that are listed in the following table in the inbound rule of the security group of CSG. After you create a cloud file gateway in the CSG console, the service ports are configured for the security group by default. You need to configure ports for LDAP and AD in the inbound rules of the security group on the domain server.

      Protocol

      Port

      HTTPS

      443 and 8080

      NFS

      111 (UDP and TCP), 875 (UDP and TCP), 892 (UDP and TCP), 2049 (UDP and TCP), 32887 (UDP and TCP), 32888 (UDP and TCP), and 32889 (UDP and TCP)

      SMB

      137 (UDP), 138 (UDP), 139 (TCP), 389 (TCP), 445 (TCP), and 901 (TCP)

      SSH

      22

      LDAP

      389 (UDP and TCP) and 636 (UDP)

      AD

      445 (UDP and TCP)

      DNS

      53 (UDP and TCP)

      Kerberos

      88 (UDP and TCP)

  • The synchronization bandwidth of a gateway is related to the bandwidth of OSS. OSS supports a maximum bandwidth of 10 Gbit/s. The bandwidth slightly varies among clusters in different regions. For more information, submit a ticket.

  • After you create a file gateway on Alibaba Cloud, a security group prefixed with Cloud_Storage_Gateway_Usage is configured for the gateway by default. Do not use this security group when you create Elastic Compute Service (ECS) instances.

  • By default, the upload bandwidth of gateways deployed on Alibaba Cloud is 1 Mbit/s. These gateways access OSS buckets across regions over the Internet. As a result, the data transmission performance may be unstable.

On-premises file gateways

  • To use on-premises file gateways, you must open the following ports in the firewall of your client.

    Protocol

    Port

    HTTPS

    443

    NFS

    111 (UDP and TCP), 875 (UDP and TCP), 892 (UDP and TCP), 2049 (UDP and TCP), 32887 (UDP and TCP), 32888 (UDP and TCP), and 32889 (UDP and TCP)

    SMB

    137 (UDP), 138 (UDP), 139 (TCP), 389 (TCP), 445 (TCP), and 901 (TCP)

    SSH

    22

    LDAP

    389 (UDP and TCP) and 636 (UDP)

    AD

    445 (UDP and TCP)

    DNS

    53 (UDP and TCP)

    Kerberos

    88 (UDP and TCP)

iSCSI gateways

  • The cache capacity of Internet Small Computer Systems Interface (iSCSI) volumes is calculated based on the following formula: Recommended on-premises cache capacity = [Application bandwidth (MB/s) - Backend bandwidth of the gateway (MB/s)] × Write duration (seconds) × 1.2.

    To obtain better performance when you access data from local clients, you can estimate the total amount of hot data. Compare the total amount of hot data with the recommended on-premises cache capacity, and select the higher value as the capacity of the on-premises cache disk.

  • The synchronization bandwidth of an iSCSI gateway is related to OSS bandwidth. OSS supports a maximum bandwidth of 10 Gbit/s. The bandwidth slightly varies among clusters in different regions. For more information, contact OSS technical support for the region where your OSS buckets reside.

  • The default input/output operations per second (IOPS) are subject to the backend disk capacity. An ultra disk supports a maximum bandwidth of 110 MB/s. An SSD disk supports a maximum bandwidth of 230 MB/s.

  • To use iSCSI gateways, you must open the following ports in the firewall of your client.

    • iSCSI gateways deployed on Alibaba Cloud

      Protocol

      Port

      iSCSI

      860 (TCP) and 3260 (TCP)

    • On-premises iSCSI gateways

      Protocol

      Port

      HTTPS

      443

      iSCSI

      860 (TCP) and 3260 (TCP)