Use GA to accelerate access to OSS buckets - Global Accelerator

When users access resources that are stored in Object Storage Service (OSS) buckets, the speed of data transfer is limited by the downstream bandwidth of OSS and the region where the OSS buckets are deployed. Global Accelerator (GA) allows clients to connect to the nearest access point of the Alibaba Cloud global transmission network. You can use GA to accelerate data transfer and improve user experience. This topic describes how to use GA to accelerate access to OSS buckets.

Scenarios

The following scenario is used as an example in this topic. The headquarters of a company is located in the US (Virginia) region. The headquarters stores a large number of files in OSS. When users from the office in the China (Hong Kong) region access the resources that are stored in OSS in the US (Virginia) region over the Internet, the network condition is unstable. Issues such as network latency, network jitter, and packet loss occur.

You can deploy a GA instance and specify the OSS bucket as an endpoint. GA uses high-bandwidth Border Gateway Protocol (BGP) lines and the global transmission network of Alibaba Cloud to accelerate data transfer when clients in the China (Hong Kong) region access the resources that are stored in OSS in the US (Virginia) region.

Prerequisites

OSS is activated in the US (Virginia) region. Resources are stored in an OSS bucket that you create.

Procedure

Note

This topic uses a pay-as-you-go standard Global Accelerator instance as an example to describe how to configure Global Accelerator to accelerate access to OSS resources. Before you create a pay-as-you-go standard Global Accelerator instance, take note of the following information:

GA instances use the pay-by-data-transfer metering method. You do not need to associate a basic bandwidth plan with pay-as-you-go GA instances. The billing of data transfer over the GA network is managed by Cloud Data Transfer (CDT). For more information, see Pay-by-data-transfer.
The first time you use a pay-as-you-go Global Accelerator instance, go to the pay-as-you-go GA activation page and activate GA as prompted.

Step 1: Configure basic information about an instance

Log on to the GA console.
On the Instances page, click Create Standard Pay-as-you-go Instance.

In the Basic Instance Configuration step, configure the following parameters and click Next.

Parameter	Description
GA Instance Name	Enter a name for the GA instance.
Instance Billing Method	Pay-As-You-Go is selected by default. You are charged instance fees, Capacity Unit (CU) fees, and data transfer fees for pay-as-you-go standard Global Accelerator instances. For more information about instance fees and CU fees, see Billing of pay-as-you-go GA instances. For more information about data transfer fees, see Pay-by-data-transfer.
Resource Group	Select the resource group to which the standard Global Accelerator instance belongs. The resource group must be created by the current Alibaba Cloud account in Resource Management. For more information, see Create a resource group.

Step 2: Configure an acceleration area

Specify acceleration regions and allocate bandwidth to each acceleration region.

In the Configure Acceleration Area step, configure the parameters and click Next. The following table describes the parameters.

Parameter	Description
Acceleration Area	Select one or more regions from the drop-down list and click Add. In this example, the China (Hong Kong) region in the Asia Pacific section is selected.
Assign Bandwidth
Maximum Bandwidth	Specify the maximum bandwidth for the acceleration region. Each acceleration region supports a bandwidth range of 2 to 10,000 Mbit/s. The maximum bandwidth is used for bandwidth throttling. The data transfer fees are managed by CDT. In this example, the default value 200 Mbit/s is used. Important If you specify a small value for the maximum bandwidth, throttling may occur and packets may be dropped. Specify a maximum bandwidth based on your business requirements.
IP Protocol	Select the IP version that is used to connect to Global Accelerator. In this example, the default value IPv4 is selected.
ISP Line Type	Select an ISP line type for the Global Accelerator instance. BGP (Multi-ISP) is selected in this example.

Step 3: Configure a listener

A listener listens for connection requests and distributes the requests to endpoints based on the port and the protocol that you specify. Each listener is associated with an endpoint group. You can associate an endpoint group with a listener by specifying the region to which you want to distribute network traffic. After you associate an endpoint group with a listener, network traffic is distributed to the optimal endpoint in the endpoint group.

In the Configure listeners step, configure the required parameters and click Next.

The following table describes only the parameters that are relevant to this topic. Use the default values for other parameters. For more information, see Add and manage intelligent routing listeners.

Parameter	Description
Listener Name	Enter a name for the listener.
Routing Type	Select a routing type. In this example, Intelligent Routing is selected.
Protocol	Select a protocol for the listener. In this example, TCP is selected. Note By default, OSS buckets can be accessed over HTTP and HTTPS. To access small-sized files that are stored in OSS buckets, you can use HTTP or HTTPS listeners. This is applicable only when you use custom domain names to access OSS buckets. If you want to access large-sized files that are stored in OSS buckets, you can use HTTP, HTTPS, and TCP listeners. The HTTP and HTTPS protocols transfer data based on TCP connections. Therefore, you can use TCP listeners.
Port	Specify a port for the listener to receive and forward requests to endpoints. Valid values: 1 to 65499. You can specify up to 30 ports for each listener. Separate multiple listener ports with commas (,). For example, you can enter `80,90,8080`. If you want to specify a port range, use a hyphen (-). Example: `80-85`. In this example, 80,443 is entered.
Client Affinity	Specify whether to enable client affinity. If client affinity is enabled, requests from the same client are forwarded to the same endpoint when the client connects to a stateful application. In this example, Source IP is selected.

Step 4: Configure an endpoint group and endpoints

In the Configure an endpoint group step, configure the parameters and click Next. The following table describes the parameters.

This topic describes only the key parameters. For more information, see Add and manage endpoint groups of intelligent routing listeners.

Parameter	Description
Region	Select the region where the endpoint group is deployed. In this example, US (Virginia) is selected.
Endpoint Configuration	Client requests are routed to endpoints. To add an endpoint, specify the following parameters: Backend Service Type: Select OSS. Backend Service: Select the OSS bucket in which the resources that you want to access are stored. Weight: Specify a weight for the endpoint. Valid values: 0 to 255. GA distributes network traffic to endpoints based on the weights. In this example, the default value 255 is used. Warning If you set the weight of an endpoint to 0, Global Accelerator stops distributing network traffic to the endpoint. Proceed with caution.
Preserve Client IP	Specify whether to preserve client IP addresses. If you enable this feature, the backend server can obtain client IP addresses. For more information, see Preserve client IP addresses. In this example, the default value Do Not Preserve is used.

In the Configuration Review step, check the configurations and click Submit.
Note
It takes 3 to 5 minutes to create a Global Accelerator instance.
(Optional) After you create a GA instance, you can click the instance ID on the Instances page to view the configurations of the instance. On the instance details page, you can click tabs such as Instance Information, Listeners, and Acceleration Areas to view more details.

Step 5: Configure the hosts file or DNS settings

If you use the OSS endpoint that is provided by Alibaba Cloud to access the backend OSS bucket, you must modify the hosts file on your on-premises machine. If you use a custom domain name to access the backend OSS bucket, you must configure Domain Name System (DNS) settings. After you configure the hosts file or DNS settings, requests that are destined for the backend OSS bucket are routed to GA for acceleration.

Configure the hosts file

You must configure the hosts file and map the OSS endpoint to the accelerated IP address that is allocated by GA. For more information about the format of OSS endpoints, see OSS domain name rules.

Note

In this example, the Alibaba Cloud Linux 3 operating system is used. The command that is used to run the test may vary based on the operating system. For more information, refer to the user guide of the operating system.

Open the CLI on a computer in the China (Hong Kong) office.
Configure the hosts file.
1. Run the following command to open the/etc/hosts file:
```
vim /etc/hosts
```
2. Press the I key to enter the edit mode and add the following content to the file:
```
<The accelerated IP address> <The name of the source OSS bucket>.<The endpoint of the source OSS bucket>
```
  After you modify the hosts file, press the Esc key, enter :wq!, and then press the Enter key to save the modified file and exit the edit mode.
Run the following command to restart the network service.
```
systemctl restart NetworkManager
```
After you modify the hosts file, run the following command to view the /etc/hosts file and check whether the hosts file takes effect:
```
cat /etc/hosts
ping <OSS endpoint>
```
The result shows that the OSS endpoint is mapped to the accelerated IP address in the /etc/hosts file, and the hosts file takes effect.

Configure DNS settings

You must configure DNS settings to route requests that are destined for the OSS bucket to GA. Select one of the following methods to configure DNS settings:

Add an A record that maps the endpoint of the OSS bucket to the IPv4 address allocated by GA.
Add a CNAME record that maps the endpoint of the OSS bucket to the CNAME that is allocated by GA.

Before you configure DNS settings, make sure that you have associated a custom domain name with the OSS bucket where the resources are stored. Otherwise, you cannot pass the domain name verification for OSS. The custom domain name is in the <Source bucket name>.<self-managed domain name> format. For more information, see Map a custom domain name to the default domain name of a bucket.

Log on to the Alibaba Cloud DNS console.
If your domain name is not registered by using Alibaba Cloud Domains, you must add your domain name to Alibaba Cloud DNS.
Note
If your domain name is not registered by using Alibaba Cloud Domains, you must add your domain name to Alibaba Cloud DNS before you configure a DNS record. For more information, see the "Add a domain name" section of the Manage domain names topic. If your domain name is registered by using Alibaba Cloud Domains, skip this step.
On the Domain Name Resolution page, find the domain name and click DNS Settings in the Actions column to go to the DNS Settings page.
Select one of the following methods to configure DNS records based on your requirements:
- If Add CNAME Record Automatically is not enabled when you associate the custom domain name with the bucket to which resources belong, click Add DNS Record.
- If Add CNAME Record Automatically is enabled when you associate the custom domain name with the bucket to which resources belong, click Modify in the Actions column.

In the Add DNS Record or Modify DNS Record panel, configure the following parameters and click OK.

A record

Parameter	Description
Record Type	Select A from the drop-down list. An A record is used to map a domain name to an IPv4 address.
Hostname	Enter the prefix of the accelerated domain name. If Add CNAME Record Automatically is not enabled when you associate the custom domain name with the bucket to which resources belong, enter the name of the OSS bucket. If Add CNAME Record Automatically is enabled when you associate the custom domain name with the bucket to which resources belong, make sure that the hostname is the name of the OSS bucket.
DNS Request Source	Select Default from the drop-down list.
Record Value	Enter the accelerated IP address that is allocated by GA.
TTL	Specify the period of time for which a record is cached by a DNS server. A smaller TTL value specifies that the resolver retains the information in the cache for a shorter period of time. In this example, the default value 10 Minutes is used.

CNAME record

Parameter	Description
Record Type	In this example, CNAME is selected. The CNAME record is used to map the endpoint of the OSS bucket to the CNAME allocated by GA.
Hostname	Enter the prefix of the accelerated domain name. If Add CNAME Record Automatically is not enabled when you associate the custom domain name with the bucket to which resources belong, enter the name of the OSS bucket. If Add CNAME Record Automatically is enabled when you associate the custom domain name with the bucket to which resources belong, make sure that the hostname is the name of the OSS bucket.
DNS Request Source	Select Default from the drop-down list.
Record Value	Enter the CNAME that is allocated by GA. You can find the CNAME on the Instances page.
TTL	Specify the period of time for which a record is cached by a DNS server. A smaller TTL value specifies that the resolver retains the information in the cache for a shorter period of time. In this example, the default value 10 Minutes is used.

After you configure DNS settings, run the following command to check whether the DNS settings take effect.

ping <Custom domain name>

In the following example, a CNAME record is configured. The result shows that the request is routed to the endpoint of the OSS bucket after you ping the custom domain name. This indicates that the DNS settings take effect.

配置DNS解析后-验证.png

Step 6: Test network connectivity

Perform the following steps to verify that data transfer is accelerated when clients in the China (Hong Kong) region access the OSS bucket in the US (Virginia) region.

In this example, ossutil provided by OSS is used to replicate the files that are stored in the OSS bucket in the US (Virginia) to a client in the China (Hong Kong) region. For more information about how to install ossutil, see Install ossutil.

Open the CLI on a client in the China (Hong Kong) region.
Run the following command to replicate files from the bucket in the US (Virginia) region to the client in the China (Hong Kong) region:
- Before GA is used:
```
ossutil64 --retry-times 10 -e <Endpoint of the source bucket> -k <AccessKey secret of the Alibaba Cloud account> -i <AccessKey ID of the Alibaba Cloud account> -f cp oss://<Source bucket name>/<Source file> ./
```
  Note
  If GA is not used, the domain name that you access is <the name of the source OSS bucket>.<the endpoint of the source OSS bucket>. Example: testBucketName.oss-us-west-1.aliyuncs.com.
  Figure 1. Network before acceleration
- When GA is used, the commands that are used to replicate files vary based on whether a custom domain name is used.
  - Run the following command if you do not use a custom domain name:
```
ossutil64 --retry-times 10 -e <Endpoint of the source bucket> -k <AccessKey secret of the Alibaba Cloud account> -i <AccessKey ID of the Alibaba Cloud account> -f cp oss://<Source bucket name>/<Source file> ./
```
    Note
    The domain name that is accessed and the command are the same as those when GA is not used. However, you must modify the hosts file and map the endpoint to the accelerated IP address that is allocated by GA. For more information, see Step 5: Configure the hosts file or DNS settings.
    If you do not use a custom domain name, the domain name that you access is <the name of the source OSS bucket>.<the endpoint of the source OSS bucket>. Example: testBucketName.oss-us-west-1.aliyuncs.com.
    Figure 2. Network latency when GA accelerates access over the bucket endpoint
  - If you use a custom domain name, run the following command:
```
ossutil64 --retry-times 10 -e <Custom domain name> -k <AccessKey secret of the Alibaba Cloud account> -i <AccessKey ID of the Alibaba Cloud account> -f cp oss://<Source bucket name>/<Source file> ./
```
    Note
    Make sure that you configure DNS settings that map the custom domain name to GA. For more information, see Step 5: Configure the hosts file or DNS settings.
    If you use a custom domain name, the domain name that you access is <the name of the source OSS bucket>.<the self-managed domain name>. Example: testBucketName.example.com.
    Figure 3. Network latency when a custom domain name is used
The test results show that GA reduces the network latency when the client in the China (Hong Kong) region accesses the OSS bucket in the US (Virginia) region.
Note
When you use GA to accelerate access to OSS buckets in the US (Virginia) region for clients in the China (Hong Kong) region, the acceleration performance varies based on your workload.