All Products
Search

This Product

    Alibaba Cloud DNS:Search engine lines

    Document Center

    Alibaba Cloud DNS:Search engine lines

    Last Updated:Mar 12, 2024

    Overview

    Search engines use web crawlers to automatically grab data in the World Wide Web according to specific rules. As programs or scrips, web crawlers are also known as spiders or spiderbots.

    Scenarios

    • The traffic and bandwidth of website servers are consumed when the websites are accessed by web crawlers. You can add Domain Name System (DNS) records to add search engine lines. The DNS records point the domain names of the websites to specific server addresses. This way, DNS requests for the domain names from web crawlers can be redirected to the specified server addresses.

    • If you need to temporarily shut down your website and maintain the search engine optimization (SEO) ranking of your website, you can specify a search engine line to ensure that web crawlers can grab the information about the website after the website is shut down.

    Procedure

    For example, point the DNS requests from Baidu web crawlers to the server IP address 192.0.2.0. This allows the Baidu web crawlers to connect to the server. This way, the web crawlers can access and collect content such as pictures on your website and users can find the web pages, pictures, and videos on your website in Baidu.

    1. Log on to the Alibaba Cloud DNS console.

    2. On the Authoritative Domain Names tab of the Domain Name Resolution page, find the desired domain name and click DNS Settings in the Actions column.

    3. On the DNS Settings page, click Add DNS Record. In the Add DNS Record panel, select a desired search engine line for the DNS Request Source parameter. After you configure all required parameters, click OK.

      image

    FAQ

    1. Am I charged for using search engine lines?

      No, you can use search engine lines free of charge.

    2. Why do DNS requests sent from Google Chrome not match search engine lines?

      You can use search engine lines to specify the egress IP addresses of local DNS servers used by web crawlers. The egress IP addresses are not those of the local DNS servers used by browsers.

    3. Which search engines are supported by search engine lines?

      Google, Baidu, Bing, and Youdao.

    4. Why do DNS requests from web crawlers of a search engine fail to match the search engine line?

      Take Baidu web crawlers as an example. If the local DNS servers used by Baidu web crawlers are not the local DNS servers that Baidu web crawlers are supposed to use, the DNS requests from Baidu web crawlers cannot match the Baidu search engine line.