Gordon
Assistant Engineer
Assistant Engineer
  • UID622
  • Fans1
  • Follows0
  • Posts52
Reads:674Replies:0

How can we reduce DNS resolution overhead using HTTPDNS

Created#
More Posted time:Mar 23, 2017 13:28 PM
1. Background
In mobile scenarios, DNS resolution overhead is a non-negligible factor for network request latency. On the one hand, UDP-based localDNS resolution is prone to timeout problems in mobile network environments with a high packet loss rate. On the other hand, DNS resolution suffers from hundreds of milliseconds of network latency under poor network conditions, greatly increasing service request workload and directly impacting user experience.
2. Solution
The Alibaba Cloud mobile service team released HTTPDNS to resolve traditional domain hijacking and dispatch accuracy problems while providing developers with flexible DNS management methods. By appropriately applying HTTPDNS management strategies on clients, zero latency of DNS resolution can be achieved, dramatically improving network communication efficiency under poor network conditions.
The main idea of implementing zero-latency DNS resolution is as follows:
• Construct client DNS caching.
By caching DNS appropriately, DNS resolution for every single network interaction always obtains IP information from memory, sharply reducing DNS resolution overhead. Also, you can customize an extensive range of caching policies for different services. For example, based on carriers' cache, you can reuse cached domain IP information for different carriers in network switchover scenarios, preventing DNS network resolution overhead introduced upon link re-selection after network switchover. Additionally, by introducing local offline storage of IP information, you can quickly read domain IP information from local systems when clients are restarted, greatly improving the loading efficiency of home pages.
• Pre-resolution of hotspot domains
During client startup, you can load the cache of hotspot domains by using pre-resolution of hotspot domains. When physical business requests occur, IP information for the target domain will be read directly from memory, preventing network overhead of legacy DNS.
• Lazy update policies
In most scenarios, IP information for business domains remains the same. Especially, during a single app utilization cycle, IP information obtained by domain resolution is normally the same (except for special business scenarios). For this reason, DNS lazy update policies can be used to implement quick DNS resolution after TTL expires. DNS lazy update policies mean when the client does not take the initiative to detect the IP address TTL time of domains but queries memory and returns the IP resolution result of a business domain when business requests need to access the business domain. If TTL of the IP resolution result expires, asynchronous DNS network resolution and cached result updating will be performed in the backend. With the policies above, DNS resolution always interacts with memory, preventing the latency caused by the introduction of network interaction.
2.1 Demo example
HTTPDNS Demo github provides Android/iOS SDKs and HTTPDNS API use cases. The following uses the Android SDK as the example to explain how to implement a zero-latency HTTPDNS service.
public class NetworkRequestUsingHttpDNS {

    private static HttpDnsService httpdns;
    //Enter your HTTPDNS accountID information, which can be obtained from the HTTPDNS console.
    private static String accountID = "100000";
    //Enter your hotspot domain.
    private static final String[] TEST_URL = {"http://www.aliyun.com", "http://www.taobao.com"};

    public static void main(final Context ctx) {
        try {
            //Set APP Context and Account ID and initialize HTTPDNS.
            httpdns = HttpDns.getService(ctx, accountID);
            //DegradationFilter is used to customize degrade logic.
            //By implementing the shouldDegradeHttpDNS method, you can determine whether to degrade or not as required.
            DegradationFilter filter = new DegradationFilter() {
                @Override
                public boolean shouldDegradeHttpDNS(String hostName) {
                    //You can customize degrade logic here. For example, do not use HTTPDNS resolution for www.taobao.com.
                    //Follow the HTTPDNS API document. Select to degrade and use local DNS when an intermediate HTTP agent exists.
                    return hostName.equals("www.taobao.com") || detectIfProxyExist(ctx);
                }
            };
            //Pass "filter" to httpdns. The shouldDegradeHttpDNS method will be invoked during resolution to determine whether to degrade or not.
            httpdns.setDegradationFilter(filter);
            //Set the pre-resolution domain list. In practice, you are recommended to execute pre-resolution operations in the app startup function. Pre-resolution operations are asynchronous and do not disrupt the startup process.
            httpdns.setPreResolveHosts(new ArrayList<>(Arrays.asList("www.aliyun.com", "www.taobao.com")));
            //Expired IP information can be returned. By setting expired IP information that is allowed to return and using the async query interface, you can implement DNS lazy update policies.
            httpdns.setExpiredIPEnabled(true);

            //Send a network request.
            String originalUrl = "http://www.aliyun.com";
            URL url = new URL(originalUrl);
            HttpURLConnection conn = (HttpURLConnection) url.openConnection();
            //Obtain IP information from the async interface. When IP TTL expires, you can directly obtain the latest DNS resolution result from memory while the HTTPDNS SDK will automatically update the resolution result of the corresponding domain.
            ip = httpdns.getIpByHostAsync(url.getHost());
            if (ip != null) {
                //When IP information is successfully obtained by HTTPDNS, URL replacement and HOST header configuration will be performed.
                Log.d("HTTPDNS Demo", "Get IP: " + ip + " for host: " + url.getHost() + " from HTTPDNS successfully!");
                String newUrl = originalUrl.replaceFirst(url.getHost(), ip);
                conn = (HttpURLConnection) new URL(newUrl).openConnection();
                //Set a Host domain for the HTTP request header.
                conn.setRequestProperty("Host", url.getHost());
            }
            DataInputStream dis = new DataInputStream(conn.getInputStream());
            int len;
            byte[] buff = new byte[4096];
            StringBuilder response = new StringBuilder();
            while ((len = dis.read(buff)) != -1) {
                response.append(new String(buff, 0, len));
            }
            Log.e("HTTPDNS Demo", "Response: " + response.toString());

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    /**
     * Check whether an agent has been set for the system. For details, refer to the HTTPDNS API document.
     */
    public static boolean detectIfProxyExist(Context ctx) {
        boolean IS_ICS_OR_LATER = Build.VERSION.SDK_INT >= Build.VERSION_CODES.ICE_CREAM_SANDWICH;
        String proxyHost;
        int proxyPort;
        if (IS_ICS_OR_LATER) {
            proxyHost = System.getProperty("http.proxyHost");
            String port = System.getProperty("http.proxyPort");
            proxyPort = Integer.parseInt(port != null ? port : "-1");
        } else {
            proxyHost = android.net.Proxy.getHost(ctx);
            proxyPort = android.net.Proxy.getPort(ctx);
        }
        return proxyHost != null && proxyPort != -1;
    }
}


Developers using HTTPDNS APIs can customize HTTPDNS management logic on clients that provide better efficiency and better meet the needs.
Guest