HTTPDNS:Best practices for using HTTPDNS with WebView and a local proxy on Android

2.1 Network request limitations of Android WebView

In Android, WebView uses the system's built-in browser kernel, which is based on Chromium, to load web pages. Its network request pipeline is separate from the application's network library. Domain name resolution also uses the kernel's native DNS process. Many requests cannot be intercepted at the Java layer. This closed system creates a natural barrier to integrating HTTPDNS.

2.2 Evolution of technical solutions

As the Android system and WebView evolve, so do the technical solutions for integrating HTTPDNS.

Interception solution:

This solution intercepts resource requests in the shouldInterceptRequest() callback and re-issues them using an in-app network library, such as OkHttp, with integrated HTTPDNS resolution. This process bypasses the system DNS.

This method is simple to implement and works for some GET requests. However, it has a major limitation. Because the request body for methods such as POST is not exposed, you can only fully intercept GET requests. This limits the solution's coverage of total WebView traffic.

Proxy solution (Recommended):

This solution starts a local proxy server and uses the ProxyController from AndroidX WebKit to configure a unified HTTP, HTTPS, and WebSocket proxy for WebView. This method transparently forwards all kernel traffic to a network stack controlled by the application. This lets you intercept all protocols and request methods for HTTPDNS resolution, which overcomes the coverage bottleneck of the shouldInterceptRequest() method.

3.1 Solution overview

This solution starts a local HTTP proxy server in your application. It uses the `ProxyController` from AndroidX WebKit to route all WebView network requests to this local proxy. The proxy performs HTTPDNS domain name resolution and then forwards the requests to the actual server.

Workflow:

3.2 Dependency integration

Add the following dependencies to your app/build.gradle file:

// Adjust the dependency versions as needed
dependencies {
    implementation 'androidx.webkit:webkit:1.7.0'
    implementation 'com.aliyun.ams:alicloud-android-httpdns:2.6.5'
    implementation 'io.github.littleproxy:littleproxy:2.4.4'
    implementation 'io.netty:netty-all:4.1.42.Final'
    implementation 'com.google.guava:guava:30.1.1-android'
    implementation 'org.slf4j:slf4j-android:1.7.30'
}

3.3 Integration steps

Integrating the HTTPDNS WebView proxy involves three core steps:

public class WebviewProxyApplication extends Application {
    public static String accountId = "your_account_id";
    public static String secretKey = "your_secret_key"; // Optional
    @Override
    public void onCreate() {
        super.onCreate();
        initHttpDns();
    }
    
    private void initHttpDns() {
        try {
            InitConfig config = new InitConfig.Builder()
                .setContext(this)
                .setTimeout(5000)
                .setEnableCacheIp(true)
                .setEnableExpiredIp(true)
                .setSecretKey(secretKey) // Optional, for authentication
                .build();
            HttpDns.init(accountId, config);
            Log.i("HTTPDNS", "Initialization successful");
        } catch (Exception e) {
            Log.e("HTTPDNS", "Initialization failed", e);
        }
    }
}

public class ProxyService extends Service {
    private HttpProxyServer proxyServer;
    private HttpDnsResolver httpDnsResolver;
    private boolean isRunning = false;
    
    public boolean startProxyServer() {
        if (isRunning) return true;
        
        try {
            // Create an HTTPDNS resolver
            httpDnsResolver = new HttpDnsResolver(this);
            
            // Start the LittleProxy server
            int port = generateRandomPort();
            proxyServer = DefaultHttpProxyServer.bootstrap()
                .withPort(port)
                .withAddress(new InetSocketAddress("127.0.0.1", port))
                .withServerResolver(httpDnsResolver) // Use the HTTPDNS resolver
                .withConnectTimeout(10000)
                .withIdleConnectionTimeout(30000)
                .start();
                
            isRunning = true;
            Log.i("Proxy", "Proxy server started successfully: 127.0.0.1:" + port);
            return true;
        } catch (Exception e) {
            Log.e("Proxy", "Failed to start proxy server", e);
            return false;
        }
    }
    
    public void stopProxyServer() {
        if (proxyServer != null) {
            proxyServer.stop();
            isRunning = false;
            Log.i("Proxy", "Proxy server stopped");
        }
    }
    
    public boolean isProxyRunning() {
        return isRunning;
    }
}

// HTTPDNS resolver implementation
class HttpDnsResolver implements HostResolver {
    private HttpDnsService httpDnsService;
    
    public HttpDnsResolver(Context context) {
        httpDnsService = HttpDns.getService(WebviewProxyApplication.accountId);
    }
    
    @Override
    public InetSocketAddress resolve(String host, int port) throws UnknownHostException {
        try {
            // Use HTTPDNS for resolution
            HTTPDNSResult result = httpDnsService.getHttpDnsResultForHostSyncNonBlocking(
                host, RequestIpType.auto);
            
            if (result != null && result.getIps() != null && result.getIps().length > 0) {
                String ip = result.getIps()[0];
                Log.d("DNS", "HTTPDNS resolution: " + host + " -> " + ip);
                return new InetSocketAddress(ip, port);
            }
            
            // Fall back to system DNS
            return new InetSocketAddress(InetAddress.getByName(host), port);
        } catch (Exception e) {
            Log.w("DNS", "Resolution failed, using system DNS: " + host);
            return new InetSocketAddress(InetAddress.getByName(host), port);
        }
    }
}

public class MainActivity extends AppCompatActivity {
    private WebView webView;
    private ProxyService proxyService;
    private boolean isProxyRunning = false;
    
    // Main thread Executor for ProxyController
    private final Executor mainExecutor = ContextCompat.getMainExecutor(this);
    
    private void configureWebViewProxy() {
        // Check if the system supports proxy configuration
        if (!WebViewFeature.isFeatureSupported(WebViewFeature.PROXY_OVERRIDE)) {
            Log.w("WebView", "Proxy configuration is not supported on the current system");
            return;
        }
        
        if (isProxyRunning) {
            String proxyAddress = proxyManager.getProxyAddress();
        
            // Configure WebView to use the local proxy
            ProxyConfig proxyConfig = new ProxyConfig.Builder()
                .addProxyRule(proxyAddress, ProxyConfig.MATCH_HTTP)
                .addProxyRule(proxyAddress, ProxyConfig.MATCH_HTTPS)
                .addDirect(ProxyConfig.MATCH_ALL_SCHEMES) // Connect directly for other protocols
                .build();
                
            ProxyController.getInstance().setProxyOverride(proxyConfig, mainExecutor, () -> {
                Log.i("WebView", "Proxy configured successfully");
            });
        } else {
            // Clear the proxy configuration
            ProxyController.getInstance().clearProxyOverride(mainExecutor, () -> {
                Log.i("WebView", "Proxy configuration cleared");
            });
        }
    }
    
    private void loadUrl(String url) {
        // Make sure the proxy configuration is applied before loading the URL
        configureWebViewProxy();
        webView.loadUrl(url);
    }
}

3.4 Complete implementation reference

Creating a local proxy service, integrating HTTPDNS, and configuring WebView can be complex. A complete, open-source implementation is available on GitHub: HTTPDNS WebView Demo.

The core components include the following:

• HttpDnsResolver: Implements LittleProxy's `HostResolver` interface and integrates HTTPDNS resolution.

• ProxyService: An Android Service that manages the proxy server's lifecycle.

• ProxyManager: Manages the proxy status and service binding.

• MainActivity: Handles UI interaction and WebView proxy configuration.

The solution includes a multilayer backoff mechanism to ensure stability. It automatically skips `ProxyController` configuration on unsupported system versions. It falls back to the system DNS if HTTPDNS resolution fails. It also automatically switches to another IP if a resolved IP is unavailable.

4.1 Solution overview

The traditional interception solution overrides the WebViewClient.shouldInterceptRequest() method to intercept WebView network requests. It then re-issues the requests using an OkHttp client with integrated HTTPDNS.

Key features:

• Supports interception of GET requests only.

• Requires manual handling of data such as cookies.

• Has higher implementation complexity and maintenance costs.

• Offers good compatibility and supports older WebView versions.

4.2 Detailed implementation

The traditional interception solution involves many technical details and edge cases. For the complete implementation and instructions, see Best practices for using HTTPDNS with OkHttp on Android.