This document describes how to intercept requests in WebView scenarios when using HTTPDNS and implement native requests based on HttpURLConnection to achieve "direct IP connection".
1. Background
Alibaba Cloud HTTPDNS is an effective way to avoid DNS hijacking. When making network requests, you can bypass the system's default DNS resolution by calling the API provided by HTTPDNS, eliminating the risk of DNS hijacking and obtaining higher resolution accuracy, thereby improving client network performance and stability.
In web pages loaded by WebView, when the web page initiates a network request, we can intercept the request within WebView and let the native layer (App) execute the actual network request. During this process, you can use the resolution results provided by HTTPDNS to replace system DNS resolution to prevent DNS hijacking.
This document describes how to intercept requests and implement native requests based on HttpURLConnection when using HTTPDNS in WebView scenarios. However, considering that OkHttp is now the mainstream network framework on Android, using OkHttp is a more concise and elegant choice. We recommend referring to Android HTTPDNS+Webview+OkHttp best practices for implementation, and only refer to this document when you cannot use OkHttp.
Important
This topic describes a solution that uses WebView in Android apps to access HTTPDNS. The code provided in this topic is for reference only. We recommend that you read this topic carefully and evaluate this solution before you implement it.
Due to the severe fragmentation of the Android ecosystem and different levels of customization by various manufacturers, we recommend that you implement this solution gradually and monitor online exceptions. If you encounter any issues, feel free to provide feedback to us through technical support so that we can optimize it promptly.
This best practice document only focuses on how to use the IP resolved by HTTPDNS when used in combination. For the HTTPDNS resolution service itself, please first check Android SDK integration.
2. Limitations and impacts of WebView only intercepting GET requests
In Android WebView, through the WebViewClient.shouldInterceptRequest() method, you can only obtain detailed information about GET requests. For other request methods such as POST, the body is not provided to developers, so it is impossible to fully intercept and replace with HTTPDNS resolution. However, this does not mean that integrating HTTPDNS is meaningless due to the following reasons:
Most resource requests in web pages (images, CSS, JS, and other static resources) are GET requests Generally, static resources required by web pages, such as images, scripts, style sheets, etc., are almost all obtained using GET requests. Therefore, by intercepting GET requests and using HTTPDNS to resolve domain names, the main resource loading process can be covered.
A significant proportion of API calls also use GET requests Although some data interactions may use POST, many simple APIs or query requests also use GET for convenience and caching considerations. In this case, they will also be intercepted and use HTTPDNS resolution.
Intercepting only part of the requests can effectively avoid most hijacking risks DNS hijacking often targets popular domain names or static resource domain names, and these domain requests are usually GET. Therefore, even if HTTPDNS is only used for GET requests, it can avoid most hijacking problems for public resources or main service interfaces, achieving 80% or even higher protection.
In summary, although at the WebView level, currently only GET requests can be intercepted through native methods, in actual business scenarios, this approach usually covers the main traffic, thus providing more than 80% of DNS security protection.
3. Code example
For the complete code of HTTPDNS+WebView best practices, please refer to WebView+HTTPDNS Android Demo.
4. Interception interface description
void setWebViewClient (WebViewClient client);WebView provides the setWebViewClient interface to intercept network requests. By overriding the shouldInterceptRequest method in WebViewClient, you can intercept all network requests:
public class WebViewClient {
// API < 21
public WebResourceResponse shouldInterceptRequest(WebView view, String url) {
...
}
// API >= 21
public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request) {
...
}
......
}There are two versions of shouldInterceptRequest:
When
API < 21, the version of theshouldInterceptRequestmethod is:public WebResourceResponse shouldInterceptRequest(WebView view, String url)Only the request URL is returned. You cannot obtain the request method, headers, and request body. If you call this method to intercept requests, WebView may fail to load all requested resources. Therefore, when
API < 21, do not intercept requests:public WebResourceResponse shouldInterceptRequest(WebView view, String url) { return super.shouldInterceptRequest(view, url); }When
API >= 21,shouldInterceptRequestprovides a new version:public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request)The structure of
WebResourceRequestis:public interface WebResourceRequest { Uri getUrl(); // The request URL boolean isForMainFrame(); // Indicates whether the request is initiated by the primary MainFrame boolean isRedirect(); // Indicates whether the request is redirected boolean hasGesture(); // Indicates whether the request is triggered by a certain behavior, such as clicking String getMethod(); // The method of the request Map<String, String> getRequestHeaders(); // The headers of the request }
As you can see, when API >= 21, you can obtain the following information when intercepting requests:
The request URL
The request method, such as POST and GET
The request headers
5. Practical use
The following figure shows how requests are intercepted in WebView scenarios:
Intercept only GET requests
Configure headers
Configure HTTPS certificates
Handle HTTPS requests with the SNI
Handle redirects
MIME&Encoding
6. Intercept only GET requests
Because WebResourceRequest does not provide request body information, only GET requests can be intercepted. Here is a code example:
override fun shouldInterceptRequest(view: WebView?, request: WebResourceRequest?): WebResourceResponse? {
val scheme = request!!.url.scheme!!.trim()
val method = request.method
val headerFields = request.requestHeaders
val url = request.url.toString()
// Requests with bodies cannot be intercepted. Only requests without bodies can be processed properly.
if ((scheme.equals("http", ignoreCase = true) || scheme.equals("https", ignoreCase = true))
&& method.equals("get", ignoreCase = true)
) {
//TODO For more information about TODO, see the "Configure headers" section.
} else {
return super.shouldInterceptRequest(view, request)
}
}public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request) {
String scheme = request.getUrl().getScheme().trim();
String method = request.getMethod();
Map<String, String> headerFields = request.getRequestHeaders();
String url = request.getUrl().toString();
// Requests with bodies cannot be intercepted. Only requests without bodies can be processed properly.
if ((scheme.equalsIgnoreCase("http") || scheme.equalsIgnoreCase("https"))
&& method.equalsIgnoreCase("get")) {
//TODO For more information about TODO, see the "Configure headers" section.
} else {
return super.shouldInterceptRequest(view, request);
}
}7. Configure headers
The method of requesting resources is abstracted.
fun recursiveRequest(
path: String,
headers: Map<String?, String?>?,
reffer: String?
): URLConnection? {
val conn: HttpURLConnection
var url: URL? = null
try {
url = URL(path)
// Obtain an IP address by calling the synchronization operation
val httpdnsResult = HttpDns.getService(accountID)
.getHttpDnsResultForHostSync(url.host, RequestIpType.both)
var ip: String? = null
if (httpdnsResult.ips != null && httpdnsResult.ips.isNotEmpty()) {
ip = httpdnsResult.ips[0]
} else if (httpdnsResult.ipv6s != null && httpdnsResult.ipv6s.isNotEmpty()) {
ip = httpdnsResult.ipv6s[0]
}
if (!TextUtils.isEmpty(ip)) {
// Obtain the IP address by using HTTPDNS, replace the value of the Host field of the HTTP request header with the resolved IP address, and then configure the Host header
val newUrl = path.replaceFirst(url.host.toRegex(), ip!!)
conn = URL(newUrl).openConnection() as HttpURLConnection
conn.connectTimeout = 30000
conn.readTimeout = 30000
conn.instanceFollowRedirects = false
// Add the original headers
if (headers != null) {
headers.forEach{ entry ->
conn.setRequestProperty(entry.key, entry.value)
}
}
// Set the Host header field of the HTTP request
conn.setRequestProperty("Host", url.host)
//TODO For more information about TODO, see the "Configure HTTPS certificates" section.
} else {
return null
}
//TODO For more information about TODO, see the "Handle redirects" section.
} catch (e: MalformedURLException) {
Log.w(TAG, "recursiveRequest MalformedURLException")
} catch (e: IOException) {
Log.w(TAG, "recursiveRequest IOException")
} catch (e: Exception) {
Log.w(TAG, "unknow exception")
}
return null
}public URLConnection recursiveRequest(String path, Map<String, String> headers, String reffer) {
HttpURLConnection conn;
URL url = null;
try {
url = new URL(path);
// Obtain the IP address by calling the synchronization API
HTTPDNSResult httpdnsResult = HttpDns.getService(accountID).getHttpDnsResultForHostSync(url.getHost(), RequestIpType.both);
String ip = null;
if (httpdnsResult.getIps() != null && httpdnsResult.getIps().length > 0) {
ip = httpdnsResult.getIps()[0];
} else if (httpdnsResult.getIpv6s() != null && httpdnsResult.getIpv6s().length > 0) {
ip = httpdnsResult.getIpv6s()[0];
}
if (!TextUtils.isEmpty(ip)) {
// After the IP address is obtained by using HTTPDNS, replace the original value of the HOST field in the HTTP request URL with the IP address
String newUrl = path.replaceFirst(url.getHost(), ip);
conn = (HttpURLConnection) new URL(newUrl).openConnection();
conn.setConnectTimeout(30000);
conn.setReadTimeout(30000);
conn.setInstanceFollowRedirects(false);
// Add the original headers
if (headers != null) {
for (Map.Entry<String, String> field : headers.entrySet()) {
conn.setRequestProperty(field.getKey(), field.getValue());
}
}
// Set the Host header field of the HTTP request
conn.setRequestProperty("Host", url.getHost());
//TODO For more information about TODO, see the "Configure HTTPS certificates" section.
} else {
return null;
}
//TODO For more information about TODO, see the "Handle redirects" section.
} catch (MalformedURLException e) {
Log.w(TAG, "recursiveRequest MalformedURLException");
} catch (IOException e) {
Log.w(TAG, "recursiveRequest IOException");
} catch (Exception e) {
Log.w(TAG, "unknow exception");
}
return null;
}override fun shouldInterceptRequest(
view: WebView?,
request: WebResourceRequest?
): WebResourceResponse? {
val scheme = request!!.url.scheme!!.trim()
val method = request.method
val headerFields = request.requestHeaders
val url = request.url.toString()
// Requests with bodies cannot be intercepted. Only requests without bodies can be processed properly.
if ((scheme.equals("http", ignoreCase = true) || scheme.equals("https", ignoreCase = true))
&& method.equals("get", ignoreCase = true)
) {
try {
val connection = recursiveRequest(url, headerFields, null)
?: return super.shouldInterceptRequest(view, request)
//TODO For more information about TODO, see the "MIME&Encoding" section.
} catch (e: MalformedURLException) {
e.printStackTrace()
} catch (e: IOException) {
e.printStackTrace()
}
}
return super.shouldInterceptRequest(view, request)
}public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request) {
String scheme = request.getUrl().getScheme().trim();
String method = request.getMethod();
Map<String, String> headerFields = request.getRequestHeaders();
String url = request.getUrl().toString();
// Requests with bodies cannot be intercepted. Only requests without bodies can be processed properly.
if ((scheme.equalsIgnoreCase("http") || scheme.equalsIgnoreCase("https"))
&& method.equalsIgnoreCase("get")) {
try {
URLConnection connection = recursiveRequest(url, headerFields, null);
if (connection == null) {
return super.shouldInterceptRequest(view, request);
}
//TODO For more information about TODO, see the "MIME&Encoding" section.
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
return super.shouldInterceptRequest(view, request);
}8. Configure HTTPS certificates
If the intercepted request is an HTTPS request, certificate verification is required:
fun recursiveRequest(
path: String,
headers: Map<String?, String?>?,
reffer: String?
): URLConnection? {
val conn: HttpURLConnection
var url: URL? = null
try {
url = URL(path)
// Obtain an IP address by calling the synchronization operation
val httpdnsResult = HttpDns.getService(accountID)
.getHttpDnsResultForHostSync(url.host, RequestIpType.both)
var ip: String? = null
if (httpdnsResult.ips != null && httpdnsResult.ips.size > 0) {
ip = httpdnsResult.ips[0]
} else if (httpdnsResult.ipv6s != null && httpdnsResult.ipv6s.size > 0) {
ip = httpdnsResult.ipv6s[0]
}
if (!TextUtils.isEmpty(ip)) {
// Obtain the IP address by using HTTPDNS, replace the value of the Host field of the HTTP request header with the resolved IP address, and then configure the Host header
val newUrl = path.replaceFirst(url.host.toRegex(), ip!!)
conn = URL(newUrl).openConnection() as HttpURLConnection
conn.connectTimeout = 30000
conn.readTimeout = 30000
conn.instanceFollowRedirects = false
// Add the original headers
headers?.forEach{ entry ->
conn.setRequestProperty(entry.key, entry.value)
}
// Set the Host header field of the HTTP request
conn.setRequestProperty("Host", url.host)
if (conn is HttpsURLConnection) {
// Verify the certificate for an HTTPS request
conn.hostnameVerifier =
HostnameVerifier { _, session ->
var host = conn.getRequestProperty("Host")
if (null == host) {
host = conn.url.host
}
HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session)
}
//TODO For more information about TODO, see the "Handle HTTPS requests with the SNI" section.
}
//TODO For more information about TODO, see the "Handle redirects" section.
} else {
return null
}
} catch (e: MalformedURLException) {
Log.w(TAG, "recursiveRequest MalformedURLException")
} catch (e: IOException) {
Log.w(TAG, "recursiveRequest IOException")
} catch (e: Exception) {
Log.w(TAG, "unknow exception")
}
return null
}public URLConnection recursiveRequest(String path, Map<String, String> headers, String reffer) {
HttpURLConnection conn;
URL url = null;
try {
url = new URL(path);
// Obtain the IP address by calling the synchronization API
HTTPDNSResult httpdnsResult = HttpDns.getService(accountID).getHttpDnsResultForHostSync(url.getHost(), RequestIpType.both);
String ip = null;
if (httpdnsResult.getIps() != null && httpdnsResult.getIps().length > 0) {
ip = httpdnsResult.getIps()[0];
} else if (httpdnsResult.getIpv6s() != null && httpdnsResult.getIpv6s().length > 0) {
ip = httpdnsResult.getIpv6s()[0];
}
if (!TextUtils.isEmpty(ip)) {
// After the IP address is obtained by using HTTPDNS, replace the original value of the HOST field in the HTTP request URL with the IP address
String newUrl = path.replaceFirst(url.getHost(), ip);
conn = (HttpURLConnection) new URL(newUrl).openConnection();
conn.setConnectTimeout(30000);
conn.setReadTimeout(30000);
conn.setInstanceFollowRedirects(false);
// Add the original headers
if (headers != null) {
for (Map.Entry<String, String> field : headers.entrySet()) {
conn.setRequestProperty(field.getKey(), field.getValue());
}
}
// Set the Host header field of the HTTP request
conn.setRequestProperty("Host", url.getHost());
if (conn instanceof HttpsURLConnection) {
HttpsURLConnection httpsURLConnection = (HttpsURLConnection)conn;
// Verify the certificate for an HTTPS request
httpsURLConnection.setHostnameVerifier(new HostnameVerifier() {
@Override
public boolean verify(String hostname, SSLSession session) {
String host = httpsURLConnection.getRequestProperty("Host");
if (null == host) {
host = httpsURLConnection.getURL().getHost();
}
return HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session);
}
});
//TODO For more information about TODO, see the "Handle HTTPS requests with the SNI" section.
}
//TODO For more information about TODO, see the "Handle redirects" section.
} else {
return null;
}
} catch (MalformedURLException e) {
Log.w(TAG, "recursiveRequest MalformedURLException");
} catch (IOException e) {
Log.w(TAG, "recursiveRequest IOException");
} catch (Exception e) {
Log.w(TAG, "unknow exception");
}
return null;
}9. Handle HTTPS requests with the SNI
If the request involves an SNI scenario, you need to customize the SSLSocket. For users who are not familiar with SNI scenarios, please refer to SNI:
9.1 Customize SSLSocket
class TlsSniSocketFactory constructor(conn: HttpsURLConnection): SSLSocketFactory() {
private val mConn: HttpsURLConnection
init {
mConn = conn
}
override fun createSocket(plainSocket: Socket?, host: String?, port: Int, autoClose: Boolean): Socket {
var peerHost = mConn.getRequestProperty("Host")
if (peerHost == null) {
peerHost = host
}
val address = plainSocket!!.inetAddress
if (autoClose) {
// we don't need the plainSocket
plainSocket.close()
}
// create and connect SSL socket, but don't do hostname/certificate verification yet
val sslSocketFactory =
SSLCertificateSocketFactory.getDefault(0) as SSLCertificateSocketFactory
val ssl = sslSocketFactory.createSocket(address, R.attr.port) as SSLSocket
// enable TLSv1.1/1.2 if available
ssl.enabledProtocols = ssl.supportedProtocols
// set up SNI before the handshake
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
// setting sni hostname
sslSocketFactory.setHostname(ssl, peerHost)
} else {
// No documented SNI support on Android <4.2, trying with reflection
try {
val setHostnameMethod = ssl.javaClass.getMethod(
"setHostname",
String::class.java
)
setHostnameMethod.invoke(ssl, peerHost)
} catch (e: Exception) {
}
}
// verify hostname and certificate
val session = ssl.session
if (!HttpsURLConnection.getDefaultHostnameVerifier()
.verify(peerHost, session)
) throw SSLPeerUnverifiedException(
"Cannot verify hostname: $peerHost"
)
return ssl
}
}public class TlsSniSocketFactory extends SSLSocketFactory {
private HttpsURLConnection mConn;
public TlsSniSocketFactory(HttpsURLConnection conn) {
mConn = conn;
}
@Override
public Socket createSocket(Socket plainSocket, String host, int port, boolean autoClose) throws IOException {
String peerHost = mConn.getRequestProperty("Host");
if (peerHost == null)
peerHost = host;
InetAddress address = plainSocket.getInetAddress();
if (autoClose) {
// we don't need the plainSocket
plainSocket.close();
}
// create and connect SSL socket, but don't do hostname/certificate verification yet
SSLCertificateSocketFactory sslSocketFactory = (SSLCertificateSocketFactory) SSLCertificateSocketFactory.getDefault(0);
SSLSocket ssl = (SSLSocket) sslSocketFactory.createSocket(address, port);
// enable TLSv1.1/1.2 if available
ssl.setEnabledProtocols(ssl.getSupportedProtocols());
// set up SNI before the handshake
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
// setting sni hostname
sslSocketFactory.setHostname(ssl, peerHost);
} else {
// No documented SNI support on Android <4.2, trying with reflection
try {
java.lang.reflect.Method setHostnameMethod = ssl.getClass().getMethod("setHostname", String.class);
setHostnameMethod.invoke(ssl, peerHost);
} catch (Exception e) {
}
}
// verify hostname and certificate
SSLSession session = ssl.getSession();
if (!HttpsURLConnection.getDefaultHostnameVerifier().verify(peerHost, session))
throw new SSLPeerUnverifiedException("Cannot verify hostname: " + peerHost);
return ssl;
}
}9.2 Configure custom SSLSocket
fun recursiveRequest(
path: String,
headers: Map<String?, String?>?,
reffer: String?
): URLConnection? {
val conn: HttpURLConnection
var url: URL? = null
try {
url = URL(path)
// Obtain an IP address by calling the synchronization operation
val httpdnsResult = HttpDns.getService(accountID)
.getHttpDnsResultForHostSync(url.host, RequestIpType.both)
var ip: String? = null
if (httpdnsResult.ips != null && httpdnsResult.ips.isNotEmpty()) {
ip = httpdnsResult.ips[0]
} else if (httpdnsResult.ipv6s != null && httpdnsResult.ipv6s.isNotEmpty()) {
ip = httpdnsResult.ipv6s[0]
}
if (!TextUtils.isEmpty(ip)) {
// Obtain the IP address by using HTTPDNS, replace the value of the Host field of the HTTP request header with the resolved IP address, and then configure the Host header
val newUrl: String = path.replaceFirst(url.host, ip)
conn = URL(newUrl).openConnection() as HttpURLConnection
conn.connectTimeout = 30000
conn.readTimeout = 30000
conn.instanceFollowRedirects = false
// Add the original headers
headers?.forEach { entry ->
conn.setRequestProperty(entry.key, entry.value)
}
// Set the Host header field of the HTTP request
conn.setRequestProperty("Host", url.host)
if (conn is HttpsURLConnection) {
// Verify the certificate for an HTTPS request
conn.hostnameVerifier = HostnameVerifier { _, session ->
var host = conn.getRequestProperty("Host")
if (null == host) {
host = conn.url.host
}
HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session)
}
// Handle HTTPS requests with the SNI Create an SSLScoket
conn.sslSocketFactory = TlsSniSocketFactory(conn)
}
//TODO For more information about TODO, see the "Handle redirects" section.
} else {
return null
}
} catch (e: MalformedURLException) {
Log.w(TAG, "recursiveRequest MalformedURLException")
} catch (e: IOException) {
Log.w(TAG, "recursiveRequest IOException")
} catch (e: Exception) {
Log.w(TAG, "unknow exception")
}
return null
}public URLConnection recursiveRequest(String path, Map<String, String> headers, String reffer) {
HttpURLConnection conn;
URL url = null;
try {
url = new URL(path);
// Obtain the IP address by calling the synchronization API
HTTPDNSResult httpdnsResult = HttpDns.getService(accountID).getHttpDnsResultForHostSync(url.getHost(), RequestIpType.both);
String ip = null;
if (httpdnsResult.getIps() != null && httpdnsResult.getIps().length > 0) {
ip = httpdnsResult.getIps()[0];
} else if (httpdnsResult.getIpv6s() != null && httpdnsResult.getIpv6s().length > 0) {
ip = httpdnsResult.getIpv6s()[0];
}
if (!TextUtils.isEmpty(ip)) {
// After the IP address is obtained by using HTTPDNS, replace the original value of the HOST field in the HTTP request URL with the IP address
String newUrl = path.replaceFirst(url.getHost(), ip);
conn = (HttpURLConnection) new URL(newUrl).openConnection();
conn.setConnectTimeout(30000);
conn.setReadTimeout(30000);
conn.setInstanceFollowRedirects(false);
// Add the original headers
if (headers != null) {
for (Map.Entry<String, String> field : headers.entrySet()) {
conn.setRequestProperty(field.getKey(), field.getValue());
}
}
// Set the Host header field of the HTTP request
conn.setRequestProperty("Host", url.getHost());
if (conn instanceof HttpsURLConnection) {
HttpsURLConnection httpsURLConnection = (HttpsURLConnection)conn;
// Verify the certificate for an HTTPS request
httpsURLConnection.setHostnameVerifier(new HostnameVerifier() {
@Override
public boolean verify(String hostname, SSLSession session) {
String host = httpsURLConnection.getRequestProperty("Host");
if (null == host) {
host = httpsURLConnection.getURL().getHost();
}
return HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session);
}
});
// Handle HTTPS requests with the SNI. Create an SSLSocket.
httpsURLConnection.setSSLSocketFactory(new TlsSniSocketFactory(httpsURLConnection));
}
//TODO For more information about TODO, see the "Handle redirects" section.
} else {
return null;
}
} catch (MalformedURLException e) {
Log.w(TAG, "recursiveRequest MalformedURLException");
} catch (IOException e) {
Log.w(TAG, "recursiveRequest IOException");
} catch (Exception e) {
Log.w(TAG, "unknow exception");
}
return null;
}10. Handle redirects
When you attempt to intercept a GET request and the server returns an HTTP redirect response, check whether the request contains the Cookie header:
If the request contains the Cookie header, the request is not intercepted because cookies are saved by domain names and change after redirects.
If the request does not contain the Cookie header, the request is re-initiated.
fun recursiveRequest(
path: String,
headers: Map<String, String?>?,
reffer: String?
): URLConnection? {
val conn: HttpURLConnection
var url: URL? = null
try {
url = URL(path)
// Obtain an IP address by calling the synchronization operation
val httpdnsResult = HttpDns.getService(accountID)
.getHttpDnsResultForHostSync(url.host, RequestIpType.both)
var ip: String? = null
if (httpdnsResult.ips != null && httpdnsResult.ips.isNotEmpty()) {
ip = httpdnsResult.ips[0]
} else if (httpdnsResult.ipv6s != null && httpdnsResult.ipv6s.isNotEmpty()) {
ip = httpdnsResult.ipv6s[0]
}
if (!TextUtils.isEmpty(ip)) {
// Obtain the IP address by using HTTPDNS, replace the value of the Host field of the HTTP request header with the resolved IP address, and then configure the Host header
val newUrl = path.replaceFirst(url.host.toRegex(), ip!!)
conn = URL(newUrl).openConnection() as HttpURLConnection
conn.connectTimeout = 30000
conn.readTimeout = 30000
conn.instanceFollowRedirects = false
// Add the original headers
headers?.forEach { entry ->
conn.setRequestProperty(entry.key, entry.value)
}
// Set the Host header field of the HTTP request
conn.setRequestProperty("Host", url.host)
if (conn is HttpsURLConnection) {
// Verify the certificate for an HTTPS request
conn.hostnameVerifier =
HostnameVerifier { _, session ->
var host = conn.getRequestProperty("Host")
if (null == host) {
host = conn.url.host
}
HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session)
}
// Handle HTTPS requests with the SNI Create an SSLScoket
conn.sslSocketFactory = TlsSniSocketFactory(conn)
}
} else {
return null
}
val code = conn.responseCode // Network block
return if (code in 300..399) {
// If the original headers include a cookie, the interception is canceled
var containCookie = false
if (headers != null) {
for (item in headers.keys) {
if (item.contains("Cookie")) {
containCookie = true
break
}
}
}
if (containCookie) {
return null
}
var location = conn.getHeaderField("Location")
if (location == null) {
location = conn.getHeaderField("location")
}
if (location != null) {
if (!(location.startsWith("http://") || location
.startsWith("https://"))
) {
// In some cases, the server returns only the path of the new URL, and you need to complete the URL by replacing the original domain name with the IP address
val originalUrl = URL(path)
location = (originalUrl.protocol + "://"
+ originalUrl.host + location)
}
recursiveRequest(location, headers, path)
} else {
// The location information cannot be obtained. Enable the browser to obtain the information
null
}
} else {
// redirect finish.
conn
}
} catch (e: MalformedURLException) {
Log.w(TAG, "recursiveRequest MalformedURLException")
} catch (e: IOException) {
Log.w(TAG, "recursiveRequest IOException")
} catch (e: Exception) {
Log.w(TAG, "unknow exception")
}
return null
}public URLConnection recursiveRequest(String path, Map<String, String> headers, String reffer) {
HttpURLConnection conn;
URL url = null;
try {
url = new URL(path);
// Obtain the IP address by calling the synchronization API
HTTPDNSResult httpdnsResult = HttpDns.getService(accountID).getHttpDnsResultForHostSync(url.getHost(), RequestIpType.both);
String ip = null;
if (httpdnsResult.getIps() != null && httpdnsResult.getIps().length > 0) {
ip = httpdnsResult.getIps()[0];
} else if (httpdnsResult.getIpv6s() != null && httpdnsResult.getIpv6s().length > 0) {
ip = httpdnsResult.getIpv6s()[0];
}
if (!TextUtils.isEmpty(ip)) {
// After the IP address is obtained by using HTTPDNS, replace the original value of the HOST field in the HTTP request URL with the IP address
String newUrl = path.replaceFirst(url.getHost(), ip);
conn = (HttpURLConnection) new URL(newUrl).openConnection();
conn.setConnectTimeout(30000);
conn.setReadTimeout(30000);
conn.setInstanceFollowRedirects(false);
// Add the original headers
if (headers != null) {
for (Map.Entry<String, String> field : headers.entrySet()) {
conn.setRequestProperty(field.getKey(), field.getValue());
}
}
// Set the Host header field of the HTTP request
conn.setRequestProperty("Host", url.getHost());
if (conn instanceof HttpsURLConnection) {
final HttpsURLConnection httpsURLConnection = (HttpsURLConnection) conn;
// Verify the certificate for an HTTPS request
httpsURLConnection.setHostnameVerifier(new HostnameVerifier() {
@Override
public boolean verify(String hostname, SSLSession session) {
String host = httpsURLConnection.getRequestProperty("Host");
if (null == host) {
host = httpsURLConnection.getURL().getHost();
}
return HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session);
}
});
// Handle HTTPS requests with the SNI. Create an SSLSocket.
httpsURLConnection.setSSLSocketFactory(new TlsSniSocketFactory(httpsURLConnection));
}
} else {
return null;
}
int code = conn.getResponseCode();// Network block
if (code >= 300 && code < 400) {
// If the original headers include a cookie, the interception is canceled
boolean containCookie = false;
if (headers != null) {
for (Map.Entry<String, String> headerField : headers.entrySet()) {
if (headerField.getKey().contains("Cookie")) {
containCookie = true;
break;
}
}
}
if (containCookie) {
return null;
}
String location = conn.getHeaderField("Location");
if (location == null) {
location = conn.getHeaderField("location");
}
if (location != null) {
if (!(location.startsWith("http://") || location
.startsWith("https://"))) {
// In some cases, the server returns only the path of the new URL, and you need to complete the URL by replacing the original domain name with the IP address
URL originalUrl = new URL(path);
location = originalUrl.getProtocol() + "://"
+ originalUrl.getHost() + location;
}
return recursiveRequest(location, headers, path);
} else {
// The location information cannot be obtained. Enable the browser to obtain the information
return null;
}
} else {
// redirect finish.
return conn;
}
} catch (MalformedURLException e) {
Log.w(TAG, "recursiveRequest MalformedURLException");
} catch (IOException e) {
Log.w(TAG, "recursiveRequest IOException");
} catch (Exception e) {
Log.w(TAG, "unknow exception");
}
return null;
}11. Handle MIME&Encoding
If you intercept a network request, you need to return a WebResourceResponse:
public WebResourceResponse(String mimeType, String encoding, InputStream data) ;To create a WebResourceResponse object, you need to provide:
The Multipurpose Internet Mail Extensions (MIME) type of the request
The character encoding of the request
The input stream of the request
The request input stream can be obtained through URLConnection.getInputStream(), while the MIME type and encoding can be obtained through the ContentType of the request, that is, through URLConnection.getContentType(), such as:
text/html;charset=utf-8However, not all requests can obtain complete contentType information. In this case, you can refer to the following strategy:
override fun shouldInterceptRequest(
view: WebView?,
request: WebResourceRequest?
): WebResourceResponse? {
val scheme = request!!.url.scheme!!.trim()
val method = request.method
val headerFields = request.requestHeaders
val url = request.url.toString()
// Requests with bodies cannot be intercepted. Only requests without bodies can be processed properly.
if ((scheme.equals("http", ignoreCase = true) || scheme.equals("https", ignoreCase = true))
&& method.equals("get", ignoreCase = true)
) {
try {
val connection = recursiveRequest(url, headerFields, null)
?: return super.shouldInterceptRequest(view, request)
// Note: The body data of a POST request is not provided in the WebResourceRequest operation, and cannot be processed
val contentType = connection.contentType
var mime: String? = null
val charset: String? = null
if (contentType != null) {
mime = contentType.split(";".toRegex()).dropLastWhile { it.isEmpty() }
.toTypedArray()[0]
val fields = contentType.split(";".toRegex()).dropLastWhile { it.isEmpty() }
.toTypedArray()
if (fields.size > 1) {
var charset = fields[1]
if (charset.contains("=")) {
charset = charset.substring(charset.indexOf("=") + 1)
}
}
}
val httpURLConnection = connection as HttpURLConnection
val statusCode = httpURLConnection.responseCode
val response = httpURLConnection.responseMessage
val headers = httpURLConnection.headerFields
val headerKeySet: Set<String> = headers.keys
// Requests without the MIME type are not intercepted
return if (TextUtils.isEmpty(mime)) {
super.shouldInterceptRequest(view, request)
} else {
// Encoding information is not required for binary resources
if (!TextUtils.isEmpty(charset) || (mime!!.startsWith("image")
|| mime.startsWith("audio")
|| mime.startsWith("video"))
) {
val resourceResponse =
WebResourceResponse(mime, charset, httpURLConnection.inputStream)
resourceResponse.setStatusCodeAndReasonPhrase(statusCode, response)
val responseHeader: MutableMap<String, String> = HashMap()
for (key in headerKeySet) {
// HttpUrlConnection may contain request headers with the null key, which points to the status code of the HTTP request
responseHeader[key] = httpURLConnection.getHeaderField(key)
}
resourceResponse.responseHeaders = responseHeader
resourceResponse
} else {
super.shouldInterceptRequest(view, request)
}
}
} catch (e: MalformedURLException) {
e.printStackTrace()
} catch (e: IOException) {
e.printStackTrace()
}
}
return super.shouldInterceptRequest(view, request)
}public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request) {
String scheme = request.getUrl().getScheme().trim();
String method = request.getMethod();
Map<String, String> headerFields = request.getRequestHeaders();
String url = request.getUrl().toString();
// Requests with bodies cannot be intercepted. Only requests without bodies can be processed properly.
if ((scheme.equalsIgnoreCase("http") || scheme.equalsIgnoreCase("https"))
&& method.equalsIgnoreCase("get")) {
try {
URLConnection connection = recursiveRequest(url, headerFields, null);
if (connection == null) {
return super.shouldInterceptRequest(view, request);
}
// Note: The body data of a POST request is not provided in the WebResourceRequest operation, and cannot be processed
String contentType = connection.getContentType();
String mime = null;
String charset = null;
if (contentType != null) {
mime = contentType.split(";")[0];
String[] fields = contentType.split(";");
if (fields.length > 1) {
String charset = fields[1];
if (charset.contains("=")) {
charset = charset.substring(charset.indexOf("=") + 1);
}
}
}
HttpURLConnection httpURLConnection = (HttpURLConnection) connection;
int statusCode = httpURLConnection.getResponseCode();
String response = httpURLConnection.getResponseMessage();
Map<String, List<String>> headers = httpURLConnection.getHeaderFields();
Set<String> headerKeySet = headers.keySet();
// Requests without the MIME type are not intercepted
if (TextUtils.isEmpty(mime)) {
return super.shouldInterceptRequest(view, request);
} else {
// Encoding information is not required for binary resources
if (!TextUtils.isEmpty(charset) || (mime.startsWith("image")
|| mime.startsWith("audio")
|| mime.startsWith("video"))) {
WebResourceResponse resourceResponse = new WebResourceResponse(mime, charset, httpURLConnection.getInputStream());
resourceResponse.setStatusCodeAndReasonPhrase(statusCode, response);
Map<String, String> responseHeader = new HashMap<String, String>();
for (String key : headerKeySet) {
// HttpUrlConnection may contain request headers with the null key, which points to the status code of the HTTP request
responseHeader.put(key, httpURLConnection.getHeaderField(key));
}
resourceResponse.setResponseHeaders(responseHeader);
return resourceResponse;
} else {
return super.shouldInterceptRequest(view, request);
}
}
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
return super.shouldInterceptRequest(view, request);
}12. Summary
Scenario | Description |
Unavailable scenarios |
|
Applicable scenarios | Prerequisites:
Applicable scenarios:
|