All Products
Search
Document Center

How to capture HTTP messages(macOS/Charles)

Last Updated: Apr 09, 2021

Disclaimer: This topic may contain information about third-party products. Such information is for reference only. Alibaba Cloud does not make any guarantee, express or implied, with respect to the performance and reliability of third-party products and potential impacts of operations on the products.

Background

In the process of developing mobile app projects based on the mPaaS framework, various errors occur. In particular, when an error is about the interaction between the client and the server or gateway, you can capture HTTP messages to help understand and troubleshoot the error. For example, when you find that a request error occurs, you can analyze the request message and the response message to check whether the request information is invalid, whether the server returns valid responses, and whether the returned values are as expected. This helps you locate the root cause of the issue.

Tools for capturing HTTP messages

Charles and Fiddler are common tools for capturing HTTP messages. The two tools work in a similar way:

  • Deploy a local HTTP proxy. The proxy can capture and parse all HTTP messages that pass through the proxy.
  • Configure the CA Root certificate of Charles or Fiddler on the client in advance. This ensures that HTTPS messages can be parsed, the information forwarded by the intermediate proxy is trusted by the client, and the messages are decrypted.

Mode

proxy

Note: Charles and Fiddler can decrypt and display HTTPS messages only. In practice, developers may encrypt the data offline, for example, by using the data encryption feature of Mobile Gateway Service (MGS), and then exchange data over HTTPS. The original content of this part of data cannot be decrypted by Charles or Fiddler. Only the content that is encrypted offline can be displayed.

Charles (macOS platform)

This topic describes how Charles works, how to install and configure Charles, and how to configure HTTPS message decryption. Charles 4.5.5 is used as an example. Charles is an important tool for parsing HTTP messages in the macOS. This tool helps developers troubleshoot various problems and locate and resolve problems.

charles

References:

Installation and basic UI

You can download the DMG installation package of Charles 4.5.5 from the official website of Charles. Then, you can run the installation package to install Charles in the system.

When Charles is started for the first time, the permission to set the system proxy is requested. You can click Allow to grant the permission. After Charles is started, when an HTTP request passes through Charles, you can view the request in the home screen of Charles, as shown in the following figure:

message details

Configure the proxy on the client

  • The HTTP requests generated by the simulator that runs on the current device pass through the system proxy by default. You do not need to manually configure proxy information.
  • On physical mobile devices, including iPhone and Android phones, you must manually configure the proxy to make sure that all network requests generated by the devices are forwarded by the Charles proxy.

    1. Make sure that mobile devices can gain access to Mac devices with Charles by using IP addresses. We recommend that you install Charles in the same network as the mobile device.
    2. View the configurations of the Charles proxy and obtain the port number of the proxy: In the menu bar of Charles, choose Proxy > Proxy Settings to open the Proxy Settings page, as shown in the following figure.

      Note: The default port number is 8888.

      1. Go to the system network settings to view the IP address of the current device.
      2. Configure the proxy on the mobile device. For example, on an iOS device, tap Settings > Wi-Fi. Tap the icon on the end of the connected Wi-Fi to view its settings. Tap Configure Proxy on the bottom, select Manual, and then enter the IP address and port number of the Charles proxy.
         iOS proxy
      3. After the proxy is configured on the mobile device, when the first request from the mobile device arrives at Charles, the following message appears. Click Allow.
         connection
      4. In Charles, you can view all the HTTP requests generated by the mobile device. For example, you can use a browser on the mobile device to visit http://www.antfin.com/ and then you can view the request in Charles, as shown in the following figure.
         request

Configure SSL proxying

By default, Charles does not decrypt HTTPS messages. To parse the content of HTTPS messages, you must configure the SSL proxying feature. The configuration includes the following steps:

  • Install and trust Charles Root CA on the simulator and real mobile device.
  • In Charles, configure the HTTPS sites that require decryption of HTTPS messages.

Configure the mobile device

  1. Enable Charles. Then, use a browser on the mobile device to visit chls.pro/ssl. The browser automatically downloads charles-proxy-ssl-proxying-certificate.pem and instructs you to install Charles Proxy Customer Root Certificate. On iOS devices, we recommend that you use the Safari browser.
    certificate
    Android devices instruct you to name the certificate and install the certificate in the Customer Certificate.
  2. On devices that run on iOS 10 and later, go to Settings > General > About > Certificate Trust Settings. Enable full trust on the Charles certificate that you install in the previous step.
    trust
  3. Note that on Android apps, you must trust the Customer Certificate by adding the Network Security Configuration feature. To do this, perform the following steps:
    1. Add a Network Security Configuration XML file to the Portal project and store the file in res/xml/network_security_config.xml. The XML file contains the following content:
 
  1. <network-security-config>
  2. <debug-overrides>
  3. <trust-anchors>
  4. <!-- Trust user added CAs while debuggable only -->
  5. <certificates src="user" />
  6. </trust-anchors>
  7. </debug-overrides>
  8. </network-security-config>
 
  1. ![ XML resource](http://docs-aliyun.cn-hangzhou.oss.aliyun-inc.com/assets/pic/159161/AntCloud_zh/1585040328784/XML%20resource.png)
  2. 2. Update the `AndroidManifest.xml` file to use the preceding network security configuration:
 
  1. <?xml version="1.0" encoding="utf-8"?>
  2. <manifest ... >
  3. <application ...
  4. android:networkSecurityConfig="@xml/network_security_config"
  5. ... >
  6. ...
  7. </application>
  8. </manifest>
 
  1. ![ AndroidManifest](http://docs-aliyun.cn-hangzhou.oss.aliyun-inc.com/assets/pic/159161/AntCloud_zh/1585040394451/AndroidManifest.png)
  2. [Click here](http://docs-aliyun.cn-hangzhou.oss.aliyun-inc.com/assets/attach/159161/AntCloud_zh/1585040456465/3-1-trust-user-CA.zip "Click here") Download the sample code used to trust the User CA certificate.
  1. If the current device that runs on macOS want to trust the Charles certificate, go to Help > SSL Proxying > Install Charles Root Certificate.
    Install

Configure Charles

In the menu bar of Charles, choose Proxy > SSL Proxy Settings. On the SSL Proxying tab, you can add the domain names and ports that require decryption of HTTPS messages. Then, select Enable SSL Proxying.

SSL Proxy Settings

  • For Alibaba Cloud users of mPaaS, the following domain names are required:
    • *.aliyun.com
    • *.alipay.com
    • *.aliyuncs.com
  • For Apsara Stack users, add custom domain names based on the preceding configurations.

After configuration is complete, you can view the content of HTTPS messages on Charles. The following figures show examples of the content of HTTPS messages.

  • Before configuration: HTTPS messages are disordered.
    before
  • After configuration: The content of HTTPS messages is decrypted.
    after