How to set Wi-Fi HTTPS proxy, not HTTP, via adb shell?

Question

For HTTP I found a link: How to set WIFI proxy via adb shell? But I want to setup HTTPS proxy via adb shell.

Answer 1

This question is a near duplicate of questions like this and this, and closely related to this and this. In fact it can be better explained by a developer, but since it keeps on appearing again and again with slightly different situations, I'll try to explain what I know with some unnecessary details to make the picture clear from a broader perspective.

In short:

• Unlike VPN, a proxy - by design - is not meant to be set system-wide. Individual apps have to create connections through proxy instead of connecting directly to remote hosts (visited websites).
• If an app doesn't have built-in support for proxy, it's possible to enforce the proxy on per-app basis or system-wide. Most of the solutions require root access, but Android's VpnService APIs work without root.
  • Transparently proxying https traffic requires SSL Bumping (decryption) on proxy server which breaks the security model of https. This needs to be worked around from both sides; client and proxy.

Android lets proxy be configured for Wi-Fi, Mobile Data (APN) or both. As of Android 9 proxy settings are saved in

/data/misc/wifi/WifiConfigStore.xml
/data/user_de/<User_ID>/com.android.providers.telephony/databases/telephony.db

Both files aren't directly accessible for editing without root. But there is another Global HTTP Proxy setting meant for managed devices which saves proxy information to /data/system/users/<User_ID>/settings_global.xml. It's possible to change global device settings through CLI:

~$ settings put global http_proxy <host>:<port>

And to unset:

~$ settings put global http_proxy :0

Or you can manually set keys:

global_http_proxy_host
global_http_proxy_port
global_http_proxy_exclusion_list
global_proxy_pac_url

Android sets this host/port information as the following Java system properties on all DVMs of running apps:

http.proxyHost
http.proxyPort
https.proxyHost
https.proxyPort

But only default browser or other system apps necessarily honor these proxy related networking properties. That's why when setting Wi-Fi proxy it warns:

The HTTP proxy is used by the browser but may not be used by the other apps.

For non-system custom apps, it is up to the app creator to respect the proxy settings and use them for http and https proxy, as usually a server listening on single port can handle both type of requests (GET and CONNECT) as explained below.

---

HOW PROXY WORKS (EXPLICITLY)

Actually http/https proxy is used only by proxy-aware apps and Android cannot enforce this system-wide. No one can, except the (http client) app which is making http/https connections. It's because proxy is set on per connection basis i.e. client itself decides which kind of connection it's going to make when creating a network socket. For http proxy full URL is to be given after GET verb instead of simple path. For https proxy the CONNECT method is used instead, so that proxy is requested to create a http tunnel towards remote host, through which TLS handshake takes place (without proxy's involvement) and encrypted traffic flows back and forth. It's essentially like doing a single local port forwarding.

See this answer for SOCKS proxy. For further details: Why proxy set on Android does not work when used as hotspot?

CLIENTS' SUPPORT FOR PROXY

Since establishing a proxy connection is the responsibility of client app, many client libraries have in-built support to configure proxy settings. Mostly web browsers let user configure proxy in settings UI or use system provided information e.g. on Windows there are IE Proxy Settings and also netsh command for programs which are linked against WinHTTP library. On Linux environment variables http(s)_proxy are honored by many networking libraries, CLI utilities and GUI programs including web browsers ⁽¹⁾.

However native methods aren't directly applicable to Android's Java based framework. Android's HttpURLConnection ^{(2, 3)} and Apache's HttpClient (deprecated) handlers honor ^{(4, 5)} above mentioned system properties ^{(6, 7)}. So the apps making use of these APIs are able to use same proxy ^{(8, 9)} for all connections. An example is WebView ^{(10, 11)} which is based on Chromium:

by default the system-wide Android network proxy settings are used to redirect requests to appropriate proxy servers

But not all Java libraries consider the VM system properties as was the case with Apache HttpClient before version 4.2. Such apps may listen to PROXY_CHANGE_ACTION broadcast ^{(12, 13)} to get notified of changes in proxy settings and then getDefaultProxy settings - either set through CLI (global) or GUI (per network basis) or programmatically ^{(14, 15)}.

Or app may also ask user to configure proxy in app itself and use multiple available proxies ^{(16, 17)} including system properties ⁽¹⁸⁾ on per-connection basis.

Proxy-aware apps, particularly web browsers support:

• Configuring https (SSL) proxy explicitly in settings through GUI (e.g. Firefox's network.proxy.ssl preference) or CLI (Chrome's --proxy-server option) or plugins like FoxyProxy.

• Using .pac ^{(19, 20, 21)} file to import proxy settings from remote host.

  For Wi-Fi APs Android also supports importing .pac files (but not WPAD). In this case ProxyHandler and PacProcessor helper apps make a local proxy available to apps but authentication doesn't work.

• CONNECT over TLS connection towards SSL proxy (e.g. Squid or Apache) in contrast to plain TCP CONNECT method which sends unencrypted traffic from client to proxy before TLS handshake. See FoxyProxy and network.proxy.proxy_over_tls option on Firefox.
• Authentication if proxy requires, other apps usually fall back to no proxy mode when connection to proxy isn't established. Apps need to handle authentication even if loading some content in WebView.

So if you want your app to work with https proxy, consider all above factors and choose appropriate SDK/libraries when coding the app.

ENFORCING PROXY (TRANSPARENTLY)

For those libraries and programs which provide no means to configure proxy, there are different methods proxifiers/socksifiers use to force traffic towards a proxy server.

• tun2socks is a socksifier which accepts all TCP (and optionally UDP) traffic directed towards a tun interface in routing table. That's how VPN works on Android. While tun2socks is a perfect choice to be used with SOCKS tunnels, it doesn't support http(s). But there are tun2http(s) solutions (like TunProxy) which are based on Android's VpnService API and transparently redirect all http(s) traffic towards remote proxy.
• redsocks is another similar tool but it NATs all traffic to a specific port listening as SOCKS or HTTPS (HTTP/CONNECT) proxy. transparent_https is another similar custom made minimal solution.

Both above methods can be used on Android to forward traffic to an upstream proxy e.g. on internet or local network ⁽²²⁾. Also see related details in this answer.

• Setup interactive proxy like ZAP or mitmproxy on your local network (e.g. on router / gateway) using only routing / TPROXY (no DNAT/SNAT) and configure it to use your upstream proxy running in explicit mode (CONNECT).
• Use Shared library interceptor method. See details in this answer.

SSL/TLS INTERCEPTION (MITM)

If the purpose of https proxy is the Access Control like logging, anti-virus scanning, content blocking, filtering or adaptation, SSL bumping ^{(23, 24)} is required on proxy server. In the absence of CONNECT, the very first HTTP request sent is already encrypted. So the proxy doesn't know who the original intended host is, (SNI may help but even with CONNECT method, you can't do DPI) without decrypting SSL.

It means that every outgoing SSL/TLS connection is to be terminated and re-established by proxy, so the proxy must be forced intercepting (like squid). But not every https proxy supports SSL decryption and working transparently ⁽²⁵⁾; the s part needs special handling of Certificate Authorities. Proxy's root SSL certificate - which signs dynamically forged web server's certificate - needs to be added to credential store of client app and/or device (keeping this in consideration) to avoid invalid certificate errors. Or client may opt to ignore the errors, thus defeating the purpose of secure http.

If the purpose is not the filtering but only to hide visited websites from ISP / local network or to hide public IP from visited websites, an alternative is to do simple NAT on both sides i.e. on device or router and the remote proxy server. SSL Pass through is a similar approach.

---

RELATED:

• How to set up a socks5 with Android cli?
• Why is not all traffic captured from my device when routing through proxy?