I recently answered in detail to question: How to set Wi-Fi HTTPS proxy, not HTTP, via adb shell?, trying to clear some confusions many of us (including me) have regarding how proxy works.
DOES YOUR APP GENERATE HTTP TRAFFIC?
Proxy usually means
http proxy, but not all traffic generated by every app is
http - a Layer 7 protocol in OSI model. Email clients, for instance, use
SMTP (L6/7) protocols which won't work with
http GET method, instead require
http CONNECT proxies. Latter can only forward
TCP traffic, for
UDP at least
SOCKS5 proxy is needed which operates at L5. DNS and mostly games and VoIP apps generate UDP traffic, so they won't work with
http proxies at all.
ping utility - for instance - uses
ICMP (at L3) which can't be encapsulated in TCP or UDP, so it won't work with SOCKS too and requires VPN.
See more details in Why proxy set on Android does not work when used as hotspot?
DOES HTTP PROXY WORK WITH EVERY APP?
Proxy is not meant to be set system-wide, or in other words you cannot force all apps to use the same proxy you have configured in Settings. Instead apps are supposed to make proxied
http connections if they want to. Operating systems provide proxy-aware libraries (for
http connections) as part of the SDKs so the developers don't have to take care of proxy related stuff separately. Such Android apps honor proxy settings set system-wide (but may still fall back to no-proxy mode if proxy fails for some reason).
Other OSes also use similar approaches to provide system-wide proxy settings through some GUI settings menu or using environment variables on CLI. Browsers and some other apps like email clients usually have their own settings as well to set proxy. However if the app/program is not proxy-aware, it won't honor any of the proxy settings at all. That's why when setting Wi-Fi proxy it warns:
"The HTTP proxy is used by the browser but may not be used by the other apps."
Same holds true for proxy set through PAC file or in APNs (Mobile Data) or global proxy (meant for managed devices).
Coming to your questions...
While using the browser on my Android device I can see requests logged in Fiddler
However, when using certain apps I can see no requests being logged
The guide linked in the question suggests configuring Android WiFi settings to use proxy. As explained above, the browser would be proxy-aware, the "certain apps" aren't.
I want to be able to analyze the requests sent between an app on my phone which controls a wireless thermostat connected to my LAN via WiFi.
- Make sure your concerned app is generating
http traffic to be captured while sniffing on proxy. Otherwise use something at lower level - e.g. SOCKS proxy or VPN - for other protocols.
- Make sure the app regards Android's global proxy settings when making
http connections. In case of
https the app should be using
CONNECT method in place of
If the app is not proxy-aware, you need to enforce proxy on phone and (in case of
https) do forced / transparent SSL/TLS interception on proxy.
Enforcing proxy is possible by different methods e.g. by using SOCKS tunnel instead, and/or redirecting traffic towards a proxy server using some Layer 2/3 approach like VPN (
tun2http) app or
DNAT). Xposed modules like Inspeckage can also enforce per-app proxy by hooking into APIs.
Another approach is to use a local host (preferably Linux PC) as default gateway and set up proxy there to intercept traffic from any host on local network. In this way no proxy configuration is required on phone. It's also possible to setup packet analyzer tool like Wireshark in place of proxy on local network for traffic interception.
DOES YOUR APP USE SECURE HTTP (HTTPS)?
- If the purpose of proxy is DPI, logging, anti-virus scanning, content blocking, filtering or adaptation, you need to do SSL bumping / MITM on proxy server.
- MITM requires adding proxy's root SSL certificate to device's credential store (or the app should be able to ignore invalid certificate errors). But since Android 7, user-added CAs (as suggested in the linked guide) are no more trusted to establish secure connections. So you need to add the root certificate to
/system/etc/security/cacerts (requires rooted phone). "But app developers can choose to let their apps work with manually added CA certificates." (1)
- If the app is using its own trusted credential store instead of that provided by OS, you need to add proxy's root CA to the list of CAs maintained by the app (if that's possible at all).
- If the app is using SSL pinning (as the Twitter app does), it won't trust any intermediate authority except its own self-signed root CA. You may try to disable SSL Pinning by taking control of related APIs. Xposed module SSLUnpinning can disable SSL pinning for JSSE, OkHttp and Apache
http client libraries. But if the app uses some other libraries/APIs, you are again out of luck.