I recently answered in detail to question: How to set Wi-Fi HTTPS proxy, not HTTP, via adb shell?, trying to clear some confusions many of us (including me) have regarding how proxy works.
DOES YOUR APP GENERATE HTTP TRAFFIC?
Proxy usually means http proxy, but not all traffic generated by every app is http - a Layer 7 protocol in OSI model. Email clients, for instance, use IMAP/POP3/SMTP (L6/7) protocols which won't work with http GET method, instead require http CONNECT proxies. Latter can only forward TCP traffic, for UDP at least SOCKS5 proxy is needed which operates at L5. DNS and mostly games and VoIP apps generate UDP traffic, so they won't work with http proxies at all. ping utility - for instance - uses ICMP (at L3) which can't be encapsulated in TCP or UDP, so it won't work with SOCKS too and requires VPN.
See more details in Why proxy set on Android does not work when used as hotspot?
DOES HTTP PROXY WORK WITH EVERY APP?
Proxy is not meant to be set system-wide, or in other words you cannot force all apps to use the same proxy you have configured in Settings. Instead apps are supposed to make proxied http connections if they want to. Operating systems provide proxy-aware libraries (for http connections) as part of the SDKs so the developers don't have to take care of proxy related stuff separately. Such Android apps honor proxy settings set system-wide (but may still fall back to no-proxy mode if proxy fails for some reason).
Other OSes also use similar approaches to provide system-wide proxy settings through some GUI settings menu or using environment variables on CLI. Browsers and some other apps like email clients usually have their own settings as well to set proxy. However if the app/program is not proxy-aware, it won't honor any of the proxy settings at all. That's why when setting Wi-Fi proxy it warns:
"The HTTP proxy is used by the browser but may not be used by the other apps."
Same holds true for proxy set through PAC file or in APNs (Mobile Data) or global proxy (meant for managed devices).
Coming to your questions...
While using the browser on my Android device I can see requests logged in Fiddler
...
However, when using certain apps I can see no requests being logged
The guide linked in the question suggests configuring Android WiFi settings to use proxy. As explained above, the browser would be proxy-aware, the "certain apps" aren't.
I want to be able to analyze the requests sent between an app on my phone which controls a wireless thermostat connected to my LAN via WiFi.
- Make sure your concerned app is generating
http traffic to be captured while sniffing on proxy. Otherwise use something at lower level - e.g. SOCKS proxy or VPN - for other protocols.
- Make sure the app regards Android's global proxy settings when making
http connections. In case of https the app should be using CONNECT method in place of GET.
If the app is not proxy-aware, you need to enforce proxy on phone and (in case of https) do forced / transparent SSL/TLS interception on proxy.
Enforcing proxy is possible by different methods e.g. by using SOCKS tunnel instead, and/or redirecting traffic towards a proxy server using some Layer 2/3 approach like VPN (tun2socks/tun2http) app or iptables (DNAT). Xposed modules like Inspeckage can also enforce per-app proxy by hooking into APIs.
Another approach is to use a local host (preferably Linux PC) as default gateway and set up proxy there to intercept traffic from any host on local network. In this way no proxy configuration is required on phone. It's also possible to setup packet analyzer tool like Wireshark in place of proxy on local network for traffic interception.
DOES YOUR APP USE SECURE HTTP (HTTPS)?
- If the purpose of proxy is DPI, logging, anti-virus scanning, content blocking, filtering or adaptation, you need to do SSL bumping / MITM on proxy server.
- MITM requires adding proxy's root SSL certificate to device's credential store (or the app should be able to ignore invalid certificate errors). But since Android 7, user-added CAs (as suggested in the linked guide) are no more trusted to establish secure connections. So you need to add the root certificate to
/system/etc/security/cacerts (requires rooted phone). "But app developers can choose to let their apps work with manually added CA certificates." (1)
- If the app is using its own trusted credential store instead of that provided by OS, you need to add proxy's root CA to the list of CAs maintained by the app (if that's possible at all).
- If the app is using SSL pinning (as the Twitter app does), it won't trust any intermediate authority except its own self-signed root CA. You may try to disable SSL Pinning by taking control of related APIs. Xposed module SSLUnpinning can disable SSL pinning for JSSE, OkHttp and Apache
http client libraries. But if the app uses some other libraries/APIs, you are again out of luck.
