0

For context: We would like to analyze user mobile online (browsing) behavior in a user study. The study has been approved by our university's IRB, participants will have to give informed consent and will receive remuneration. Of course, monitoring network traffic is very intrusive, but we're sincerely not trying to be the bad guys here. For example, we would like to know if people get there information from social media, news sites (and from which one).

Over the last to weeks, I've explored some potential approaches. While I have some basic experiences with Android development, I'm far from an expert. Here is what I've tried and considered -- I would appreciate any feedback to see what I got wrong and/or what I'm missing:

Side note: we cannot expect study participants to root their phones :)

  • Mock-up browser (simple app with a WebView and rudimentary browser features) or a modified fork of Chrome. Here, on the one hand, I cannot really estimate the required effort for both ideas. And a big issue is that most mobile online use is done through apps that often have their on WebView.

  • Proxy. I set up a proxy with SSL support, so if the participants change their proxy setting and install our CA certificate, we would see their traffic. However, apps (e.g., Facebook) are not required to use the proxy settings, and since Android 7 (API 24) apps by default do not recognize user-installed certificates. It worked pretty well with an older Android 6.0 phone, but now it's only a limited option. We could still see, for example, that a request to facebook.com was made, but no longer the exact URL.

  • App Activity Monitoring. Since some apps wouldn't have used the proxy anyway, I tried to at least estimate which app was in the foreground. To do this, I wrote a simple app that periodically checked which app had the highest usage in the last, say, 10 seconds using the UsageStatsManager. This seemed to work reasonably well.

  • Local on-device VPN. Lastly, I've tried using a local VPN using the VpnService; I found and used example code I found here. On the plus side, even apps that ignore the proxy have to go through that local VPN. However, I face two big problems. Firstly, while it seems to work it's very unstable. Some websites won't load properly and after that, even sites that worked before fail. And secondly, there's no SLL support so I wouldn't be able to see the traffic. I lack the expertise right now to address both issues or even to say if they can be solved.

This is where we currently stand. All opinions are more than welcome.

3
  • Welcome to Android Enthusiasts! Please note that this is a site for end users – so it's probably not the best place for your case (it's not exactly off-topic, though, as "end users" also includes "power users" and you didn't exactly ask for development help which would be off-topic). A pointer, though: I'd setup a proxy on a computer, connect the devices via WiFi and define the proxy in WiFi settings. There are proxies for exactly this use case, like Charles Proxy (see e.g. here). – Izzy Mar 14 '20 at 10:49
  • @Izzy Noted, sorry for posting it here! Setting up a proxy is the easy part. The issue is that you can use it only to debug/test your own app up since the app as to accept user-install CA certificates. So for monitoring 3rd-party apps this does not work (anymore). – Christian Mar 19 '20 at 1:03
  • Strange, MobilSicher is still using it for their testing. But OK; maybe some "power user" has a good approach for you. Best luck! – Izzy Mar 19 '20 at 21:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.