0

How could you scrape data from an Android app (one without an available API)?

There are some apps that contain personal data I've entered that I'd like to be able to hook into and extract (rather than manual transcribing by hand).

closed as too broad by Andrew T., bmdixon, Matthew Read Jul 15 '15 at 13:53

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • May I know the name of the app? By knowing the app, perhaps the community can try and experiment with it easily. It's also possible that the app just use SQLite database to store the data, although it's inside internal storage and possibly need root to extract it. – Andrew T. Jul 14 '15 at 18:08
  • There is no general solution for this, apps can store data in a million different ways. – Matthew Read Jul 15 '15 at 13:53
3

The app likely uses an undocumented API to communicate with their backend server, so to reverse engineer what this is you need to monitor the app's network traffic. Fortunately this is straightforward for Android:

  1. install the Chrome extension ARC Welder so that you can run Android apps on your computer
  2. download the app's APK and load it in ARC Welder
  3. interact with the app and use a tool such as Wireshark or Fiddler to monitor how the app loads the data you are after from their backend server
  4. then replicate this network call to scrape the data directly

See this blog post for more details.

  • I have problems with Google Play Services when I use this app. – Martin Thoma Feb 26 at 15:13

Not the answer you're looking for? Browse other questions tagged or ask your own question.