0

I'm running stress tests on Android using Monkey to generate the workload. For the type of analysis I have to do, I need to generate the same sequence of events for each experiment. So, in every experiment, I always use the adb shell monkey command with the same seed. For instance:

while true; do


adb shell monkey -p com.android.chrome -s 1 --pct-appswitch 100 --ignore-crashes --ignore-timeouts --ignore-security-exceptions --monitor-native-crashes -v -v 1 


sleep 1
adb shell monkey -p com.facebook.katana -s 1 --pct-appswitch 100 --ignore-crashes --ignore-timeouts --ignore-security-exceptions --monitor-native-crashes -v -v 1


sleep 1
adb shell monkey -p com.google.android.apps.maps -s 1 --pct-appswitch 100 --ignore-crashes --ignore-timeouts --ignore-security-exceptions --monitor-native-crashes -v -v 1 

done

The applications I launch are ten in total. However, looking at the measurement data, it seems to me that the system is stressed differently with each run. In particular, I find that the average launch time of applications degrades to a different extent in each experiment. Is everything normal or is it possible that I am doing something wrong? I realize that in different executions the system may respond differently. But by always setting the same seed (-s 1), shouldn't the extent of performance degradation be similar for each experiment?

4
  • Do you reset the app state of all involved apps every time (kill process and clear private app storage)? Because if the app start state is different then monkey may take a different path in each app. Also keep in mind that external factors like network delays can have an impact: If a network request takes a few milliseconds longer so the next monkey action is ignored
    – Robert
    Apr 24 at 10:21
  • At the end of each experiment I do the Android reboot. It might not be enough? @Robert
    – Luigi
    Apr 24 at 10:33
  • Rebooting only kills the processes but leaves their app data so if an app saves internal state to app data and restores the state on next start you may get different results. So consider clearing app data using adb shell pm clear packageName
    – Robert
    Apr 24 at 10:41
  • All right, thank you. One last thing. Should I clean only the apps that are part of my workload or would it be appropriate to clean others as well? @Robert
    – Luigi
    Apr 24 at 10:57

0

You must log in to answer this question.

Browse other questions tagged .