15

For some time I've had a problem, where crucial system processes were being force killed by LowmemoryKiller/OOM.

Following help, I added a 500 MB swap file working natively a couple of days ago to try and reduce memory pressure and lowmemkiller triggers (thread with details here). The output of free showed an extra 500 MB of space, which I figured was plenty to prevent whatever low memory problem was forcibly killing system processes.

But it hasn't helped. Out of the blue I suddenly again got a fairly vicious "everything got killed by LowMemKiller" just now, with about 6 core dump files created in just 100 seconds, followed by a user space crash, and the kernel (dmesg) log suggest that low memory issues were the only reported problem, even though this time a large amount of swap was available and recognised by the system (see free below).

Here's the relevant part of dmesg >dmesg.txt shortly after init restarted userspace. In the first part of the log, long periods of several seconds pass where nothing new happens; when userspace is killed off during the 100 seconds at 2105-2208 there is nothing beforehand and no hint of cause.

Phone turned on

<6>[001992.334927,0] : Report pwrkey press event
<6>[001992.386743,0] mdss_dsi_on+: ctrl=eaec8010 ndx=0
<6>[001992.424178,0] mdss_dsi_on-: 
<6>[001992.424193,0] mdss_dsi_panel_on+: ctrl=eaec8010 ndx=0
<6>[001992.428789,0] : Report pwrkey release event
<6>[001993.730749,0] synaptics_dsx_sensor_state: state change SUSPEND -> ACTIVE
<6>[001993.731405,0] : mdss_dsi_panel_on-. Pwr_mode(0x0A) = 0x9c
<6>[001993.831117,0] qcom,leds-qpnp leds-qpnp-eb62f600: backlight on

Nothing in log for 40 seconds. Then there are several bursts of low memory killing, separated by brief gaps of about 5-20 seconds

<6>[002033.125104,1] lowmemorykiller: Killing 'system:ui' (8151), adj 1000, to free 40656kB on behalf of 'kswapd0' (81) because cache 122600kB is below limit 122880kB for oom_score_adj 1000. Free memory is 3200kB above reserved
<6>[002034.172404,2] lowmemorykiller: Killing 'changelogviewer' (7238), adj 1000, to free 32652kB on behalf of 'kswapd0' (81) because cache 121376kB is below limit 122880kB for oom_score_adj 1000. Free memory is 6776kB above reserved
<6>[002035.895963,0] : Report pwrkey press event
<6>[002035.307927,1] lowmemorykiller: Killing 'droid.deskclock' (28714), adj 1000, to free 31756kB on behalf of 'kswapd0' (81) because cache 120252kB is below limit 122880kB for oom_score_adj 1000. Free memory is 5532kB above reserved
<6>[002036.387002,0] : Report pwrkey release event
<6>[002036.478425,0] lowmemorykiller: Killing 'com.ebay.mobile' (26933), adj 1000, to free 29016kB on behalf of 'kswapd0' (81) because cache 120268kB is below limit 122880kB for oom_score_adj 1000. Free memory is 3864kB above reserved
<6>[002037.923711,2] lowmemorykiller: Killing 'omm.timeservice' (6682), adj 1000, to free 24544kB on behalf of 'kswapd0' (81) because cache 112140kB is below limit 122880kB for oom_score_adj 1000. Free memory is 3072kB above reserved
<6>[002037.070531,1] lowmemorykiller: Killing 'ndroid.settings' (5632), adj 1000, to free 24344kB on behalf of 'kswapd0' (81) because cache 108888kB is below limit 122880kB for oom_score_adj 1000. Free memory is 5484kB above reserved
<6>[002037.617332,1] lowmemorykiller: Killing 'd.process.media' (5610), adj 1000, to free 23552kB on behalf of 'kswapd0' (81) because cache 109164kB is below limit 122880kB for oom_score_adj 1000. Free memory is 3964kB above reserved
<6>[002038.335908,0] lowmemorykiller: Killing 'enmod.lockclock' (28743), adj 1000, to free 21712kB on behalf of 'kswapd0' (81) because cache 107296kB is below limit 122880kB for oom_score_adj 1000. Free memory is 4616kB above reserved
<6>[002039.245833,1] lowmemorykiller: Killing 'ainfire.supersu' (8025), adj 764, to free 32380kB on behalf of 'kswapd0' (81) because cache 98160kB is below limit 98304kB for oom_score_adj 529. Free memory is 7484kB above reserved
<6>[002040.109486,0] lowmemorykiller: Killing 'd.process.acore' (8429), adj 647, to free 34448kB on behalf of 'kswapd0' (81) because cache 97952kB is below limit 98304kB for oom_score_adj 529. Free memory is 8732kB above reserved
<6>[002043.386164,2] lowmemorykiller: Killing 'putmethod.latin' (14570), adj 764, to free 20732kB on behalf of 'kswapd0' (81) because cache 97896kB is below limit 98304kB for oom_score_adj 529. Free memory is 3564kB above reserved
<6>[002047.012564,0] tapan_codec_chargepump_vdd_event: event = 1
<6>[002048.999115,2] lowmemorykiller: Killing 'd.process.media' (8782), adj 647, to free 33804kB on behalf of 'kswapd0' (81) because cache 95364kB is below limit 98304kB for oom_score_adj 529. Free memory is 4556kB above reserved
<6>[002049.385765,0] lowmemorykiller: Killing 'com.android.mms' (370), adj 529, to free 42768kB on behalf of 'kswapd0' (81) because cache 96588kB is below limit 98304kB for oom_score_adj 529. Free memory is 5264kB above reserved
<6>[002051.397263,0] init: waitpid returned pid 30519, status = 00000000
<6>[002052.710665,1] tapan_codec_chargepump_vdd_event: event = 8
<6>[002054.259864,2] lowmemorykiller: Killing 'd.process.media' (8882), adj 647, to free 33620kB on behalf of 'kswapd0' (81) because cache 93904kB is below limit 98304kB for oom_score_adj 529. Free memory is 7752kB above reserved
<6>[002055.100549,3] lowmemorykiller: Killing 'ndroid.settings' (8939), adj 764, to free 34900kB on behalf of 'kswapd0' (81) because cache 96324kB is below limit 98304kB for oom_score_adj 529. Free memory is 3920kB above reserved
<6>[002056.176783,2] lowmemorykiller: Killing 'd.process.acore' (8930), adj 764, to free 35656kB on behalf of 'kswapd0' (81) because cache 89744kB is below limit 98304kB for oom_score_adj 529. Free memory is 3688kB above reserved
<6>[002064.446038,2] lowmemorykiller: Killing 'd.process.media' (9044), adj 764, to free 33580kB on behalf of 'kswapd0' (81) because cache 90032kB is below limit 98304kB for oom_score_adj 529. Free memory is 14876kB above reserved
<6>[002083.199303,1] lowmemorykiller: Killing 'enmod.lockclock' (8974), adj 647, to free 31236kB on behalf of 'kswapd0' (81) because cache 98132kB is below limit 98304kB for oom_score_adj 529. Free memory is 3308kB above reserved
<6>[002084.892508,1] lowmemorykiller: Killing 'ndroid.settings' (9068), adj 529, to free 53876kB on behalf of 'kswapd0' (81) because cache 95092kB is below limit 98304kB for oom_score_adj 529. Free memory is 5380kB above reserved
<6>[002105.187792,0] lowmemorykiller: Killing 'enmod.trebuchet' (20303), adj 352, to free 33356kB on behalf of 'kswapd0' (81) because cache 80972kB is below limit 86016kB for oom_score_adj 176. Free memory is 3004kB above reserved
<6>[002106.510875,0] lowmemorykiller: Killing 'com.android.mms' (8906), adj 470, to free 39524kB on behalf of 'kswapd0' (81) because cache 80488kB is below limit 86016kB for oom_score_adj 176. Free memory is 16500kB above reserved
<6>[002111.395568,1] lowmemorykiller: Killing 'd.process.media' (9718), adj 529, to free 33576kB on behalf of 'kswapd0' (81) because cache 70448kB is below limit 73728kB for oom_score_adj 117. Free memory is 3484kB above reserved
<6>[002112.755785,1] lowmemorykiller: Killing 'ndroid.settings' (9766), adj 529, to free 34872kB on behalf of 'kswapd0' (81) because cache 69160kB is below limit 73728kB for oom_score_adj 117. Free memory is 3792kB above reserved
<6>[002112.892462,1] lowmemorykiller: Killing 'com.android.mms' (9736), adj 470, to free 35356kB on behalf of 'kswapd0' (81) because cache 68408kB is below limit 73728kB for oom_score_adj 117. Free memory is 7628kB above reserved
<6>[002112.906543,3] binder: release 9736:9755 transaction 7689487 out, still active
<6>[002112.386885,0] lowmemorykiller: Killing 'd.process.acore' (9757), adj 647, to free 34860kB on behalf of 'kswapd0' (81) because cache 69340kB is below limit 73728kB for oom_score_adj 117. Free memory is 4092kB above reserved
<6>[002113.952913,0] lowmemorykiller: Killing 'enmod.lockclock' (9808), adj 647, to free 31048kB on behalf of 'kswapd0' (81) because cache 73224kB is below limit 73728kB for oom_score_adj 117. Free memory is 3852kB above reserved
<6>[002114.465873,3] lowmemorykiller: Killing 'd.process.media' (9872), adj 529, to free 33528kB on behalf of 'kswapd0' (81) because cache 67308kB is below limit 73728kB for oom_score_adj 117. Free memory is 3540kB above reserved
<6>[002115.944547,3] lowmemorykiller: Killing 'd.process.acore' (9920), adj 647, to free 34788kB on behalf of 'kswapd0' (81) because cache 62112kB is below limit 73728kB for oom_score_adj 117. Free memory is 3876kB above reserved
<6>[002115.463421,3] lowmemorykiller: Killing 'ndroid.settings' (9912), adj 647, to free 34804kB on behalf of 'kswapd0' (81) because cache 61372kB is below limit 61440kB for oom_score_adj 58. Free memory is 3444kB above reserved
<6>[002116.625036,3] lowmemorykiller: Killing 'enmod.lockclock' (9947), adj 764, to free 31128kB on behalf of 'kswapd0' (81) because cache 60288kB is below limit 61440kB for oom_score_adj 58. Free memory is 3540kB above reserved
<6>[002116.660223,3] lowmemorykiller: Killing 'd.process.acore' (9960), adj 764, to free 34912kB on behalf of 'kswapd0' (81) because cache 59824kB is below limit 61440kB for oom_score_adj 58. Free memory is 3560kB above reserved
<6>[002131.486032,2] lowmemorykiller: Killing 'd.process.acore' (10031), adj 764, to free 35472kB on behalf of 'kswapd0' (81) because cache 67868kB is below limit 73728kB for oom_score_adj 117. Free memory is 21432kB above reserved
<6>[002132.304917,0] lowmemorykiller: Killing 'd.process.media' (10018), adj 1000, to free 33728kB on behalf of 'Compositor' (8575) because cache 100976kB is below limit 122880kB for oom_score_adj 1000. Free memory is 3492kB above reserved
<6>[002132.374198,1] lowmemorykiller: Killing 'ndroid.settings' (10049), adj 764, to free 34792kB on behalf of 'kswapd0' (81) because cache 64912kB is below limit 73728kB for oom_score_adj 117. Free memory is 3280kB above reserved
<6>[002132.435346,1] lowmemorykiller: Killing 'enmod.lockclock' (10067), adj 647, to free 30996kB on behalf of 'Compositor' (8575) because cache 97520kB is below limit 98304kB for oom_score_adj 529. Free memory is 3364kB above reserved
<6>[002139.202616,3] lowmemorykiller: Killing 'com.android.mms' (9890), adj 470, to free 38940kB on behalf of 'kswapd0' (81) because cache 77428kB is below limit 86016kB for oom_score_adj 176. Free memory is 3352kB above reserved
<6>[002144.987619,0] lowmemorykiller: Killing 'd.process.media' (10354), adj 529, to free 33492kB on behalf of 'kswapd0' (81) because cache 70876kB is below limit 73728kB for oom_score_adj 117. Free memory is 3952kB above reserved
<6>[002145.868772,0] lowmemorykiller: Killing 'ndroid.settings' (10410), adj 647, to free 34748kB on behalf of 'kswapd0' (81) because cache 70496kB is below limit 73728kB for oom_score_adj 117. Free memory is 9208kB above reserved
<6>[002145.500683,0] lowmemorykiller: Killing 'enmod.lockclock' (10439), adj 647, to free 30964kB on behalf of 'kswapd0' (81) because cache 69148kB is below limit 73728kB for oom_score_adj 117. Free memory is 4848kB above reserved
<6>[002146.860274,2] lowmemorykiller: Killing 'd.process.acore' (10393), adj 764, to free 35316kB on behalf of 'kswapd0' (81) because cache 66996kB is below limit 73728kB for oom_score_adj 117. Free memory is 8484kB above reserved
<6>[002146.883536,0] binder: undelivered transaction 7710205
<6>[002147.021644,1] lowmemorykiller: Killing 'd.process.acore' (10507), adj 764, to free 35104kB on behalf of 'kswapd0' (81) because cache 66948kB is below limit 73728kB for oom_score_adj 117. Free memory is 8344kB above reserved
<6>[002147.496709,0] lowmemorykiller: Killing 'd.process.media' (10495), adj 764, to free 33760kB on behalf of 'kswapd0' (81) because cache 65184kB is below limit 73728kB for oom_score_adj 117. Free memory is 8972kB above reserved
<6>[002166.712750,0] lowmemorykiller: Killing 'ndroid.settings' (10529), adj 764, to free 34780kB on behalf of 'kswapd0' (81) because cache 74940kB is below limit 86016kB for oom_score_adj 176. Free memory is 4768kB above reserved
<6>[002166.041116,1] lowmemorykiller: Killing 'enmod.lockclock' (10545), adj 647, to free 30984kB on behalf of 'kswapd0' (81) because cache 76512kB is below limit 86016kB for oom_score_adj 176. Free memory is 4996kB above reserved
<6>[002183.893105,1] lowmemorykiller: Killing 'com.ebay.mobile' (10668), adj 529, to free 46620kB on behalf of 'kswapd0' (81) because cache 71236kB is below limit 73728kB for oom_score_adj 117. Free memory is 4588kB above reserved
<6>[002183.175603,1] lowmemorykiller: Killing 'com.android.mms' (10372), adj 294, to free 39168kB on behalf of 'kswapd0' (81) because cache 69516kB is below limit 73728kB for oom_score_adj 117. Free memory is 4568kB above reserved
<6>[002184.262462,1] lowmemorykiller: Killing 'putmethod.latin' (20205), adj 117, to free 31552kB on behalf of 'kswapd0' (81) because cache 64740kB is below limit 73728kB for oom_score_adj 117. Free memory is 4528kB above reserved
<6>[002185.901421,1] lowmemorykiller: Killing 'pal.androidterm' (20591), adj 117, to free 22780kB on behalf of 'kswapd0' (81) because cache 62628kB is below limit 73728kB for oom_score_adj 117. Free memory is 4548kB above reserved
<6>[002185.077441,0] init: waitpid returned pid 26289, status = 0000fc00
<6>[002185.611288,2] lowmemorykiller: Killing 'android.smspush' (20927), adj 58, to free 16176kB on behalf of 'kswapd0' (81) because cache 56236kB is below limit 61440kB for oom_score_adj 58. Free memory is 3392kB above reserved
<6>[002186.955956,0] init: waitpid returned pid 26285, status = 00000001
<6>[002186.006426,1] init: waitpid returned pid 26256, status = 00008100
<6>[002186.431086,0] lowmemorykiller: Killing 'mozilla.firefox' (6712), adj 0, to free 354160kB on behalf of 'kswapd0' (81) because cache 46540kB is below limit 49152kB for oom_score_adj 0. Free memory is 5192kB above reserved
<6>[002186.606429,1] init: waitpid returned pid 19765, status = 00000000
<6>[002191.881505,2] lowmemorykiller: Killing 'd.process.media' (10858), adj 1000, to free 33140kB on behalf of 'kswapd0' (81) because cache 87004kB is below limit 98304kB for oom_score_adj 529. Free memory is 3960kB above reserved
<6>[002191.291965,1] lowmemorykiller: Killing 'd.process.acore' (10918), adj 764, to free 35228kB on behalf of 'kswapd0' (81) because cache 81776kB is below limit 86016kB for oom_score_adj 176. Free memory is 3832kB above reserved
<6>[002191.591134,1] lowmemorykiller: Killing 'ndroid.settings' (10928), adj 647, to free 34460kB on behalf of 'kswapd0' (81) because cache 77724kB is below limit 86016kB for oom_score_adj 176. Free memory is 4280kB above reserved
<6>[002192.697565,2] lowmemorykiller: Killing 'enmod.lockclock' (10957), adj 529, to free 30720kB on behalf of 'kswapd0' (81) because cache 76880kB is below limit 86016kB for oom_score_adj 176. Free memory is 3604kB above reserved
<6>[002192.739096,2] lowmemorykiller: Killing 'com.android.mms' (10883), adj 470, to free 38292kB on behalf of 'kswapd0' (81) because cache 74704kB is below limit 86016kB for oom_score_adj 176. Free memory is 7352kB above reserved
<6>[002193.555477,2] lowmemorykiller: Killing 'enmod.trebuchet' (10844), adj 0, to free 58452kB on behalf of 'kswapd0' (81) because cache 48516kB is below limit 49152kB for oom_score_adj 0. Free memory is 3116kB above reserved
<6>[002194.171773,1] binder: undelivered transaction 7731151
<6>[002194.355847,0] lowmemorykiller: Killing 'd.process.acore' (11091), adj 529, to free 34780kB on behalf of 'kswapd0' (81) because cache 47368kB is below limit 49152kB for oom_score_adj 0. Free memory is 8656kB above reserved
<6>[002194.554944,0] lowmemorykiller: Killing 'd.process.media' (11118), adj 529, to free 33392kB on behalf of 'enmod.trebuchet' (11110) because cache 81508kB is below limit 86016kB for oom_score_adj 176. Free memory is 6088kB above reserved
<6>[002194.670126,0] lowmemorykiller: Killing 'com.android.mms' (11151), adj 294, to free 34528kB on behalf of 'kswapd0' (81) because cache 47332kB is below limit 49152kB for oom_score_adj 0. Free memory is 3564kB above reserved
<6>[002194.680946,2] binder: release 11151:11172 transaction 7731784 out, still active
<6>[002194.681060,2] binder: undelivered transaction 7731944
<6>[002195.770478,1] lowmemorykiller: Killing 're-initialized>' (11177), adj 529, to free 26236kB on behalf of 're-initialized>' (11186) because cache 80372kB is below limit 86016kB for oom_score_adj 176. Free memory is 3968kB above reserved
<6>[002195.828751,0] lowmemorykiller: Killing 'enmod.trebuchet' (11110), adj 0, to free 44916kB on behalf of 'kswapd0' (81) because cache 46796kB is below limit 49152kB for oom_score_adj 0. Free memory is 3280kB above reserved
<6>[002195.996821,2] lowmemorykiller: Killing 'd.process.acore' (11206), adj 529, to free 28148kB on behalf of 'kswapd0' (81) because cache 46216kB is below limit 49152kB for oom_score_adj 0. Free memory is 3744kB above reserved
<6>[002195.006525,3] binder: undelivered transaction 7732259
<6>[002195.111464,2] lowmemorykiller: Killing 'ndroid.settings' (11186), adj 529, to free 34436kB on behalf of 'd.process.acore' (11236) because cache 76684kB is below limit 86016kB for oom_score_adj 176. Free memory is 3412kB above reserved
<6>[002195.189925,1] lowmemorykiller: Killing 're-initialized>' (11236), adj 529, to free 27684kB on behalf of 're-initialized>' (11250) because cache 75264kB is below limit 86016kB for oom_score_adj 176. Free memory is 3748kB above reserved
<6>[002195.582460,1] lowmemorykiller: Killing 'enmod.lockclock' (11250), adj 647, to free 30716kB on behalf of 'kswapd0' (81) because cache 42068kB is below limit 49152kB for oom_score_adj 0. Free memory is 3936kB above reserved
<6>[002195.680729,3] lowmemorykiller: Killing 'd.process.acore' (11265), adj 529, to free 33160kB on behalf of 'kswapd0' (81) because cache 41232kB is below limit 49152kB for oom_score_adj 0. Free memory is 3568kB above reserved
<6>[002196.766706,0] lowmemorykiller: Killing 'enmod.trebuchet' (11217), adj 0, to free 43560kB on behalf of 'kswapd0' (81) because cache 40444kB is below limit 49152kB for oom_score_adj 0. Free memory is 3448kB above reserved
<6>[002201.374233,0] : Report pwrkey press event
<6>[002201.518183,0] : Report pwrkey release event
<6>[002202.657554,1] mdss_dsi_panel_off+: ctrl=eaec8010 ndx=0
<6>[002202.657986,0] synaptics_dsx_sensor_state: state change ACTIVE -> SUSPEND
<6>[002203.897141,0] mdss_dsi_panel_off-: 
<6>[002203.897156,0] mdss_dsi_off+: ctrl=eaec8010 ndx=0
<6>[002203.936876,0] mdss_dsi_off-: 

At this point anything that hasn't already failed, now fails. Userspace gets killed and restarted

<6>[002203.037750,0] : Core dump to |/system/bin/coredump Binder_4 19920 1417523206 pipe failed
<6>[002203.281135,0] binder: undelivered transaction 7733258
<6>[002203.281314,0] binder: undelivered transaction 7733265
<6>[002203.281444,0] binder: undelivered transaction 7733271
<6>[002203.281597,0] binder: undelivered transaction 7733366
<6>[002203.281681,0] binder: undelivered transaction 7733378
<6>[002203.282023,0] binder: undelivered transaction 7733372
<6>[002203.282108,0] binder: undelivered transaction 7733373
<6>[002203.282258,0] binder: undelivered transaction 7733374
<6>[002203.282340,0] binder: undelivered transaction 7733375
<6>[002203.282488,0] binder: undelivered transaction 7733376
<6>[002204.093925,0] : Report pwrkey press event
<6>[002204.306457,0] : Report pwrkey release event
<6>[002207.955756,0] : Core dump to |/system/bin/coredump system_server 1142 1417523210 pipe failed
<6>[002207.052286,0] mdss_dsi_on+: ctrl=eaec8010 ndx=0
<6>[002207.075781,1] alarm_release: clear alarm, pending 0
<6>[002207.075791,1] alarm_release: clear alarm, pending 0
<6>[002207.075799,1] alarm_release: clear alarm, pending 0
<6>[002207.089757,0] mdss_dsi_on-: 
<6>[002207.089772,0] mdss_dsi_panel_on+: ctrl=eaec8010 ndx=0
<6>[002207.104989,1] init: computing context for service '/system/bin/bootanimation'
<5>[002207.106139,1] init: starting 'bootanim'
<6>[002207.136155,1] : Core dump to |/system/bin/coredump Binder_7 1387 1417523210 pipe failed
<6>[002207.147220,1] binder: undelivered death notification, 60d6b368
<6>[002207.147232,1] binder: undelivered death notification, 60d6b780
<6>[002207.147241,1] binder: undelivered death notification, 5cdca680
<6>[002207.147249,1] binder: undelivered death notification, 5bf36ec0
<6>[002207.147261,1] binder: undelivered death notification, 5cdcb4a0
<6>[002207.147272,1] binder: undelivered death notification, 5cdd1880
<6>[002207.147281,1] binder: undelivered death notification, 5cdcb460
<6>[002207.185322,1] init: waitpid returned pid 385, status = 00000009
<5>[002207.185340,1] init: process 'debuggerd', pid 385 exited
<5>[002207.185355,1] init: process 'debuggerd' killing any children in process group
<6>[002207.185940,1] init: computing context for service '/system/bin/debuggerd'
<6>[002207.185963,2] : Core dump to |/system/bin/coredump Binder_2 1400 1417523210 pipe failed
<5>[002207.186211,1] init: starting 'debuggerd'
<6>[002207.186764,2] : Core dump to |/system/bin/coredump Binder_4 1413 1417523210 pipe failed
<6>[002207.194796,2] binder: undelivered death notification, 5cdcf108
<6>[002207.194807,2] binder: undelivered death notification, 60b5faa8
<6>[002207.194816,2] binder: undelivered death notification, 60b5fec0
<6>[002207.194824,2] binder: undelivered death notification, 5cdca680
<6>[002207.194832,2] binder: undelivered death notification, 601f2830
<6>[002207.194840,2] binder: undelivered death notification, 60def500
<6>[002207.194849,2] binder: undelivered death notification, 5cdd1488
<6>[002207.194857,2] binder: undelivered death notification, 60def1f0
<6>[002207.200117,2] binder: undelivered death notification, 5cdce0d8
<6>[002207.200129,2] binder: undelivered death notification, 60c6b0c8
<6>[002207.200137,2] binder: undelivered death notification, 60c6b4e0
<6>[002207.200146,2] binder: undelivered death notification, 5cdca670
<6>[002207.200154,2] binder: undelivered death notification, 5cdd1940
<6>[002207.200163,2] binder: undelivered death notification, 5d84f710
<6>[002207.200171,2] binder: undelivered death notification, 5cdd1430
<6>[002207.200179,2] binder: undelivered death notification, 5cdd1f90
<6>[002207.276277,1] init: waitpid returned pid 388, status = 00000009
<5>[002207.276294,1] init: process 'zygote', pid 388 exited
<5>[002207.285604,0] init: process 'zygote' killing any children in process group
<5>[002207.285863,0] init: service 'media' is being killed
<5>[002207.287087,0] init: service 'netd' is being killed
<6>[002207.287441,0] init: computing context for service '/system/bin/app_process'
<5>[002207.287638,0] init: starting 'zygote'
<6>[002207.289856,0] init: waitpid returned pid 383, status = 00000009
<5>[002207.289874,0] init: process 'netd', pid 383 exited
<5>[002207.289888,0] init: process 'netd' killing any children in process group
<6>[002207.290323,0] init: computing context for service '/system/bin/netd'
<5>[002207.290529,0] init: starting 'netd'
<6>[002207.297300,0] init: waitpid returned pid 391, status = 00000009
<5>[002207.297316,0] init: process 'media', pid 391 exited
<5>[002207.297331,0] init: process 'media' killing any children in process group
<6>[002207.297373,2] init: Created socket '/dev/socket/mdns' with mode '660', user '0', group '1000'
<6>[002207.297610,0] init: computing context for service '/system/bin/mediaserver'
<6>[002207.297768,2] init: Created socket '/dev/socket/dnsproxyd' with mode '660', user '0', group '3003'
<5>[002207.297778,0] init: starting 'media'
<6>[002207.298187,2] init: Created socket '/dev/socket/netd' with mode '660', user '0', group '1000'
<6>[002207.298939,2] init: Created socket '/dev/socket/zygote' with mode '660', user '0', group '1000'
<6>[002207.315254,0] : Report pwrkey press event
<6>[002207.386012,0] synaptics_dsx_sensor_state: state change SUSPEND -> ACTIVE
<6>[002207.386197,0] : mdss_dsi_panel_on-. Pwr_mode(0x0A) = 0x9c
<6>[002207.446161,0] init: waitpid returned pid 1373, status = 00000009
<6>[002208.865272,0] : Report pwrkey release event
<7>[002210.508025,0] SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
<6>[002213.793599,0] : Report pwrkey press event
<6>[002213.970096,0] : Report pwrkey release event
<6>[002213.447327,0] : Core dump to |/system/bin/coredump zygote 11352 1417523217 pipe failed
<6>[002213.508195,0] init: waitpid returned pid 11352, status = 0000000b
<5>[002213.508377,0] init: process 'zygote', pid 11352 exited
<5>[002213.508475,0] init: process 'zygote' killing any children in process group
<5>[002213.508759,0] init: service 'media' is being killed
<5>[002213.512948,0] init: service 'netd' is being killed
<6>[002213.513683,0] init: waitpid returned pid 11355, status = 00000009
<5>[002213.513805,0] init: process 'media', pid 11355 exited
<5>[002213.513981,0] init: process 'media' killing any children in process group
<6>[002213.514315,0] init: computing context for service '/system/bin/app_process'
<5>[002213.514729,0] init: starting 'zygote'
<6>[002213.515310,0] init: computing context for service '/system/bin/mediaserver'
<6>[002213.516306,1] init: Created socket '/dev/socket/zygote' with mode '660', user '0', group '1000'
<5>[002213.516644,0] init: starting 'media'
<6>[002213.520811,0] init: waitpid returned pid 11353, status = 00000009
<5>[002213.520998,0] init: process 'netd', pid 11353 exited
<5>[002213.521240,0] init: process 'netd' killing any children in process group
<6>[002213.521753,0] init: computing context for service '/system/bin/netd'
<5>[002213.522248,0] init: starting 'netd'
<6>[002213.526276,1] init: Created socket '/dev/socket/mdns' with mode '660', user '0', group '1000'
<6>[002213.526874,1] init: Created socket '/dev/socket/dnsproxyd' with mode '660', user '0', group '3003'
<6>[002213.527403,1] init: Created socket '/dev/socket/netd' with mode '660', user '0', group '1000'

However swap was active and it's not clear what's happening. Output from free immediately after userspace is restored:

             total         used         free       shared      buffers
Mem:        903272       879320        23952            0         1844
-/+ buffers:             877476        25796
Swap:       687060        19024       668036

No idea what's going on. Exotic guesses FWIW: (1) Swap is available and enabled but for some reason wasn't used in the event, when memory got low; (2) some process suddenly decided it needed about 1GB of RAM and to run in foreground, causing failed attempts to swap crucial but lower prioritised system processes; (3) Unpredictable dramatic memory leak.

I don't know where to go from here.

The phone runs rooted CyanogenMod 11 M6 and has 1GB ram, and the apps are pretty standard - firefox, whatsapp, nothing unusual, no strange "tweaking", no strange modding apps.

How can I resolve this issue?

UPDATE I realised that I could find the relevant core dumps. /data/tombstones contains 6 core dumps that were all created during the 100 second period that lowmemorykiller bursts were occurring in dmesg. The core dump files mostly show segmentation faults (SIGSEGV) but the owning process is different in each dump:

2014-12-02 12:27:33 name: Thread-20  (lbesec.loader) signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr b3baed71
2014-12-02 12:27:11 name: zygote  (zygote) signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 5dd8ce64
2014-12-02 12:27:05 name: zygote  (zygote) signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 5dd86e64
2014-12-02 12:26:57 name: zygote  (zygote) signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 5dc03e64
2014-12-02 12:26:47 name: system_server  (system_server) signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr deadf00c
2014-12-02 12:26:46 name: Binder_4  (com.android.systemui) signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr cfc08b9a

Past core dumps in the same folder were just as diverse - thread owners included "gallery" for example. Some showed SIGABRT instead of SIGSEGV. Beyond here probably needs developer competencies. What can I do to track down the cause?

  • 2
    +1 for a well formulated question. Excellent presentation. I'm having low memory? issues and FC's on my HTC. I'm not great with looking at logs, but your thoroughness has help me understand it a little bit better (and i didn't need to take a class or read a long book…lol). Thanks. – HasH_BrowN Dec 22 '14 at 17:54
  • Is the phone running slowly? The job of LowMemoryKiller is to release memory when other apps need it, and it does so by releasing memory from apps that currently don't need it. Here is google's documentation on LowMemoryKiller. – Ryan Conrad Dec 28 '14 at 5:54
  • I think it's possible CyanogenMod shutting down could have something to do with this. I've noticed some odd behavior from my system ever since they shut down their web server such as opening the advanced data settings page crashing the settings app. Again, that's a total guess. – Ethan Z Feb 22 '17 at 11:49
  • Look at the date on the question..... Cm inc shut down in late 2016............ – Stilez Feb 22 '17 at 16:32
1

Android has a built-in ram management system thats pretty good at its job. Ram management applications are notorious for being little more than snake oil. At worst actually harmful to its running processes. I would postulate that this memory killer app is the root cause of your problems. Solution. Reflash your device delete the lowmemory app and see how it behaves

  • 2
    -1. You meant well, but your answer is mistaken. LowMemoryKiller is a built-in part of Android, not a third-party RAM-management application. Please see here in order to view the android-4.4 kernel's lowmemorykiller.c code. (Note: the android-4.4 kernel is completely unconnected with Android 4.4 "KitKat". KitKat devices use a much older kernel.) – unforgettableid Aug 26 '16 at 15:46
0

I don't quite understand if the problem happen out of a sudden or if it just happens with CM11 or not. My first recommendation would be factory reset and run with a skeleton selection of apps, less you install from play store better, and see if problem happens. If it does then CM is running some really aggressive values for LMK (maybe update to something newer or build the rom with modified values). If it doesn't means one of your apps is telling android it needs ram and android (maybe due to poorly set dalvik or hwui heaps, or kernel lmk values) freaks out and just starts the mass murder of processes.

-1

If this app wasn't on your phone when you bought it, but got installed afterwards instead, it's possible that the app that is doing this might be malware. I would suggest going straight to Verizon and have them figure out what is going on. It may cost a lot, but if it can be fixed, it's most likely worth it.

  • 1
    See the reply comment by @unforgettableid . Lowmemorykiller is a built-in part of the Android system. Its not an app, much less malware. – Stilez Oct 3 '16 at 14:51
  • Also if you flashed CyanogenMod on your Phone it is most unlikely that the Carrier will have anything to do with it. It is possible that the Manufacturer would repair the Phone, even out of Warranty, as might a 'Cellphone Repair Store'. Search Engines or CyanogenMod Forums may also be useful. If you can get the Phone to operate long enough you might be able to Flash a newer version or restore your Phone with the original that you backed up prior to flashing. A new 'zero dollar phone' from your Carrier is going to cost the least and you have the value of the lesson learned. – Rob Mar 2 '17 at 14:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.