Canonical explanation of Android Encryption and Vulnerabilities

Question

Note: Well, the bounty expired and one possible reason could be effort required to address as I gather from comments. Seeing the number of upvotes, it appears to be of interest to others as well. I would still like to get an answer so here's what I propose- a good answer within a month, will get a bonus of 50. This would I hope give adequate time and incentive

---

I have been trying to understand Android Encryption process and its vulnerabilities for a while

There are many questions addressing portions of this topic on this site and also on sister site. To drive my point home, these questions address portions and not the whole (reminiscent of "blind men and an elephant ?" :)

• What is Marshmallow encryption protecting?
• Does full device encryption protect my data from Google and the government?
• https://android.stackexchange.com/questions/137334/if-we-encrypt-our-device-can-user-connecting-to-adb-get-it
• Questions pertaining to SD card encryption
• android-encryption-when-phone-is-on
• android-cyanogenmod-encryption-vs-gnu
• is-there-cryptographic-material-in-a-phones-sim-card-that-can-be-used-with-rsa
• android-device-encryption

My understanding (or misunderstanding ?)

1. Encryption password is generated from a combination of user lock screen PIN and encryption algorithm (therein lies an inherent weakness owing to limited length of PIN)
2. This is salted, and stored in root location, not accessible to users
3. This is used to generate the actual password to encrypt /decrypt and the actual password is stored in RAM
4. This was strengthened by linking Step 1 to the device SoC (Which Android Version ? Which is the hardware element that uniquely identifies the device? Can that be replaced to fake?)
5. Hence, it is not possible to decrypt data without encryption key and device (holds for external SD as well)
6. Possible recovery methods - brute force, capturing RAM information (step 3) to obtain key
7. Rooted devices appear to be more susceptible to access step 2 data through custom recovery/ possibly ROM and kernel flashing?? (if true, why is this not touted as a big risk?)
8. Even if this information is obtained , I am guessing that it is non-trivial effort wise to generate the actual password
9. Marshmallow can treat external SD as "internal storage" or "portable storage". Logically, it shouldn't make a difference but am not sure

There are gaps in my understanding, probably missing out on other key aspects as well.

So, I am looking for a canonical explanation for understanding from a user perspective

• Entire encryption process (including external SD)

• Implementation variation across Android versions- from KitKat to Marshmallow (including dual options for external SD in Marshmallow)

• Vulnerabilities at a user level

Note

• I am aware of the risk of the question being considered too broad but IMO warrants a comprehensive treatment

• Having some experience in communication security, I understand the challenge in translating Cryptographic concepts to a user level. I would prefer the answer to address this, with explanatory pointers for deeper understanding. Examples of the process need not be Cryptographically correct in a rigorous sense but should convey the essence

• A possible advantage could be "duping" future questions on related aspects

• At the cost of repetition, answers should be primarily at user level, but with adequate explanation for a deeper understanding. Splitting the answer in two portions may be a suitable way.

• I would make it a point to down vote trivial/casual/patch work answers to encourage comprehensive answers

Answer 1

I envision it works like this:

• Storage is encrypted using synchronous random key.
• When the user chooses or changes a password that is based on whatever input, be it a password comprised of letters and numbers and characters, or be it a pin code, or pattern swipe, or finger print, or any other input, an asynchronous encryption algorithm is used to encrypt the master key, such that correct identification ends up decrypting the input resulting in the master key, which in turn makes it possible to encrypt and decrypt storage.
• The moment the user logs out, the memory holding the master key is overwritten

The big trick here is that asynchronous encrypting of the master key. Once Android has the master key, it has the ability to exchange data with the storage. Only when the user is logged in, that master key is known. Asynchronous encryption is what is called public key encryption. What happens is that a public key encrypts data (the master key in this case), and a private key decrypts data. Not to be confused with the storage encryption here. The storage is just synchronous encryption. There the same key is used to encrypt and decrypt. But the finding/retrieval of that "master" key, is the biggie. It means that if at one point you have a weak login method, like for intance "1234" as a pincode, and you change your mind, and change the pincode to "5364", which is harder to guess, unless that earlier "1234" was stolen, snooped on, at any point, the security has just become better. Same deal when changing the login method to a full password that is impossible to guess or dictionary attack. The storage itself does not need to be re-encrypted at all. It's all about hiding that master key - internally. The user never sees that master key, because it's most probably just a random hash code of sorts - nothing will ever "find" or "guess" that hash code. Not even the NSA or any other security body on the planet could ever find a matching key like that. The only attack vector is hoping for a weakness on the user's part. Perhaps the user chose a pincode login. If it's 4 digits, then it's a maximum of 10000 possible pincodes. The OS might "block" the device after trying out a few in a short time. The solution is then to "hack" the OS, so that it becomes possible to try all possible pincodes without the OS intervening and blocking the device. I believe that is how the FBI eventually got access to a phone of a criminal. A third party company (some Israeli company from what I recall) did the hacking for the FBI I think. They bypassed that pincode-try limit. If the login is a full password, and if the user picked a strong password, and you're sol. Not in a life time with all the cpu power on the planet will hack that in a million years. I'm not buying any of the NSA can decrypt anything rumors. I think those people watched one too many men-in-black movies. All one has to do is look at the scientific documents about the various encryption algorithms (eg. AES), and you'll know that hacking simply won't happen, except in the olden days when there were 40 bit keys. Those days are long gone. AES128 is already unhackable I think, and if anyone is concerned, jumping to AES256 makes it more secure by a magnitude to the size of the universe pretty much. Maybe one day quantum computers might decrypt it, but I'm skeptical. Not sure if it is possible to have a probability system simply highlight the solution. We'll see about that, eventually. Perhaps that's a few lifetimes away anyway. Nothing to worry about right now.

So, end of day, the security limitation lies entirely on the login method that is used. One can change the method without having to re-encrypt the storage. All this because of the asynchronous public key encryption of the master key.

Answer 2

Because updates are frequent, the way encryption on the phone (Android-based OS) is handled can change from one build to the next. Therefore, primary concern is not with encryption itself, but where the process is running. And if that platform has vulnerabilities, then the strength of encryption algorithm itself becomes of little or no importance.

Basically, once your device decrypts file(s), they can be directly accessed by a process with Super User privileges. This process could gain access to your device by exploiting a weakness in the ROM (Android OS) itself. (This was recently in the news as some flaws were exposed by WikiLeaks)

Rooted devices appear to be more susceptible to access step 2 data through custom recovery/ possibly ROM and kernel flashing?? (if true, why is this not touted as a big risk?)

Before root: To root a device you have to use external tools all of which have deep access into internal structure of the device. Some of these tools are precompiled and are not open source. They do have "official" websites, but who are these people? (twrp.me, supersu.com for example, but there are others like KingoRoot) Can we really trust them? I trust some more than the others. For example KingoRoot installed a program on my pc which behaved in a virus-like fashion (had to use dual-boot to remove it).

After you root: giving a compiled program (APK) a SU access means it can do anything it wants without restrictions or stating which Intent it will be using. (Intents are way for APKs to access things like WiFi, Camera etc) So a "well trusted app", after given root access, can easily access any kind of information and send it back to its server.

Does full device encryption protect my data from Google and the government?

Google - yes. It does not have the key to unlocking.

Government (or hacker) - no. because Government or hacker can essentially use an exploit which will intercept the file(s) as I mentioned above.

The complexities of security procedures/algorithms are of little use if they can be intercepted and bypassed.

Edit: It's worth mentioning that Google actually has ability to download and install/update apps onto your android device without asking your permission, or even notifying you that update has taken place. And even on rooted device, there doesn't seem to be a way to block this without loosing key functions (Play Store, Maps, Sync, etc)