Take the 2-minute tour ×
Android Enthusiasts Stack Exchange is a question and answer site for enthusiasts and power users of the Android operating system. It's 100% free, no registration required.

I created a backup of my Galaxy Nexus with adb backup. The resulting file is named backup.db and it's somehow encrypted.

I wanted to restore the backup, but it stops when it comes to restoring com.android.providers.contacts. I used adb logcat to find out what's going on and found out that com.android.acore crashes during the restore process.

I'd like to gain access to the data in the backup and remove the contacts database to restore everything back to my phone. Are there any other ways restoring the data from the backup?

share|improve this question

migrated from stackoverflow.com May 23 '12 at 18:09

This question came from our site for professional and enthusiast programmers.

3 Answers 3

up vote 6 down vote accepted

The file is not encrypted, unless your specify so when creating the backup. It is however compressed (using deflate). You can find out the exact format by looking at Android source (com/android/server/BackupManagerService.java) code, and, technically, should be able to extract specific data from it. However, IIRC, there are some file integrity checks in place, so it most probably won't work if you just delete a bunch of data from it. Unfortunately the restore command doesn't seem to have an option to restore a particular app/package only or exclude a package.

share|improve this answer
    
Thanks! That's at least a starting point to look inside the file. Would have been easier if I hadn't provided a password for the backup. –  ingorichter May 23 '12 at 5:44
    
If you provided a password, it is indeed encrypted. `BackupManagerService' has details on the actual encryption algorithms, and key derivation paramters (salt, iteration count etc) are written in the file header. Since you know the password, you can derive the key and decrypt the data. So it's still doable, but not particularly easy... –  Nikolay Elenkov May 23 '12 at 5:57
    
Yes, I'm currently extracting everything from BackupManagerService to read the contents of the backup file. It's a good amount of work, but I need my data back... –  ingorichter May 23 '12 at 6:20
    
@ingorichter any progress? –  jon May 29 '12 at 16:38
    
@ingorichter I started working on this and posted a ton of notes below, in a "community wiki" answer. Feel free to add to it. –  jon May 29 '12 at 23:58

I started working on this. I'm posting my results so far here as a "community wiki" answer for two reasons: first, if someone else wants to join in, there's a place to talk; second, if I get pulled away from this project, there'll be hints for someone else to start working.

 

The backup logic on the host is entirely contained within https://github.com/android/platform_system_core/blob/master/adb/commandline.c, in the function named backup. The function is very simple: it validates the command line options, sends the command mostly as-is to the adb daemon on the phone, and writes the phone's output to the file. There isn't even error-checking: if, for example, you refuse the backup on the phone, adb just writes out an empty file.

On the phone, the backup logic starts in service_to_fd() in https://github.com/android/platform_system_core/blob/master/adb/services.c. The function identifies that the command from the host is "backup", and passes the unparsed command to backup_service() in https://github.com/android/platform_system_core/blob/master/adb/backup_service.c. That, in turn, is a thin wrapper around /system/bin/bu, which is a trivial shell script to launch com.android.commands.bu.Backup as then main-class of a new Android app process. That calls ServiceManager.getService("backup") to get the backup service as an IBackupManager, and calls IBackupManager.fullBackup(), passing it the still-unused file descriptor (very indirectly) connected to the backup.ab file on the host.

Control passes to fullBackup() in com.android.server.BackupManagerService, which pops up the GUI asking the user to confirm/reject the backup. When the user do so, acknowledgeFullBackupOrRestore() (same file) is called. If the user approved the request, acknowledgeFullBackupOrRestore() figures out if the backup is encrypted, and passes a message to BackupHandler (same file.) BackupHandler then instantiates and kicks off a PerformFullBackupTask (same file, line 2228 as of time of writing)

We finally start generating output there, in PerformFullBackupTask.run(), between line 2367 and line 2453.

First, run() writes a header, which consists of either 4 or 9 ASCII lines:

  1. "ANDROID BACKUP"
  2. the backup format version: currently always "1"
  3. either "0" if the backup is uncompressed or "1" if it is
  4. the encryption method: currently either "none" or "AES-256"
  5. (if encrypted), the "user password salt" encoded in hex, all caps
  6. (if encrypted), the "master key checksum salt" encoded in hex, all caps
  7. (if encrypted), the "number of PBKDF2 rounds used" as a decimal number
  8. (if encrypted), the "IV of the user key" encoded in hex, all caps
  9. (if encrypted), the "master IV + key blob, encrypted by the user key" encoded in hex, all caps

The actual backup data follows, either as (depending on compression and encryption) tar, deflate(tar), encrypt(tar), or encrypt(deflate(tar)).

 

TODO: write up the code path that generates the tar output -- you can simply use tar as long as entries are in the proper order (see below).

Tar archive format

App data is stored under the app/ directory, starting with a _manifest file, the APK (if requested) in a/, app files in f/, databases in db/ and shared preferences in sp/. If you requested external storage backup (using the -shared option), there will also be a shared/ directory in the archive containing external storage files.

$ tar tvf mybackup.tar
-rw------- 1000/1000      1019 2012-06-04 16:44 apps/org.myapp/_manifest
-rw-r--r-- 1000/1000   1412208 2012-06-02 23:53 apps/org.myapp/a/org.myapp-1.apk
-rw-rw---- 10091/10091     231 2012-06-02 23:41 apps/org.myapp/f/share_history.xml
-rw-rw---- 10091/10091       0 2012-06-02 23:41 apps/org.myapp/db/myapp.db-journal
-rw-rw---- 10091/10091    5120 2012-06-02 23:41 apps/org.myapp/db/myapp.db
-rw-rw---- 10091/10091    1110 2012-06-03 01:29 apps/org.myapp/sp/org.myapp_preferences.xml

Encryption details

  1. An AES 256 key is derived from the backup encryption password using 10000 rounds of PBKDF2 with a randomly generated 512 bit salt.
  2. An AES 256 master key is randomly generated
  3. A master key 'checksum' is generated by running the master key through 10000 rounds of PBKDF2 with a new randomly generated 512 bit salt.
  4. A random backup encryption IV is generated.
  5. The IV, master key, and checksum are concatenated and encrypted with the key derived in 1. The resulting blob is saved in the header as a hex string.
  6. The actual backup data is encrypted with the master key and appended to end of the file.

Sample pack/unpack code implementation (produces/uses) tar archives: https://github.com/nelenkov/android-backup-extractor

Some more details here: http://nelenkov.blogspot.com/2012/06/unpacking-android-backups.html

Perl scripts for packing/unpacking and fixing broken archives:

http://forum.xda-developers.com/showthread.php?p=27840175#post27840175

share|improve this answer
    
If you put the code somewhere, I might able to join. The OP (@ngorichter) probably has some working code by now too :) A utility that decompresses and extracts the actual files might be useful, so that you can restore only parts (if you have root of course). –  Nikolay Elenkov May 30 '12 at 1:12
1  
As for the encryption part, I have to look it up for the details, but the key is derived using PBKDF2 by the salt and device unlock PIN, password or pattern (converted to string). The master key is generated randomly, and encrypted with the password-derived key. Get it to work for unencrypted archives first. I can implement the decryption part if you are having trouble with it. –  Nikolay Elenkov May 30 '12 at 1:14
    
Sorry, the key is actually derived based on the password you specify when starting the backup. –  Nikolay Elenkov May 30 '12 at 4:16
    
@NikolayElenkov I don't have any code yet, but I do plan on writing a utility to manipulate ab files. W.r.t. encryption, I don't think it'll be difficult; it's just that I've only glanced over that part of the code. Similarly, I've traced the code path (not yet written above) that generates the tar stream, but not yet checked that the actual format is GNU tar. –  jon May 30 '12 at 10:31
    
Wow, I'm impressed with your analysis. I extracted some code from BackupManagerService into a simple groovy script, but when I run the program the result is always the same: wrong password entered! I created a new backup with a simple password, but the password verification failed again. Currently I try to follow the program as described above to find my mistake. –  ingorichter May 31 '12 at 21:28

Great and detailed answer from Nikolay Elenkov. However I should added that somebody already develop a software that do just that and package it here: http://sourceforge.net/projects/adbextractor/

The package contains both Java and Perl tool. I myself prefer Perl over Java any day, so I extracted the Perl codes, make sure they are executable, installed the required Perl library, and run the backupdecrypt.pl against an adb backup file, and it convert it into a tar or gzipped tar file without any issue.

I even formed a one liner in Bash 3 that allow me to do adb backup directly to gzipped tar file:

adb backup -f >(backupdecrypt.pl -D -z - backup.tgz) -all

Hope it helps.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.