How do you extract an App's data from a full backup made through "adb backup"?

Question

I have been backing up my Nexus 7 with adb backup to back up all files into an encrypted backup. I see that you can restore from a backup with adb restore, but that would wipe all my existing data on the device.

How exactly would I extract one App's data from this encrypted backup file?

Answer 1

How to extract ab files

There is an open source project under the Apache 2.0 license, written by Nikolay Elenkov that will allow you to extract the .ab in to a tar file.

Usage:

java -jar abe.jar unpack <backup.ab> <backup.tar> <password>

Background

Just for reference of others, here is some background on the .ab file format.

The Android Backup (*.ab) file is a compressed TAR file. It is compressed using the DEFLATE algorithm. On top of that, there can be AES encryption used. This is determined when you create the backup, if you enter a password then the backup is encrypted, otherwise; there is no encryption, it is only compressed.

The HEADER of the file is a little different than a normal DEFLATE archive. It contains information about the backup and looks like the following:

ANDROID BACKUP
1
1
none

The first line is the "Magic" line. The next line is the version of the Android Backup file format. The next line is a boolean (true or false, 1 or 0) indicating if the file is compressed. The last line is the type of encryption. This example is not using any encryption. If there was a password, the line would read "AES-256". After that is the encryption cipher. If no password, then the DEFLATE "archive" starts.

It is compressed using the Java Deflater. Which, from a developers perspective, causes issues if you want to use anything besides Java to extract it. I haven't been able to find anything that can deflate it using the same algorithm, even though all that I have found (for like C#) are supposed to follow the "SPEC".

If you are not sure how to really use that (which is beyond the scope of this answer) Droid Explorer since v0.8.8.7 (available here) allows you to do exactly this, and more, right from Explorer. You can read more about the features on my blog (yes, i know, shameless plug. I do that when it fits the question)

Answer 2

Or with a one-liner:

( printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" ; tail -c +25 backup.ab ) |  tar xfvz -

Answer 3

One more option is to use bash, cat and gunzip (gzip).

The full process could be this (with an unencrypted backup):

1. backup one app's data (for example "Override DNS for KitKat"):

   $ adb backup -f net.mx17.overridedns.ab -noapk net.mx17.overridedns
   Now unlock your device and confirm the backup operation.
   

2. extract the compressed data

   $ dd if=net.mx17.overridedns.ab bs=1 skip=24 > compressed-data
   1285+0 records in
   1285+0 records out
   1285 bytes (1,3 kB) copied, 0,00745877 s, 172 kB/s
   

3. decompress the compressed data

   $ printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" \
       | cat - compressed-data | gunzip -c > decompressed-data.tar
   gzip: stdin: unexpected end of file
   

4. "untar" the tar file

   $ tar xf decompressed-data.tar
   

Answer 4

Based on the information by others, now I know that the backup file is just a prefixed Deflated (GZip) stream, based on this information this simple program can unpack it for you:

import java.io.*;
import java.util.zip.*;

/** Run: javac unab.java && java unab backupfile.ab */
public class unab {
    private static final int BACKUP_HEADER_LENGTH = 24;
    public static void main(String[] args) throws IOException {
        InputStream in = new FileInputStream(args[0]);
        try {
            OutputStream out = new FileOutputStream(args[0] + ".tar");
            try {
                if (in.skip(BACKUP_HEADER_LENGTH) != BACKUP_HEADER_LENGTH) {
                    throw new IOException("Unexpected end of file while skipping backup header.");
                }
                byte[] buffer = new byte[100 * 1024];
                int count;
                InputStream zip = new InflaterInputStream(in);
                while ((count = zip.read(buffer)) > 0) {
                    out.write(buffer, 0, count);
                }
            } finally {
                out.close();
            }
        } finally {
            in.close();
        }
    }
}

I wrote this because I don't have any of the Unix tools mentioned above, and it was easier than installing Cygwin or other tools.

Advantages:

• cross-platform
• simple (no esoteric parameters)
• no need for piping tools

Disadvantages:

• need a JDK (which you likely already have because you're messing with Android SDK)
• no support for encrypted backups
• need something to extract the resulting tar file (I use Total Commander)

To make it a command line tool create unab.bat with contents: java -cp "%~dp0." unab %* and the directory to PATH.

Answer 5

As the implicit question is also, "how to restore a single apps data", I would like to mention this nifty script, that splits a given full-backup.ab in single-app.ab files.

It requires these jar files: abe.jar and tar-bin-split.jar.

At least for my testcase, it worked using the referenced resources.

Answer 6

Another option is using the Perl AdbBackupRoutines from this XDA thread. They have a few requirements, though: Perl obviously, plus libterm-readkey-perl, libcrypt-cbc-perl, and libcrypt-pbkdf2-perl (if your backups are unencrypted, you can skip the last dependency by simply commenting out line 103 of backupdecrypt.pl where it's included -- worked fine for me).

Usage is quite easy:

./backupdecrypt.pl [options] <backupfile.ab> <outfile.tar>

The resulting .tar file then can be investigated like any other tarball. Its structure is quite interesting in at least one aspect: it does not reflect the real paths where the files have been taken from (e.g. not /data/data/com.app.name/databases/whatever.db, but instead apps/com.app.name/db/whatever.db) -- which indicates an app backed-up on one device/ROM might be restored to any other device/ROM without trouble, as adb restore must figure out the real paths itself.

Answer 7

To add another option here's a Windows PowerShell version of the header manipulation:

$file_in = (Get-Location).Path + "\backup.ab"
$file_temp = (Get-Location).Path + "\temp.tgz"
[int32[]]$header = 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00
$bytes = [System.IO.File]::ReadAllBytes($file_in)
$bytesout = $header + $bytes[24..($bytes.Length - 1)]
[System.IO.File]::WriteAllBytes($file_temp, $bytesout)

You can try to use the tar executable in current Windows 10 versions 17063 or later to extract the file, however it seems that it currently doesn't handle the misformed file produced in this process as well as the linux version. YMMV:

tar -xvzf $file_temp

To get the full contents you can manually extract with 7zip or similar. Or could combine with one of the answers from Native .tar extraction in Powershell.

Answer 8

Just to add to other answers, I finally got it using nelenkov's app and the instructions from here. I was trying to avoid the extra 800mb disk space for Java. Most phones are moving toward encrypted by default, and I think nelenkov's app is the only solution that works well with encrypted.

Here is a direct link to nelenkov's compiled jar release. Please check if its the latest, then follow the instructions from the link above. It has more info about Helium and Splitting an ab file up in to individual app files. And also a way to change apps not set to use adb backup without root. Substitute abe-all.jar if using this release file.

1. Convert the original adb backup to tar format:

java -jar abe.jar unpack nexus7.ab nexus7.tar <password>

2. Extract the contents of the tar archive. This should be done on a filesystem where the permissions of the files inside the tar are preserved, for example using linux, mac or bsd. Up to two folders may appear, apps and shared:

tar -xvf nexus7.tar

Now you should have new folders to look in. Now you can see which apps are backed up and which arent.

BTW, Izzy's answer is good, but his script requires several perl modules to be installed, the 3rd of which I couldnt find easily. His app is full featured, but find adb2tar in Adebar-master/tools/. Again, it doesnt work with encrypted ab files though.

Answer 9

This worked for me in Linux environment:

dd if=backup.adb skip=1 bs=24 | zlib-flate -uncompress | cat > somewhere.tar
tar xvf somewhere.tar

Answer 10

You can download the latest binary of the Android Backup Processor. Once downloaded, extract the zip file and run:

java -jar abp.jar unpack <backup.ab> <backup.tar> <password>

Commands

Usage:
    info:       java -jar abp.jar [-debug] [-useenv=yourenv] info <backup.ab> [password]
    unpack:     java -jar abp.jar [-debug] [-useenv=yourenv] unpack <backup.ab> <backup.tar> [password]
    pack:       java -jar abp.jar [-debug] [-useenv=yourenv] pack <backup.tar> <backup.ab> [password]
    pack-kk:    java -jar abp.jar [-debug] [-useenv=yourenv] pack-kk <backup.tar> <backup.ab> [password]