129

I have been backing up my Nexus 7 with adb backup to back up all files into an encrypted backup. I see that you can restore from a backup with adb restore, but that would wipe all my existing data on the device.

How exactly would I extract one App's data from this encrypted backup file?

124

Just for reference of others, here is some background on the .ab file format.

The Android Backup (*.ab) file is a compressed TAR file. It is compressed using the DEFLATE algorithm. On top of that, there can be AES encryption used. This is determined when you create the backup, if you enter a password then the backup is encrypted, otherwise; there is no encryption, it is only compressed.

The HEADER of the file is a little different than a normal DEFLATE archive. It contains information about the backup and looks like the following:

ANDROID BACKUP
1
1
none

The first line is the "Magic" line. The next line is the version of the Android Backup file format. The next line is a boolean (true or false, 1 or 0) indicating if the file is compressed. The last line is the type of encryption. This example is not using any encryption. If there was a password, the line would read "AES-256". After that is the encryption cipher. If no password, then the DEFLATE "archive" starts.

It is compressed using the Java Deflater. Which, from a developers perspective, causes issues if you want to use anything besides Java to extract it. I haven't been able to find anything that can deflate it using the same algorithm, even though all that I have found (for like C#) are supposed to follow the "SPEC".

With that said, there is an open source project under the Apache 2.0 license, written by Nikolay Elenkov that will allow you to extract the .ab in to a tar file.

Usage:

java -jar abe.jar unpack <backup.ab> <backup.tar> <password>

If you are not sure how to really use that (which is beyond the scope of this answer) the next version of Droid Explorer v0.8.8.7 (available here) will allow you to do exactly this, and more, right from Explorer. You can read more about the features on my blog (yes, i know, shameless plug. I do that when it fits the question)

unpack

| improve this answer | |
  • 2
    I would vote you up if I had the reputation, my god man thanks! – user11553 Aug 30 '12 at 1:41
  • your welcome. I only know all that information because this is a feature that I just recently added to the Droid Explorer code base, so I had to do some research on it. – Ryan Conrad Aug 30 '12 at 1:44
  • Hi Ryan, tried to use Droid Explorer but it fails to start complaining about Android SDK location, even though it is installed and I'm supplying the right path. – Umar Farooq Khawaja Jan 1 '14 at 16:11
  • @UmarFarooqKhawaja Take a look at the FAQ for droid explorer. The last Q/A addresses your issue. – Ryan Conrad Jan 2 '14 at 1:21
  • OK, have extracted the relevant 'com.app.name' folder from the tar but how do I use that to restore the app data? Simply copying the folder to sdcard/Android/data/ doesn't seem to work... – pelms Apr 27 '14 at 0:07
98

Or with a one-liner:

( printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" ; tail -c +25 backup.ab ) |  tar xfvz -
| improve this answer | |
  • 4
    Nice! Do you also have the equivalent of "pack" (to take the uncompressed directory tree and create a .db file)? – dailyglen May 3 '15 at 23:46
  • 4
    Could you add an explanation to the answer.? – Lucky Jul 31 '15 at 9:23
  • 2
    THIS is the best answer :D Works perfectly fine! Extracting now. – xdevs23 Apr 1 '16 at 17:43
  • 21
    Explanation: author creates a standard Zlib file header for a .tar.gz file with printf command (0x1F 0x8B is a signature, 0x08 is the compression method, 0x00 are flags and 4 x 0x00 is timestamp), then appends to this header the contents of backup.ab file, starting from offset 25d. Such stream is a valid .tar.gz file, and the tar xfvz command recognizes it as such, so it can successfully uncompress the stream. – antonone Jun 12 '16 at 7:31
  • 4
    I made a backup with adb 1.0.36 (Revision 1.7.0.0+r33-2) and this method produced an error from zgip: invalid compressed data--format violated – Rache Nov 21 '17 at 19:42
35

One more option is to use bash, cat and gunzip (gzip).

The full process could be this (with an unencrypted backup):

  1. backup one app's data (for example "Override DNS for KitKat"):

    $ adb backup -f net.mx17.overridedns.ab -noapk net.mx17.overridedns
    Now unlock your device and confirm the backup operation.
    
  2. extract the compressed data

    $ dd if=net.mx17.overridedns.ab bs=1 skip=24 > compressed-data
    1285+0 records in
    1285+0 records out
    1285 bytes (1,3 kB) copied, 0,00745877 s, 172 kB/s
    
  3. decompress the compressed data

    $ printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" \
        | cat - compressed-data | gunzip -c > decompressed-data.tar
    gzip: stdin: unexpected end of file
    
  4. "untar" the tar file

    $ tar xf decompressed-data.tar
    
| improve this answer | |
  • 5
    Is the "unexpected end of file" to be expected? – hft Sep 2 '16 at 23:57
  • The extraction in 2 surprisingly worked on an encrypted file. How is this possible? The decompression then failed with gzip: stdin: invalid compressed data--format violated. I assume the extraction succeeded, as dd reports 22763821+0 records in 22763821+0 records out. – Tom Russell Jul 31 '18 at 2:28
  • @TomRussell The extraction merely trims bytes off the start. – Solomon Ucko Apr 25 at 22:33
5

As the implicit question is also, how to restore a single apps data, i would like to mention this nifty script, that splits a given full-backup.ab in single-app.ab files:

https://sourceforge.net/projects/adb-split/

It requires the jar files: abe.jar and tar-bin-split.jar that can be found here:

It worked at least for my testcase.

| improve this answer | |
4

Another option is using the Perl AdbBackupRoutines from this XDA thread. They have a few requirements, though: Perl obviously, plus libterm-readkey-perl, libcrypt-cbc-perl, and libcrypt-pbkdf2-perl (if your backups are unencrypted, you can skip the last dependency by simply commenting out line 103 of backupdecrypt.pl where it's included -- worked fine for me).

Usage is quite easy:

./backupdecrypt.pl [options] <backupfile.ab> <outfile.tar>

The resulting .tar file then can be investigated like any other tarball. Its structure is quite interesting in at least one aspect: it does not reflect the real paths where the files have been taken from (e.g. not /data/data/com.app.name/databases/whatever.db, but instead apps/com.app.name/db/whatever.db) -- which indicates an app backed-up on one device/ROM might be restored to any other device/ROM without trouble, as adb restore must figure out the real paths itself.

| improve this answer | |
  • There's another tool (a little better maintained) that does this: github.com/nelenkov/android-backup-extractor – lid May 23 '15 at 15:22
  • Thanks, @lid! I've also included a little shell-script (almost a one-liner only) with my tool Adebar (Android DEvice Backup And Report), which converts ADB backups (.ab) to .tar.gz Archives :) – Izzy May 23 '15 at 15:27
4

Based on the information by others, now I know that the backup file is just a prefixed Deflated (GZip) stream, based on this information this simple program can unpack it for you:

import java.io.*;
import java.util.zip.*;

/** Run: javac unab.java && java unab backupfile.ab */
public class unab {
    private static final int BACKUP_HEADER_LENGTH = 24;
    public static void main(String[] args) throws IOException {
        InputStream in = new FileInputStream(args[0]);
        try {
            OutputStream out = new FileOutputStream(args[0] + ".tar");
            try {
                if (in.skip(BACKUP_HEADER_LENGTH) != BACKUP_HEADER_LENGTH) {
                    throw new IOException("Unexpected end of file while skipping backup header.");
                }
                byte[] buffer = new byte[100 * 1024];
                int count;
                InputStream zip = new InflaterInputStream(in);
                while ((count = zip.read(buffer)) > 0) {
                    out.write(buffer, 0, count);
                }
            } finally {
                out.close();
            }
        } finally {
            in.close();
        }
    }
}

I wrote this because I don't have any of the Unix tools mentioned above, and it was easier than installing Cygwin or other tools.

Advantages:

  • cross-platform
  • simple (no esoteric parameters)
  • no need for piping tools

Disadvantages:

  • need a JDK (which you likely already have because you're messing with Android SDK)
  • no support for encrypted backups
  • need something to extract the resulting tar file (I use Total Commander)

To make it a command line tool create unab.bat with contents: java -cp "%~dp0." unab %* and the directory to PATH.

| improve this answer | |
1

Just to add to other answers, I finally got it using nelenkov's app (https://github.com/nelenkov/android-backup-extractor) and instructions here: https://forum.xda-developers.com/showthread.php?t=2011811. I was trying to avoid the extra 800mb disk space for Java. Most phones are moving toward encrypted by default, and I think nelenkov's app is the only solution that works with encrypted.

Here is a direct link to nelenkov's compiled jar release. Check if its the latest: https://github.com/nelenkov/android-backup-extractor/releases/tag/20181012025725-d750899

Then follow these instructions from the link above. It has more info about Helium and Splitting an ab file up in to individual app files. And also a way to change apps not set to use adb backup without root. Substitute abe-all.jar if using this release file.

1) Convert the original adb backup to tar format:

java -jar abe.jar unpack nexus7.ab nexus7.tar <password>

2) Extract the contents of the tar archive. This should be done on a filesystem where the permissions of the files inside the tar are preserved, for example using linux, mac or bsd. Up to two folders may appear, apps and shared:

tar -xvf nexus7.tar

Now you should have new folders to look in. Now you can see which apps are backed up and which arent.

BTW, Izzy's answer is good (https://android.stackexchange.com/a/42224/95893), but his script requires several perl modules to be installed, the 3rd I couldnt find easily. His app is full featured, but find adb2tar in Adebar-master/tools/. Again, it doesnt work with encrypted ab files though.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy