138

I have been backing up my Nexus 7 with adb backup to back up all files into an encrypted backup. I see that you can restore from a backup with adb restore, but that would wipe all my existing data on the device.

How exactly would I extract one App's data from this encrypted backup file?

3
130

How to extract ab files

There is an open source project under the Apache 2.0 license, written by Nikolay Elenkov that will allow you to extract the .ab in to a tar file.

Usage:

java -jar abe.jar unpack <backup.ab> <backup.tar> <password>

Background

Just for reference of others, here is some background on the .ab file format.

The Android Backup (*.ab) file is a compressed TAR file. It is compressed using the DEFLATE algorithm. On top of that, there can be AES encryption used. This is determined when you create the backup, if you enter a password then the backup is encrypted, otherwise; there is no encryption, it is only compressed.

The HEADER of the file is a little different than a normal DEFLATE archive. It contains information about the backup and looks like the following:

ANDROID BACKUP
1
1
none

The first line is the "Magic" line. The next line is the version of the Android Backup file format. The next line is a boolean (true or false, 1 or 0) indicating if the file is compressed. The last line is the type of encryption. This example is not using any encryption. If there was a password, the line would read "AES-256". After that is the encryption cipher. If no password, then the DEFLATE "archive" starts.

It is compressed using the Java Deflater. Which, from a developers perspective, causes issues if you want to use anything besides Java to extract it. I haven't been able to find anything that can deflate it using the same algorithm, even though all that I have found (for like C#) are supposed to follow the "SPEC".

If you are not sure how to really use that (which is beyond the scope of this answer) Droid Explorer since v0.8.8.7 (available here) allows you to do exactly this, and more, right from Explorer. You can read more about the features on my blog (yes, i know, shameless plug. I do that when it fits the question)

unpack

7
  • 2
    I would vote you up if I had the reputation, my god man thanks! – user11553 Aug 30 '12 at 1:41
  • your welcome. I only know all that information because this is a feature that I just recently added to the Droid Explorer code base, so I had to do some research on it. – Ryan Conrad Aug 30 '12 at 1:44
  • Hi Ryan, tried to use Droid Explorer but it fails to start complaining about Android SDK location, even though it is installed and I'm supplying the right path. – Umar Farooq Khawaja Jan 1 '14 at 16:11
  • @UmarFarooqKhawaja Take a look at the FAQ for droid explorer. The last Q/A addresses your issue. – Ryan Conrad Jan 2 '14 at 1:21
  • OK, have extracted the relevant 'com.app.name' folder from the tar but how do I use that to restore the app data? Simply copying the folder to sdcard/Android/data/ doesn't seem to work... – pelms Apr 27 '14 at 0:07
108

Or with a one-liner:

( printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" ; tail -c +25 backup.ab ) |  tar xfvz -
13
  • 5
    Nice! Do you also have the equivalent of "pack" (to take the uncompressed directory tree and create a .db file)? – dailyglen May 3 '15 at 23:46
  • 4
    Could you add an explanation to the answer.? – Lucky Jul 31 '15 at 9:23
  • 2
    THIS is the best answer :D Works perfectly fine! Extracting now. – xdevs23 Apr 1 '16 at 17:43
  • 26
    Explanation: author creates a standard Zlib file header for a .tar.gz file with printf command (0x1F 0x8B is a signature, 0x08 is the compression method, 0x00 are flags and 4 x 0x00 is timestamp), then appends to this header the contents of backup.ab file, starting from offset 25d. Such stream is a valid .tar.gz file, and the tar xfvz command recognizes it as such, so it can successfully uncompress the stream. – antonone Jun 12 '16 at 7:31
  • 4
    I made a backup with adb 1.0.36 (Revision 1.7.0.0+r33-2) and this method produced an error from zgip: invalid compressed data--format violated – Rache Nov 21 '17 at 19:42
35

One more option is to use bash, cat and gunzip (gzip).

The full process could be this (with an unencrypted backup):

  1. backup one app's data (for example "Override DNS for KitKat"):

    $ adb backup -f net.mx17.overridedns.ab -noapk net.mx17.overridedns
    Now unlock your device and confirm the backup operation.
    
  2. extract the compressed data

    $ dd if=net.mx17.overridedns.ab bs=1 skip=24 > compressed-data
    1285+0 records in
    1285+0 records out
    1285 bytes (1,3 kB) copied, 0,00745877 s, 172 kB/s
    
  3. decompress the compressed data

    $ printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" \
        | cat - compressed-data | gunzip -c > decompressed-data.tar
    gzip: stdin: unexpected end of file
    
  4. "untar" the tar file

    $ tar xf decompressed-data.tar
    
4
  • 5
    Is the "unexpected end of file" to be expected? – hft Sep 2 '16 at 23:57
  • The extraction in 2 surprisingly worked on an encrypted file. How is this possible? The decompression then failed with gzip: stdin: invalid compressed data--format violated. I assume the extraction succeeded, as dd reports 22763821+0 records in 22763821+0 records out. – Tom Russell Jul 31 '18 at 2:28
  • @TomRussell The extraction merely trims bytes off the start. – Solomon Ucko Apr 25 '20 at 22:33
  • Isn't dd a bit of an overkill to just skip 24 bytes? This is also not failsafe unless conv=noerror,sync iflag=fullblock is added: superuser.com/a/1523120/910769 You might consider this answer: android.stackexchange.com/a/78183/340401 – Cadoiz Mar 8 at 1:08
8

Based on the information by others, now I know that the backup file is just a prefixed Deflated (GZip) stream, based on this information this simple program can unpack it for you:

import java.io.*;
import java.util.zip.*;

/** Run: javac unab.java && java unab backupfile.ab */
public class unab {
    private static final int BACKUP_HEADER_LENGTH = 24;
    public static void main(String[] args) throws IOException {
        InputStream in = new FileInputStream(args[0]);
        try {
            OutputStream out = new FileOutputStream(args[0] + ".tar");
            try {
                if (in.skip(BACKUP_HEADER_LENGTH) != BACKUP_HEADER_LENGTH) {
                    throw new IOException("Unexpected end of file while skipping backup header.");
                }
                byte[] buffer = new byte[100 * 1024];
                int count;
                InputStream zip = new InflaterInputStream(in);
                while ((count = zip.read(buffer)) > 0) {
                    out.write(buffer, 0, count);
                }
            } finally {
                out.close();
            }
        } finally {
            in.close();
        }
    }
}

I wrote this because I don't have any of the Unix tools mentioned above, and it was easier than installing Cygwin or other tools.

Advantages:

  • cross-platform
  • simple (no esoteric parameters)
  • no need for piping tools

Disadvantages:

  • need a JDK (which you likely already have because you're messing with Android SDK)
  • no support for encrypted backups
  • need something to extract the resulting tar file (I use Total Commander)

To make it a command line tool create unab.bat with contents: java -cp "%~dp0." unab %* and the directory to PATH.

6

As the implicit question is also, "how to restore a single apps data", I would like to mention this nifty script, that splits a given full-backup.ab in single-app.ab files.

It requires these jar files: abe.jar and tar-bin-split.jar.

At least for my testcase, it worked using the referenced resources.

4

Another option is using the Perl AdbBackupRoutines from this XDA thread. They have a few requirements, though: Perl obviously, plus libterm-readkey-perl, libcrypt-cbc-perl, and libcrypt-pbkdf2-perl (if your backups are unencrypted, you can skip the last dependency by simply commenting out line 103 of backupdecrypt.pl where it's included -- worked fine for me).

Usage is quite easy:

./backupdecrypt.pl [options] <backupfile.ab> <outfile.tar>

The resulting .tar file then can be investigated like any other tarball. Its structure is quite interesting in at least one aspect: it does not reflect the real paths where the files have been taken from (e.g. not /data/data/com.app.name/databases/whatever.db, but instead apps/com.app.name/db/whatever.db) -- which indicates an app backed-up on one device/ROM might be restored to any other device/ROM without trouble, as adb restore must figure out the real paths itself.

2
  • There's another tool (a little better maintained) that does this: github.com/nelenkov/android-backup-extractor – lid May 23 '15 at 15:22
  • Thanks, @lid! I've also included a little shell-script (almost a one-liner only) with my tool Adebar (Android DEvice Backup And Report), which converts ADB backups (.ab) to .tar.gz Archives :) – Izzy May 23 '15 at 15:27
2

To add another option here's a Windows PowerShell version of the header manipulation:

$file_in = (Get-Location).Path + "\backup.ab"
$file_temp = (Get-Location).Path + "\temp.tgz"
[int32[]]$header = 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00
$bytes = [System.IO.File]::ReadAllBytes($file_in)
$bytesout = $header + $bytes[24..($bytes.Length - 1)]
[System.IO.File]::WriteAllBytes($file_temp, $bytesout)

You can try to use the tar executable in current Windows 10 versions 17063 or later to extract the file, however it seems that it currently doesn't handle the misformed file produced in this process as well as the linux version. YMMV:

tar -xvzf $file_temp

To get the full contents you can manually extract with 7zip or similar. Or could combine with one of the answers from Native .tar extraction in Powershell.

2
  • 7zip and also other archive tools feature command-line-tools – Cadoiz Mar 8 at 1:19
  • I like this technique, but I get an OOM (Out Of Memory) error when I try using it. It's the next-to-last line in the PS script that seems to eat up all available RAM. – RockPaperLz- Mask it or Casket Mar 9 at 8:31
1

Just to add to other answers, I finally got it using nelenkov's app and the instructions from here. I was trying to avoid the extra 800mb disk space for Java. Most phones are moving toward encrypted by default, and I think nelenkov's app is the only solution that works well with encrypted.

Here is a direct link to nelenkov's compiled jar release. Please check if its the latest, then follow the instructions from the link above. It has more info about Helium and Splitting an ab file up in to individual app files. And also a way to change apps not set to use adb backup without root. Substitute abe-all.jar if using this release file.

  1. Convert the original adb backup to tar format:

java -jar abe.jar unpack nexus7.ab nexus7.tar <password>

  1. Extract the contents of the tar archive. This should be done on a filesystem where the permissions of the files inside the tar are preserved, for example using linux, mac or bsd. Up to two folders may appear, apps and shared:

tar -xvf nexus7.tar

Now you should have new folders to look in. Now you can see which apps are backed up and which arent.

BTW, Izzy's answer is good, but his script requires several perl modules to be installed, the 3rd of which I couldnt find easily. His app is full featured, but find adb2tar in Adebar-master/tools/. Again, it doesnt work with encrypted ab files though.

1
  • I wasn't aware ab2tar requires any Perl modules. What it basically does is what rmil's answer describes ;) But maybe it did in an older version… – Izzy Mar 8 at 12:29
0

This worked for me in Linux environment:

dd if=backup.adb skip=1 bs=24 | zlib-flate -uncompress | cat > somewhere.tar
tar xvf somewhere.tar
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy