Cannot restore an adb backup due to a tarball with a broken header

Question

I have two devices that have Android 5.0.2 and 6.0.1 respectively. I would like to migrate some non-cloud apps from the device with the older Android version to the new one using adb. The backing up command is simple:

adb backup -apk -f foo.bar.baz.ab foo.bar.baz

Now, after connecting the device with the newer Android version, the following command is expected to work:

adb restore foo.bar.baz.ab

Unfortunately it isn't. As usual, the restoring process fails silently just reporting that the restoring process is ended. Actually nothing really happens. Here are the logs:

adb logcat -s BackupManagerService

08-27 01:02:44.100 1300 2300 I BackupManagerService: Beginning full restore...
08-27 01:02:44.100 1300 2300 D BackupManagerService: Starting restore confirmation UI, token=1956031088
08-27 01:02:44.120 1300 2300 D BackupManagerService: Waiting for full restore completion...
08-27 01:02:45.650 1300 2450 D BackupManagerService: acknowledgeFullBackupOrRestore : token=1956031088 allow=true
08-27 01:02:45.660 1300 18181 I BackupManagerService: --- Performing full-dataset restore ---
08-27 01:02:45.680 1300 18181 I BackupManagerService: Package foo.bar.baz not installed; requiring apk in dataset
08-27 01:02:45.680 1300 18181 D BackupManagerService: APK file; installing
08-27 01:02:45.680 1300 18181 D BackupManagerService: Installing from backup: foo.bar.baz
08-27 01:02:45.690 1300 18181 E BackupManagerService: Unable to transcribe restored apk for install
08-27 01:02:45.690 1300 18181 E BackupManagerService: Parse error in header: Invalid number in header: 'я╛В' for radix 8
08-27 01:02:45.710 1300 18181 W BackupManagerService: io exception on restore socket read
08-27 01:02:45.710 1300 18181 W BackupManagerService: java.io.IOException: Invalid number in header: 'я╛В' for radix 8
08-27 01:02:45.710 1300 18181 W BackupManagerService: at com.android.server.backup.BackupManagerService$PerformAdbRestoreTask.extractRadix(BackupManagerService.java:7380)
08-27 01:02:45.710 1300 18181 W BackupManagerService: at com.android.server.backup.BackupManagerService$PerformAdbRestoreTask.readTarHeaders(BackupManagerService.java:7179)
08-27 01:02:45.710 1300 18181 W BackupManagerService: at com.android.server.backup.BackupManagerService$PerformAdbRestoreTask.restoreOneFile(BackupManagerService.java:6396)
08-27 01:02:45.710 1300 18181 W BackupManagerService: at com.android.server.backup.BackupManagerService$PerformAdbRestoreTask.run(BackupManagerService.java:6254)
08-27 01:02:45.710 1300 18181 W BackupManagerService: at java.lang.Thread.run(Thread.java:818)
08-27 01:02:45.710 1300 2300 I BackupManagerService: Full restore processing complete.
08-27 01:02:45.720 1300 18181 D BackupManagerService: Full restore pass complete.

It looks somewhat strange that the same backup format cannot be restored on a newer device. I've also tried to repack the underlying tarball using nelenkov/android-backup-extractor preserving the same file order (tar cvf ... -T files.lst) hoping that the broken header gets repaired. No luck.

What does cause such a backup/restore behavior and how do I resolve the issue to migrate the application from the older device to the newer one? Any help is highly appreciated and thanks in advance.

• Android Debug Bridge version 1.0.31
• LG Optimus E975, Android 5.0.2, CyanogenMod 12-2015...
• Samsung Galaxy S5, Android 6.0.2, flashed to a newer Sprint stock firmware (initially Android 5.0.x earlier)

Answer 1

After a whole day of various experiments with another tools like TitaniumBackup (neither worked nice, TB was only able to restore APKs, not data) I've figured out that specifying the -apk key was the problem. Simply omitting the flag during a backup operation makes it fully restorable.

So my new backup/restore plan became as follows:

• device 1: backup data only
• device 2: backup APK+data retaining the APK only later (requires nelenkov/android-backup-exractor by Nikolay Elenkov; can be easily built using JDK 1.8.x and Gradle 2.7 (couldn't build it using Gradle 2.7+ and Gradle 3.0+) with simple gradle command)
• device 2: restore the APK using adb install
• device 2: restore the data using adb restore

Here are the scripts:

backup.sh

#!/bin/bash

set -e

PACKAGE=$1

if [ -z "$PACKAGE" ]; then
    echo no package specified
    exit
fi

echo ">>> Backing up $PACKAGE data..."
adb backup -f "$PACKAGE".DATA.ab $PACKAGE

echo ">>> Backing up $PACKAGE APK..."
adb backup -apk -f "$PACKAGE".APK.ab $PACKAGE

echo ">>> Extracting APK..."
java -jar abe-all.jar unpack "$PACKAGE".APK.ab "$PACKAGE".APK.ab.tar
rm "$PACKAGE".APK.ab
APK_FILENAME=$(basename $(tar tf "$PACKAGE".APK.ab.tar | grep '^apps/'"$PACKAGE"'/a/'))
tar xvf "$PACKAGE".APK.ab.tar -C ./ "apps/$PACKAGE/a/$APK_FILENAME" --strip-components=3
mv "$APK_FILENAME" "$PACKAGE".apk
rm "$PACKAGE".APK.ab.tar

echo
echo "*************"
echo "*** Done ***"
echo "*************"

This script requires a package name as the input parameter and stores an APK and its data as foo.bar.apk and foo.bar.DATA.ab respectively. I also implemented it as a twice-backup script (a device asks for making a backup two times) because I didn't want to re-pack underlying tarballs preserving the original file order just making sure that the DATA.ab file is in its original state.

restore.sh

#!/bin/bash

set -e

PACKAGE=$1

if [ -z "$PACKAGE" ]; then
    echo no package specified
    exit
fi

echo ">>> Installing $PACKAGE APK..."
adb install -r $PACKAGE.apk

echo ">>> Restoring $PACKAGE data..."
adb restore $PACKAGE.DATA.ab

echo
echo "*************"
echo "*** Done ***"
echo "*************"

This script simply installs an APK on a target device and restores its data. The only input parameter is the application package name.

So the overall backup/restore process is:

• Connect device #1
• Execute backup.sh passing an application package name
• Click "Backup..." on the device #1 (one will be asked two times for the data and APK+data backups respectively)
• Repeat the previous two steps for other packages if necessary
• Disconnect the device #1 and connect the device #2
• Execute restore.sh passing an application package name
• Click "Restore..." on the device #2 once asked
• Repeat the previous two steps for other packages if necessary

Answer 2

I just want to add to your answer, for people going from KitKat (4.4) to Nougat (7.1). (BTW, I finally came across your question/answer after 2 days of fighting with this, thanks a ton)

There are various bugs in adb from version 1.0.32+ when trying to backup, but version 1.0.31 works great. However, you seem to need 1.0.36 to restore to Nougat.

To Backup: platform-tools_r20 (You MUST set a password)

To Restore: platform-tools_r25.0.3 (No password needed)

As such, I added to your script a bit... (you'll need to add symlinks for adb-1.0.31 and adb-1.0.36)

backup.sh

#!/bin/bash

set -e

DEVICE=$1
PACKAGE=$2
PASSWD=$3

if [ -z "$PACKAGE" ]; then
    echo no package specified
    exit
fi

echo ">>> Backing up $PACKAGE data..."
adb-1.0.31 -s $DEVICE backup -f "$PACKAGE".DATA.ab $PACKAGE

sleep 5

echo ">>> Backing up $PACKAGE APK..."
adb-1.0.31 -s $DEVICE backup -apk -f "$PACKAGE".APK.ab $PACKAGE

echo ">>> Extracting APK..."
java -jar abe.jar unpack "$PACKAGE".APK.ab "$PACKAGE".APK.ab.tar $PASSWD
rm "$PACKAGE".APK.ab
APK_FILENAME=$(basename $(tar tf "$PACKAGE".APK.ab.tar | grep '^apps/'"$PACKAGE"'/a/'))
tar xvf "$PACKAGE".APK.ab.tar -C ./ "apps/$PACKAGE/a/$APK_FILENAME" --strip-components=3
mv "$APK_FILENAME" "$PACKAGE".apk
rm "$PACKAGE".APK.ab.tar

echo
echo "*************"
echo "*** Done ***"
echo "*************"

restore.sh

#!/bin/bash

set -e

DEVICE=$1
PACKAGE=$2

if [ -z "$PACKAGE" ]; then
    echo no package specified
    exit
fi

echo ">>> Installing $PACKAGE APK..."
adb-1.0.36 -s $DEVICE install -r $PACKAGE.apk

echo ">>> Restoring $PACKAGE data..."
adb-1.0.36 -s $DEVICE restore $PACKAGE.DATA.ab

echo
echo "*************"
echo "*** Done ***"
echo "*************"

Answer 3

I got the same error when to .ab file I packed .7z archive instead of .tar. I used 7z.exe, so changing command from:

7z.exe a -t7z app.package.name.7z app.package.name

to:

7z.exe a -ttar app.package.name.tar app.package.name

helped. Regards.

Answer 4

I know it's not a direct answer to your question, but other people may be in the same situation and looking for a similar solution:

This bit me today as well during a Lineage 14.1->15.1 upgrade. ADB 1.0.39 does not really seem to improve the situation.

I did not want to use old adb versions, but wanted it to be simple and open source.

I found oandbackup (https://f-droid.org/en/packages/dk.jens.backup/) to do exactly what I want. Just select all user apps, backup apk+data, copy the whole folder to the other phone, restore all apps+apks, and voila, (nearly) everything back as it should be. Account entries (imapnotes, davdroid) have not worked, and backing up the "account" app added an invisible imapnotes account which did not work and could not be re-added. Had to delete app data and re-add the account manually. So I would not recommend backing up account info.

Other than that, I am happy.

I used to use adebar (https://github.com/IzzySoft/Adebar) which was perfect, but now that adb backup/restore do not work reliable, it also does not work.

Every adb version creates different error messages:

DefContainer: Failed to parse package or

Unable to transcribe restored apk for install

Hopefully some search engine will pick these strings up.