3

I am trying to do a full copy of /sdcard of my unrooted OnePlus 3T to my Windows PC.

So far I've tried:

  • adb pull /sdcard/ C:\mylocalfolder\

    Works OK but as soon as there is an error it stops the process, so the copy process has been truncated. The error is due to a file containing ":" in the name, so it can't be copied to Windows.


  • adb exec-out tar c sdcard > sdcard_backup.tar

    Doesn't work, it stops inmediatly and creates a corrupted 4kb file.


  • Copy the files using Windows via Data Transfer mode

    Windows halts calculating time to copy files.


Adb pull would be the ideal solution so far if is there a way to skip/ignore errors so it can continue copying the files, or perhaps try to skip the files with certain characters.

I've tryed both Adb version 1.0.32 and 1.0.41 (latest version as of July 2020).

Any suggestion?

8
  • 1
    Keep a small Linux distro installed on PC as secondary OS (even on a USB). Make life easy. // tar is the best way to handle cross platform file name issues. It should work. Try adb shell tar c sdcard > sdcard_backup.tar. Also unlike exec-out this won't merge STDERR with SRDOUT. – Irfan Latif Jul 7 '20 at 18:59
  • 1
    try adebar or adb-sync – alecxs Jul 7 '20 at 19:58
  • @alecxs is there any way to tell how much progress this command made? It seems to me like the target tar file size is not updating – lucidbrot Feb 12 at 21:05
  • @lucidbrot maybe via logcat but i don't know. consider adb is broken, i can't for the sake pull files from samsung usb-c. small files are fine, but after ~ 1 min adb crashes. got the feeling it is related to usb speed – alecxs Feb 12 at 21:38
  • @alecxs I have let the exec-out run overnight and it managed to grab nothing for about two hours, then the tar was suddenly 8.3GB, and now the rest of the night it doesn't seem like anything new was added... it still does not think it's done though. I have aborted it now, but the tar is not even a valid tar file. – lucidbrot Feb 13 at 9:01
2

Quoting is recommended to make it more clear which commands will executed on client and which pipes/redirections are designated to run on host side. Basically this command should work:

adb exec-out "tar -c /sdcard" > sdcard_backup.tar

Have a look into corrupted 4 kb file. you will see the tar header and file content. there is a leading string before the tar file begin:

removing leading '/' from member names

$ hexdump -C sdcard_backup.tar
00000000  72 65 6d 6f 76 69 6e 67  20 6c 65 61 64 69 6e 67  |removing leading|
00000010  20 27 2f 27 20 66 72 6f  6d 20 6d 65 6d 62 65 72  | '/' from member|
00000020  20 6e 61 6d 65 73 0a 73  64 63 61 72 64 00 00 00  | names.sdcard...|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000080  00 00 00 00 00 00 00 00  00 00 00 30 30 30 30 37  |...........00007|
00000090  37 37 00 30 30 30 30 30  30 30 00 30 30 30 30 30  |77.0000000.00000|
000000a0  30 30 00 30 30 30 30 30  30 30 30 30 30 30 00 30  |00.00000000000.0|
000000b0  30 30 30 30 30 30 30 30  30 30 00 30 31 34 37 35  |0000000000.01475|
000000c0  37 00 20 32 2f 73 74 6f  72 61 67 65 2f 73 65 6c  |7. 2/storage/sel|
000000d0  66 2f 70 72 69 6d 61 72  79 00 00 00 00 00 00 00  |f/primary.......|

problematic exec-out will redirect stdin and stderr. at least avoid error messages, or even better catch stderr completely

adb exec-out "tar -c sdcard" > sdcard_backup.tar
adb exec-out "tar -c sdcard 2> /dev/null" > sdcard_backup.tar

Note the skipped leading / root dir in command line. This time a 4 kb tar file is produced

$ hexdump -C sdcard_backup.tar
00000000  73 64 63 61 72 64 00 00  00 00 00 00 00 00 00 00  |sdcard..........|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000060  00 00 00 00 30 30 30 30  37 37 37 00 30 30 30 30  |....0000777.0000|
00000070  30 30 30 00 30 30 30 30  30 30 30 00 30 30 30 30  |000.0000000.0000|
00000080  30 30 30 30 30 30 30 00  30 30 30 30 30 30 30 30  |0000000.00000000|
00000090  30 30 30 00 30 31 34 37  35 37 00 20 32 2f 73 74  |000.014757. 2/st|
000000a0  6f 72 61 67 65 2f 73 65  6c 66 2f 70 72 69 6d 61  |orage/self/prima|
000000b0  72 79 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |ry..............|

looks like a healthy tar archive. let tar do it's work. it reveals the file is a symbolic link

$ tar -vtf sdcard_backup.tar
lrwxrwxrwx root/root         0 1970-01-01 01:00 sdcard -> /storage/self/primary

In boot mode /sdcard is just a symlink to /storage/emulated/0
while in recovery mode /sdcard is a bind-mount of /data/media/0

tar does not traverse symlinks by default, therefore the command works in recovery mode. for boot mode use tar -h flag, or provide directory path instead of it's symlink.

furthermore on windows cmd.exe may add additional \r carriage return for each \n linefeed. therefore avoid streaming plain data, instead use compression algorithm which produces no linefeeds at all. this can be done with gzip

adb exec-out "tar -ch sdcard | gzip" > sdcard_backup.tar.gz
adb exec-out "tar -c storage/emulated/0 | gzip" > sdcard_backup.tar.gz

This tarball archive can be extracted from tar as most tar implementations will auto-detect and invoke gzip automatically.

$ tar -vtf sdcard_backup.tar.gz
drwxrwx--x root/sdcard_rw    0 2018-01-01 01:00 storage/emulated/0/
drwxrwx--x root/sdcard_rw    0 2018-01-01 01:00 storage/emulated/0/Samsung/
drwxrwx--x root/sdcard_rw    0 2018-01-01 01:00 storage/emulated/0/Samsung/Music/

For windows we can use 7-Zip as alternative unpacking program, it will handle most of compression algorithms.

5
  • Very thorough answer, thanks! I will hopefully be able to try it over night again – lucidbrot Feb 13 at 22:40
  • Holy! That was faster than expected... oh right, I grabbed the wrong sd card. But for the internal SD card I can confirm that this method is working :D creating a 3GB backup of the internal sd card only took a few minutes. – lucidbrot Feb 13 at 23:33
  • At the time of writing, 9 hours later, my external SD card is still not done backing up with that command (from recovery, screen unlocked). I have aborted and got a 7GB file. That seems like it was very slow.. 7zip reports a compression rate of 95% but fails to open the (aborted) file as it is apparently faulty. I think I will go the way of just taking the card out and reading it on a different device to copy it. – lucidbrot Feb 14 at 9:15
  • adb pull on the external sdcard did stop at 48% though. blkid reports my ext sdcard as type exfat – lucidbrot Feb 14 at 9:27
  • 1
    You (and android) were right! My problems backing up were because my fs on the sdcard is broken as reported by fsck.exfat /dev/block/mmcblk0p1. Sadtimes. Thanks! – lucidbrot Feb 14 at 15:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.