4

Device Details

  • Moto X Play - Android 6.0.1, Rooted (Systemless root)

  • Samsung 64 GB ext SD

  • San Disk 128 GB USB - came shipped with FAT 32 reformatted to exFAT since I have folders > 4 GB on external SD

Use Case

Sync entire contents of ext SD to USB (one way sync without deletion of SD card contents ) on weekly schedule / on demand , as incremental backup without using a PC ( I do have one but reasons for not wanting to use it are not relevant)

Other Details

  • USB drive recognised by device correctly connected by standard OTG (Settings → Storage & USB )

  • Busybox installed . It came bundled with ROM Toolbox Pro. I haven't used it directly. If another busy box is to be installed, it's fine as long it doesn't clash

  • Mount points as below

enter image description here enter image description here

Things tried

  • Unsuccessful app search- I wasn't able to find an app that can do this (Folder Sync claims to but it can't see USB)

  • Googling for alternatives

  • MacroDroid automation fails, since app can't see USB (raised this in app forum)

Question: How can I do this ? I am open to any means

Please note that I intend placing a bounty of 100 , any answer meeting needs before I post bounty will be awarded similarly (I may delay posting the bounty to ensure I get a solution)

Please avoid solutions that

  • are not tested - it should have been tested on your device and USB configuration to give it credibility

  • Involve PC , uploading to cloud, transferring to remote server etc

  • References to claims from doubtful sources which can't be verified

  • Involving use of File Managers ( I had mixed results with these but then that is for another question)

Aim of the above is not to deter solutions but to discourage frivolous attempts. If clarity is needed, please ask

I discussed this need on chat and some potential candidates for solution included BusyBox, rsync and Tasker. I know nothing of these means so Please do provide complete details. Being a bounty question (almost), I expect that I should be able to straightaway implement successfully without seeking clarifications.

( Being a rooted device with systemless root, solutions using root are fine. However, if OS is to be modified on any way, please highlight as systemless doesn't permit it, though there is a work around )

I am strapped for time and need to get this working ASAP

  • I'LL see if I can work on something When I can :) – Dan Brown Dec 16 '16 at 8:07
  • Ah, Fair Enough :) Hadn't Checked the Chat. – Dan Brown Dec 16 '16 at 9:50
3
+100

As we figured out the solution you now use in chat, this answer should comply with "being tested", "not involving a PC or file manager" and "relying on verified sources". Though in the meantime there was another rsync answer posted, you've asked me to present our solution as well – so here we go:

The rsync command can be used to accomplish that. Taken all your requirements into account (including those mentioned later in chat), the command line would look like this:

rsync -a --delete /mnt/sdcard /mnt/usb

Explanations on the command:

  • rsync obviously is the executable/command
  • -a stands for "archive", i.e. keep all attributes. This includes to recursively copying contents (directories, subdirectories, files), keeping filestamps and ownership (the latter usually requires root), preserve permissions, etc. For details, please refer to What is archive mode in rsync? on our sister-site.
  • --delete takes care to remove files on the target (here: /mnt/usb) which are not/no-longer available at the source (here: /mnt/sdcard)
  • remaining parameters specify the source to copy from (/mnt/sdcard) and the target to copy to (/mnt/usb). Note that we specified both as directories, so all content, also hidden files, should be included as well (the latter wouldn't if we had used a mask like /mnt/sdcard/*).
    Note: make sure to specify the real mount points or at least real directories here (e.g. not the /mnt/sdcard symlink as in the example), or it might just copy the symlink over and think that's all you wanted :)

You might wish to try the above on a small source first to see how it works.


Now that the "how" is covered, question remains: Where is rsync? It usually doesn't come pre-installed on the device, and most Busyboxes seem to not include it either. There once was an app on Playstore called rsync backup for Android, but it's no longer there. As this wiki points out, you can directly download the rsync binary from here¹, then push it to your Android device, and install it. Assuming you've pushed the binary to /mnt/sdcard:

su
mount -o remount,rw /system
cd /system/xbin
cp /mnt/sdcard/rsync .
chmod 755 /system/xbin/rsync
chown root.shell /system/xbin/rsync
mount -o remount,ro /system

The above should be done either in a terminal app or via adb shell.

That done, and successfully tested from a shell prompt, remains the question of . I know that can execute shell commands as root (in the past this was done by simply prefix them with an exclamation mark, today there might even be a checkbox for it – I didn't check that for a while), but your Macrodroid might provide that as well. Alternatively, you can do it the Unix/Linux way and utilize Cron, which is provided by Busybox; for details please see: How to use crontab in Android?


Also see:


¹ Of course you can also try to find the original app on sites like APKMirror or ApkPure, or download the app from here – but then you had to first install that app, and afterwards grab the binary from it's data directory /data/data/eu.kowalczuk.rsync4android/files/rsync – which I find a bit inconvenient ;)

  • Thank you, the solution worked fine for me. A couple of points that may help the readers : 1. I tried with all Busybox versions, which are well rated and none of them have rsync available the pro version of this ( I mailed developer requesting him to add it), so that leaves with no choice but other means ( incidentally couldn't find the app on reputed sites mentioned ) 2. Automating the execution of command on demand / schedule successful. Instructions here – beeshyams Dec 17 '16 at 16:10
  • 3. TWRP could see the backup on USB , so presumably it should execute fine 4. Same with TiBu , though this requires a bit of workaround (Backup folder location > Storage Provider, click on that and choose options per your OS Version) 5. Though you are not a windows user, for the sake of completeness, you may consider adding extension of rsync on windows as well, as you did for Linux- I did see references. Will add more later if it can help readers – beeshyams Dec 17 '16 at 16:26
  • @beeshyams ptspts.blogspot.com/2015/03/… has instructions on where you can find and manually install rsync for Android. You'll have to be a little creative since you don't want to use a PC. I'd recommend trying to use webget and Root Explorer to download, rename, set +x, and move the binary. – Chris Olin Dec 18 '16 at 4:41
  • @ChrisOlin: Thanks and I had seen that earlier, but not being familiar with Linux, did it the awkward way of downloading the app and extracting rsync and shared it with Izzy and he included that in his answer., In foot notes. Doing it can be learnt, but at a later time, when time permits. – beeshyams Dec 18 '16 at 5:52
  • Update : Verified TWRP and it restores fine from OTG. TiBu runs into problems. It sees the copied folder on USB but unable to select for restore. However, if I create a new Backup folder on USB , it restores. In case of extSD being replaced, I suppose copying the backed up folder should work since all attributes and permissions are intact – beeshyams Dec 18 '16 at 6:18
0

Be forewarned. I'm not going to provide you with complete details, mainly because what you're asking for is so unspecific and obnoxious that the bounty on this question isn't worth spending the time to test a solution for you that I have no way to guarantee will work on your device. You can either ignore my advice and suffer or reason out any difficulties through critical reasoning and Google. If you don't like these options, you can always go buy an iPhone. They're designed for people that want something that "just works". If you can't spare the time to research or tweak a solution to work for you, Android may not be for you.

There are a variety of ways to accomplish what you're trying to do. Arguably, the easiest and most reliable method is using software found on most Android devices by default, which is what I'm going to focus on. Since you're trying to run something on a regular schedule, I would say the best solution would be to use cron. On that note, take a look at this StackOverflow thread since it's specific to running it on Android. Also, so we're all on the same page, cron is a job scheduler available on most (all?) Unix-like operating systems, which includes Android. Since you didn't go in to detail on when exactly you want this job to kick off weekly, I'll leave it to you to configure the crontab (which is well-documented on the internet and isn't difficult).

This will take care of scheduling everything, but the next step is to figure out how you want to back up everything, which is a very important question you were painfully unclear about. Do you simply want to copy new/modified data on the external SD card to the flash drive or do you want full backups of the SD card stored by date on the flash drive? These are questions you need to figure out for yourself before you ask the internet for advice.

Without knowing what exactly you're trying to accomplish here, I'm going to assume the first scenario works best. Since you're copying file level data (opposed to block level), rsync is probably your best choice here. It's very versatile and should work straight out of the box without needing to be configured. You should already have it available and if you don't, it's easily obtained through Busybox.

Assuming the mount points stay the same, the command I would use to accomplish this would be: rsync -rlpt /mnt/media_rw/0000-0000 /mnt/media_rw/E82C-B290. For your convenience, I've included a primer on the flags I used below:

-r, --recursive recurse into directories

-l, --links copy symlinks as symlinks

-p, --perms preserve permissions

-t, --times preserve modification times

There's also the -a flag (short for archive mode), but it does a bunch of other needless crap in this situation like syncing uid/gid. In most backup scenarios, using -a would make sense since you would want to preserve ownership if you have to restore from your backups. However, external SD storage uses FAT for its filesystem, which doesn't support Unix file permissions. While files on your external SD card do have an owner/group set, they're statically set by the operating system. If you try changing them, I'm pretty sure you'll get an error or it'll silently fail in the background. The flags I used make this process much more efficient, but if for some reason you want rsync to sync all that extra stuff, use the archive mode flag instead.

So, now that we have the command you need to run, it's just a matter of configuring the crontab and having it execute that rsync command. This isn't a very elegant solution, but it will work. If you wanted to set up some sort of logging (Mike Rubel has done a lot of work you could draw inspiration from), you could write a shell script, put it on your phone's internal storage, and have cron run the shell script instead of the rsync command.

To back everything up manually outside of cron, you can run that rsync command using ConnectBot or any other app that gives you shell access. If you plan on doing that often, it might be faster to put the command in a file (e.g. backupsd), chmod +x it, and stick it in a folder that's also in the system path (like /system/bin). This way, you can type backupsd instead of rsync -rlpt /mnt/media_rw/0000-0000 /mnt/media_rw/E82C-B290.

  • That was basically the solution I worked out with the OP in chat :) Just one side-note: instead of -rlpt, simply use -a (archive mode), that implies all the other switches :) – Izzy Dec 16 '16 at 19:59
  • I addressed why I didn't use -a immediately following the summary of the flags I used and left it for the OP to decide. Clearly, great minds think alike. – Chris Olin Dec 17 '16 at 18:09
  • Ah, sorry Chris, I missed that part. But to me, syncing UID/GID is an essential part, for ownership is relevant so the owners can deal with it correctly. Guess you play on the fact that though being true, that doesn't affect the SDCard due to FAT, which I agree to :) You might have wanted to point out your reasons in your post, though :) – Izzy Dec 17 '16 at 19:15
  • 1
    You make a good point. I didn't state specifically why it's pointless. I'll edit and clarify. – Chris Olin Dec 17 '16 at 21:11
  • wtf with the paragraph rant at the beginning? – YetAnotherRandomUser Oct 20 '18 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.