Why would the TRWP Nandroid backups of system and vendor partitions be different between backups?

Question

For background info, I just picked up a Xiaomi Mi 9 Global version and am running the latest weekly Xiaomi.eu Pure MIUI ROM on it (if it matters, the latest Xiaomi.eu weeklies for the Mi 9 are Android 10 based).

I've started taking full backups using TWRP and I noticed something strange. For reference these backups have compression disabled and no encryption to try and rule out any variables.

When I check the sha2 hashes of the various partitions, I've noticed that all of the various partitions (boot, recovery, modem, efs, etc) have the same sha2 hash between backups (meaning the images are exactly the same). However even just from one night to the next morning, or even from one boot to the next as I tested today, the hashes of the vendor and system partition backups are different. I don't have any custom system modules or kernels installed that could be modifying those partitions, though I am rooted with Magisk.

I used adb shell to browse around the system and vendor partitions and all of the date stamps of the folders and files are either "1969-12-31" (basically meaning the date is 0 since that's the beginning of "unix time if you take into account my time zone) or "2019-09-06" which is the probably the date that the ROM or it's base ROM was created (the original Xiaomi ROM they made the Xiaomi.eu version from). I even used the find command from an adb shell to search both the system and vendor partitions for any updated files (any file with a date newer than "2019-09-06") and there are none.

So my question is this: if I haven't flashed a new ROM or installed anything in the system partition, why would two backups be different?

I would think that only the data partition would change as I use the phone. I thought that since the system and vendor images are the ROM itself, they should never change unless I change ROMs. I was under the impression that even system settings are stored on the data partition because performing a factory reset essentially just wipes that partition...so I don't see why the system and especially vendor partitions should ever change unless I specifically flash a different ROM or update the ROM which would obviously write to those partitions (or at minimum the system partition)

Are there some kind of auto-generated files or folders in the system and vendor partitions that change slightly on ever boot or something?

I would just like to understand this better so I know whether for example it's ok to just do regular backups of the data partition and only back up system and vendor when I do a new ROM update.

Hopefully someone with more Android ROM experience can shed some light on this.

Answer 1

Answer to your question is dm-verity:

The dm-verity feature lets you look at a block device, the underlying storage layer of the file system, and determine if it matches its expected configuration. It does this using a cryptographic hash tree. For every block (typically 4k), there is a SHA256 hash.

So /system and /vendor partitions are always mounted read-only because mounting R/W causes changed hash values of the partition/filesystem (at least some inodes belonging to filesystem structure may change even if you don't touch a file), thus breaking the security feature. To make any changes to them, we need to disable dm-verity which must be your case.

TWRP creates backups in two ways. boot, recovery etc. are just dumped (dd) as a whole, so their hashes don't change because those aren't mountable filesystems. /system and /vendor are either dumped as images to retain dm-verity as discussed above. Or otherwise a tar archive is created to keep the backup size minimum. In second case even the filesystem is unchanged, a new archive file is created every time you do the backup. If the files aren't changed and are read/written by the archiving utility in the same order, using same options/parameters every time, then created archives should be identical. But even a minor factor e.g. compression may cause a different hash value every time.

So you must have dm-verity disabled, and therefore you will keep on getting changed hashes. But the contents don't change (if you don't change yourself or Magisk/GApps don't put some addon.d script), so a regular backup of both partitions isn't required. In fact only /data backup is enough, boot, /system, /vendor are already part of flashable zips.