3

I have built LineageOS 17.1 for Motorola X4 / payton with unlocked bootloader with reverted commit 81cc203c06596878d2beb62ab6e07f36e278018e. The common question is how to disable dm-verity, but I want to know how to properly enable it for /system . The vendor partition is called oem on Motorola.

AVB was deactivated (the device is missing fastboot flash avb_custom_key but has a vbmeta_a/b partition?) During build these options were set:

PRODUCT_SUPPORTS_BOOT_SIGNER := true
PRODUCT_SUPPORTS_VERITY := true
PRODUCT_SUPPORTS_VERITY_FEC := true

I checked system.img and vendor.img with verity_verifier script and the verity_key and it outputs VERIFIED. I flashed boot, vendor and system to the corresponding partitions. /verity_key is inside the boot.img. The boot cmdline contains androidboot.veritymode=eio and veritykeyid=id:47b1fe9xxxxxx. The boot.img only contains recovery.fstab, verify option is set there, too. adb enable-verity is succesful for / and for /vendor. /vendor/etc/fstab.qcom contains the verify option for system and vendor partition.

The following obversations were made:

  • During boot I see a message "Verity mode is set to disable". What is this referring to?
  • Modifiying data (touch test) on the vendor/oem partition from Lineage Recovery results in the errors being corrected by FEC, messages shown from dmesg after next boot:
[    3.023786] init: [libfs_mgr]Enabling dm-verity for vendor (mode 0)
[    3.175842] device-mapper: verity-fec: 259:31: FEC 0: corrected 21 errors
[    3.257369] device-mapper: verity-fec: 259:31: FEC 4096: corrected 17 errors
[...]

The created file test vanishes after boot.

  • Modifying data on the system partition from Lineage Recovery is not corrected by FEC and no messages show up. The created file test is visible on the partition after next boot.
  • There is no difference between locked and unlocked bootloader (fastboot flash lock).

What am I missing here?

1

1 Answer 1

3

Warning! Use at your own risk! The sense and especially added security of all of this may be disputed. Find some notes about security in the end.

tl;dr:

It took me some days to find a solution to get this:

[    2.251328] init: init first stage started!
[    2.251561] init: Switching root to '/first_stage_ramdisk'
[    2.252614] init: Using Android DT directory /proc/device-tree/firmware/android/
[    2.603025] init: [libfs_mgr]Enabling dm-verity for system (mode 0)
[    2.603181] init: [libfs_mgr]Verity table: updated block device from '/dev/block/platform/soc/c0c4000.sdhci/by-name/system' to '/dev/block/platform/soc/c0c4000.sdhci/by-name/system_b'
[    2.603195] init: [libfs_mgr]Verity table: updated block device from '/dev/block/platform/soc/c0c4000.sdhci/by-name/system' to '/dev/block/platform/soc/c0c4000.sdhci/by-name/system_b'
[    2.753873] device-mapper: verity-fec: 259:33: FEC 0: corrected 22 errors
[    2.830804] device-mapper: verity-fec: 259:33: FEC 4096: corrected 7 errors
[    2.831080] init: [libfs_mgr]superblock s_max_mnt_count:65535,/dev/block/dm-0
[    2.854916] device-mapper: verity-fec: 259:33: FEC 0: corrected 22 errors
[    2.878336] device-mapper: verity-fec: 259:33: FEC 4096: corrected 7 errors
[    3.001281] device-mapper: verity-fec: 259:33: FEC 1052672: corrected 18 errors
[    3.001825] EXT4-fs (dm-0): mounted filesystem without journal. Opts: barrier=1
[    3.001947] init: [libfs_mgr]__mount(source=/dev/block/dm-0,target=/system,type=ext4)=0: Success
[    3.002215] init: Switching root to '/system'
[    3.139199] device-mapper: verity-fec: 259:33: FEC 3133440: corrected 12 errors
[    3.272389] init: [libfs_mgr]Enabling dm-verity for vendor (mode 0)
[    3.272662] init: [libfs_mgr]Verity table: updated block device from '/dev/block/platform/soc/c0c4000.sdhci/by-name/oem' to '/dev/block/platform/soc/c0c4000.sdhci/by-name/oem_b'
[    3.272679] init: [libfs_mgr]Verity table: updated block device from '/dev/block/platform/soc/c0c4000.sdhci/by-name/oem' to '/dev/block/platform/soc/c0c4000.sdhci/by-name/oem_b'
[    3.276744] init: [libfs_mgr]superblock s_max_mnt_count:65535,/dev/block/dm-1
[    3.279830] EXT4-fs (dm-1): mounting with "discard" option, but the device does not support discard
[    3.279845] EXT4-fs (dm-1): mounted filesystem without journal. Opts: barrier=1,discard
[    3.279935] init: [libfs_mgr]__mount(source=/dev/block/dm-1,target=/vendor,type=ext4)=0: Success
  • Build LOS 17.1 with the reverted commit from the initial question

  • Place verity_key in ramdisk/first_stage_ramdisk folder

  • Append the following cmdline options, this is the android testing key:

    BOARD_KERNEL_CMDLINE += root=/dev/dm-0 dm=\"system none ro,0 1 android-verity Android:\#7e4333f9bba00adfe0ede979e28ed1920492b40f /dev/block/by-name/system_b\" androidboot.slot_suffix=_b rootwait androidboot.force_normal_boot=1

  • Add system partition to the DTB fstab and include verify option

There are more caveats here than I thought. I will also try to include all words I was searching for in this text.

Key findings regarding booting and dm-verity about the payton

  • It seems to be a legacy SAR (system-as-root) device (most important read!). The kernel mounts the root/system filesystem itself, no ramdisk or ramdisk partition involved. Typical cmdline arguments for the kernel are root=/dev/block/mmcblk0p6x, where x is 5 or 6 for the system A/B partition and init=/init.
  • The original motorola files do not contain a fstab in the ramdisk
  • There is no system partition defined in the DTB in the motorola files
  • Therefore the bootloader seems to verify the boot image, most likely with some certificates baked into the bootloader? (strings -a on the bootloader hints at this). Then it sets the appropriate start partition and maybe the command line for verity verification, more on this later.
  • vendor is called oem on motorola.
  • Like mentioned the payton is missing fastboot flash avb_custom_key, so no new shiny Android Verified Boot (AVB) 2.0 with vbmeta.img. Most likely because it was initially shipped with Android 7.0 and nobody cared updating the structures.

Some things about building LineageOS 17.1

  • Build LOS like written in their help
  • LOS seems to also boot legacy SAR, except when you enter recovery mode.
  • Using the instructions below will give you something like Android Verified Boot 1.0
  • The build process prepares the following things for dm-verity integration: Creating boot.img, system.img and vendor.img, adds dm-verity metadata to the end of system.img and vendor.img and signs boot.img. vendor is verified in the default LOS configuration by adding it to fstab and additionally creating a DTB fstab entry for it.

The general build process

  • Codebase is impressively huge: 190 GB without ccache

  • Revert the mentioned commit in dm-android-verity.c

  • Use the following Kernel cmdline.

  • This circumevenes A/B and slotselect mechanism for selecting the proper system_a or system_b. Without bootloader support I do not know if it is possible to ever reenable A/B.

  • Also this configuration always skips recovery, even if you try to boot to recovery from bootloader.

  • This did not work with androidboot.veritykeyid, I do not know why (yet).

  • You have to escape " and # here!

device/motorola/sdm660-common/BoardConfigCommon.mk:

BOARD_KERNEL_CMDLINE += root=/dev/dm-0 rootwait BOARD_KERNEL_CMDLINE += dm=\"system none ro,0 1 android-verity Android:\#7e4333f9bba00adfe0ede979e28ed1920492b40f /dev/block/by-name/system_b\" BOARD_KERNEL_CMDLINE += androidboot.slot_suffix=_b BOARD_KERNEL_CMDLINE += androidboot.force_normal_boot=1 BOARD_BUILD_SYSTEM_ROOT_IMAGE := false

  • without force_normal_boot a "normal" boot goes to recovery.

kernel/motorola/msm8998/arch/arm/boot/dts/qcom/sdm630.dtsi:

system { compatible = "android,system"; dev = "/dev/block/platform/soc/c0c4000.sdhci/by-name/system"; type = "ext4"; mnt_flags = "ro,barrier=1"; fsmgr_flags = "wait,verify,slotselect"; };

  • Maybe you can also leave out the DTB change (mounting partitions with DTB will be obsolete in Android 11) and use a fstab file (somewhere, maybe in first_stage_ramdisk).

  • make sure verity is enabled in ./make/target/product/verity.mk

PRODUCT_SUPPORTS_BOOT_SIGNER := true PRODUCT_SUPPORTS_VERITY := true PRODUCT_SUPPORTS_VERITY_FEC := true

  • When changing the verity key, do not forget to add it to the kernel in kernel/motorola/msm8998/certs and check the certificates in the running system in /proc/keys
  • You can get the last (failed) boot messages in /sys/fs/pstore/console-ramoops
  • The resulting system behaves like 2SI ramdisk SAR

Main Question: Does this add security to the device?

  • You can lock the bootloader afterwards with fastboot flashing lock
  • The boot partition has to be verified manually with the key ids shown while booting (at least I think it is some signatures of the signed boot.img)
  • Flashing partitions from bootloader does not work
  • Entering recovery also does not work
  • fastboot boot file.img gives an error
  • fastboot flashing unlock clears userdata, this is good!
  • In case anything goes wrong with the running system, you have to clear your device.
  • Updating just got a little bit harder without changes to the updater because the A/B partition is hardcoded
  • Root needs to be enabled for updates (at least ADB root), so this will stay a userdebug build.
  • Changes to /system can be executed without messing the dm-verity, just sign the changed system.
  • dm-verity is in eio mode here, I actually did not try to set it to enforcing.

Open questions

  • Can you change the kernel cmdline from bootloader?
  • Maybe you can blankflash (fastboot oem blankflash) the device, I do not know what happens in this case.

Have fun!

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .