Softbricked my device with encryption

Question

Because numeric passwords are trivial to bruteforce, I decided to change my encryption password with a passphrase with the following commands:

su -c vdc cryptfs changepw newpass

To my horror, I discovered upon rebooting that Lollipop only offers a keypad to input decryption PINs, and I have no way at all to type in letters. I have tried using TWRP to unlock my /data partition, but I get this error message:

could not mount data and unable to find crypto footer

Then I tried using the command line from adb to mount it manually with

/system/bin/vdc cryptfs checkpw newpass

But I get another error:

CANNOT LINK EXECUTABLE: cannot locate symbol "__FD_SET_chk" referenced by "/system/bin/vdc"...

I'm totally at loss here... Can anyone help me remount this partition?

Answer 1

Okay, there were actually three separate issues here.

1. TWRP 2.8.6.0 is broken for my Nexus 4 (5.1.0)

Regarding the following error message:

could not mount data and unable to find crypto footer

The current Team Win Recovery Project (2.8.6.0) has a regression for some encrypted Lollipop devices.

Solution: revert to TWRP 2.8.5.2, which is able to decrypt the /data partition properly.

2. TWRP does not provide a working shell for some binaries

Regarding the following error message:

CANNOT LINK EXECUTABLE: cannot locate symbol "__FD_SET_chk" referenced by "/system/bin/vdc"...

TWRP doesn't seem to set up the shell properly for vdc.

Solution: Switching to ClockworkMod Recovery provides a working environment for the vdc binary.

3. Lollipop ignores changepw

The command su -c vdc cryptfs changepw newpass had no effect on my device after upgrading to Lollipop (it had worked perfectly on KitKat), and the PIN of the device was still the encryption password.

I hope this helps someone in the future!