I tested with my tablet and my phone and both react the same way:
shut down the device completely.
plug to computer USB.
device powers up (boots into android, not just show charging animations).
Is it an Android feature, or just for some devices?
This must be specific to some devices/kernels as I have just tried this and my Nexus 7 2012 briefly showed a "charging" battery animation before the screen going off - device then stays off. My HTC One X didn't show any charging animation, or power on but is showing the charging LED.