Alright, I went through the sourcecode and the answer is surprisingly obvious:
Yes, this can often be muted by simply using the volume keys while using the app or adjusting the volume of the media audio channel in the preferences. The media volume is used, regardless if the device is in normal, silent or vibrate mode. But it depends on where you bought your device if this works (I guess), some devices enforce audio over a channel that can't be muted.
More (technical) details:
The camera app checks the device configuration (system property
ro.camera.sound.forced) whether it should play the camera sounds over a system-enforced audio channel that can't be muted. If this setting is off, it uses the media and music channel instead.
In case that it uses the music channel, it ignores when the device is in silent mode though. This might be a bug/small oversight by the authors. This is why you have to adjust the channel volume instead.
I guess the property mentioned above is set depending on where you buy your device, since the sound is required by law in some places/countries as explained by eldarerathis in the comments.