To install the stock ICS build: go to https://developers.google.com/android/nexus/images and choose your device (mine, for example, a US Nexus S that works with T-Mobile, is https://developers.google.com/android/nexus/images#sojuimm76d), download the file, then download the surprisingly-not-very-large Android SDK from http://developer.android.com/sdk/index.html. Run the sdk manager (it's called "tools/android") which will open a GUI; make sure "Android SDK" and "Android SDK Platform Tools" are checked. You can clear other check boxes, since those will download very large stuff that you don't need.
Once you've got all that, follow the instructions in the tarball that you first downloaded; roughly, you want to put the sdk's "platform tools" directory on your PATH, power off your phone, then hold the power-up key and power it back on (that'll get it to display the Sekrit Startup Screen); and then on your computer run "fastboot oem unlock" and then "flash-all.sh".
I've done this twice in the last few days (for fun :-) and while it may sound complicated, it's actually pretty easy and worked fine. (I installed 2.3.6 just for the nostalgia of it and then installed Cyanogenmod and then went back to stock 4.0.4).