I would like to get a clear picture of what is in the ROM vs. what is in the OS. This will help me understand what those custom ROMs are all about.
Well, the OS is in the ROM, to start with.
You're suffering from a terminology problem: "ROM"s these days are all typically Flash memory, which is erasable and rewritable, but not byte-by-byte. So the OS may even live in the same physical ICs as the 'main' phone memory.
Basically, custom ROMs can enhance/modify/break EVERY aspect of your phone's operation. There is a separate 'radio rom' which includes the low-level drivers for the wifi/gps/3g/gsm/bluetooth functions, and these typically aren't 'customised' but you may still find newer versions available that can improve certain aspects of your phone.
ROMs are still mostly compiled official Android source code (or apps, etc. ripped from dumps of newer phones backported to older phones) with tweaks and some minor custom utilities or features that are not included in the stock ROM for a specific phone.
The biggest point of ROMs is getting newer versions of the Android operating system (or as much as the phone's hardware can support - see the G1 as limited device example when it comes to Android 2.1+) on older phones. This is perhaps the main point of the CyanogenMod.
ROMs also may be very similar to stock ROMs but are of different "themes" that change the colors of the notification bar, etc.