Assuming an Android version of 3.1+, external drives should be supported via USB host mode by most devices -- see Compro01's comments below:
Not all devices support USB host mode, 3.1+ or not. Chainfire has a nice app to test compatability (and an extensive database of reports from that app on which devices work and which don't).
Also, if we're talking about an actual hard drive, rather than a flash drive, power is likely to be an issue unless the drive is self-powered (has a power brick of its own). If it's bus-powered drive and it fails to spin up, you'll need another source of power, such as a self-powered hub.
Of course they must carry a supported file system. You assume correctly that FAT32 definitly should be supported. Some others might be supported as well, depending on what was compiled into the kernel for your ROM. In many cases, this also includes extfs (EXT2/3/4). If you're either re-formatting the drive, you can simply try-and-err in case you prefer one of those.