Specifically, I'm looking for the drivers responsible for the tf101 and sl101 detachable and slideout keyboards in ICS.
But in general, where are the drivers located inside the Android OS itself?
They're most of the time baked right into the monolithic Linux kernel (or sometimes available as loadable .ko kernel modules located in /system/lib/modules/). Note: Linux kernel modules are build-specific and cannot be interchanged across devices, different firmware versions.
See the Wikipedia entry for Linux kernel modules