I have many times come across this word. I even know a little about it but don't know exactly what it is.
In simple terms, the kernel is a key component of operating systems, which mediates between software and the hardware. When an application needs the hardware to do something (this is known as a system call), it sends a request to the kernel, which in turn interprets the command for the hardware.
Both the hardware and the software communicate both ways with the kernel.
In the context of Android, the kernel is not user-accessible without superuser permissions. Thus, many users flash a custom kernel, to tweak hardware settings such as CPU clock speed and File System Architectures.