Why there is not one driver file compatible with all android versions ?
Because it's not possible to write one driver that's compatible with all Linux kernels. The driver has to interface with the kernel, and different kernel versions change this interface for many reasons - to enable new features, to improve efficiency, and to fix security problems. This is what makes them "different versions". Each new Android version standardises on a different version of the Linux kernel, so the drivers need some source code changes to work with the new version.
There's also a lot of testing that needs to be done with the new versions, to make sure the changes didn't break something.
how xda developers make to compile android 8.0 for the nexus 4 while android oreo 8.0 driver binaries are not available for the nexus 4 ?
As the summary you've screenshotted clearly says, many features such as NFC just don't work with this port - likely because drivers are not available. Sometimes, even though a driver for the particular hardware in this phone might not be available for newer Android versions, there might be a newer version for desktop Linux, or a driver for a similar device, which is unsupported but can be made to mostly work with some fiddling.
Also, hobbyists might be more interested to try something even when it's not economically worthwhile for the manufacturer, and there's a lower bar for them. If Google or LG shipped a release where only half the features worked and even those had only been tested on one phone, you'd be pretty upset, but if some guy on XDA Developers does it, that's a success.
It really is a success: I don't want to detract from what Ziyan has done here. It's just that the quality bar is higher for an official port from the manufacturer, and they don't have free time to spend supporting old devices.