Bought a cheaper tablet and it only supports Android 8. I'd like to install OneNote for Android but Android V8 does not support it. Is there a way to force the upgrade of the Android OS version past the device's supported version limit?
Not in general. Android has some important differences from Linux and Windows, which are biting you here.
Were you sold this device as compatible with Microsoft OneNote?
If so, return it, because it isn't, and buy one that runs a modern version of Android: 12 or later.
Assuming not:
Upgrading Android is hard when the manufacturer doesn't help
This is sadly intrinsic to the way Android is developed, customised and distributed.
The hardware is much less standardised than on PCs
The world of Windows and Linux PCs started off with clones of the early IBM PC, XT and AT machines, and still has some compatibility with those 40-year-old computers. That means that the machines all have a lot in common. In contrast, Android devices' hardware varies a lot, even though they all use the same kind of CPU, and most of them look pretty similar on the outside.
Device drivers are proprietary
"Device drivers" are the add-on pieces of software for operating systems that let the operating system control hardware. They are different for each operating system: you can't use a Linux device driver on Windows or Android (there are some obscure and highly technical exceptions to this, but they won't help you here).
Windows and Linux ship with lots of device drivers, enough to let them install on many models of computer, and run well enough for them to download the right device drivers.
Android is not set up to allow downloading and installing of device drivers. Device drivers are written by the hardware manufacturers, and are not open source. They normally get distributed as part of a complete Android system image, which is specific to a model of device (or sometimes a few closely-related devices).
The hardware manufacturers keep their source confidential. One reason for that is that they're afraid of their hardware being copied by other manufacturers, who can sell it cheaper, because they haven't had to pay the development costs.
The reason for the closed-up, locked-down setup is that Android devices have to be usable by people with no technical skills, and thus need to be quite hard to mess up. Desktop and laptop computers are more flexible, but easier to disable by tinkering with their software.
The resulting problem
Producing an Android image to install on a device requires having the source for appropriate device drivers. If the device manufacturer does not produce a new version of Android for a device, you are stuck, to a first approximation.
Manufacturers of cheap tablets rarely produce updates for them, because it's work for which they don't get any money. They'd far rather sell you a new tablet.
You might find a workaround here ...
There are lots of groups that produce alternative Android distributions. They can do this because the core of the operating system is open source. None of them are within light years of supporting all the devices available. Supporting a device requires reverse-engineering the hardware well enough to write open source device drivers for it. That takes enough time that most of the devices that are supported are obsolete. The most widely-used one is LineageOS, and there are lists of them here, here, and here.
OneNote probably needs Google Play Services
Sadly, Microsoft OneNote probably requires Google Play Services. That's Google proprietary software that device manufacturers have to license from Google if they want to use the Android trademark. There is an alternative implementation, MicroG, that comes with some of the alternative versions of Android. I don't know if OneNote runs on it.