DNS resolution in Android is handled by Net Daemon (
netd) which runs in background as an
netd uses the DNS server set in Wi-Fi settings or that received through DHCP. Or it can possibly be set through CLI. See details in How does Android OS do DNS name resolution?. But there is no way to specify DNS server on per app basis.
Using Android's VPNService API it's possible to control network traffic on per UID basis (every Android app owns a unique UNIX UID). Or with root access the same can be achieved using
iptables to NAT traffic going on port
53. An example VPN based open-source app which lets whitelist apps and configure DNS server is personalDNSfilter (I have no affiliation). It's configurable enough and supports root mode, encrypted DNS (DoH, DoT) and other features.
However in both cases (VPN or
iptables), you won't be able to use Android's Private DNS (DoT) feature. It's because DoT (DNS over TLS) queries aren't plain DNS (like HTTP) but encrypted (like HTTPS), going on port
853. So the only entity that can receive and respond to the encrypted DNS queries is the DNS server set in Private DNS settings. Though it's possible to intercept DoT (like we can intercept HTTPS) but the hassle doesn't worth a try.
If you don't want to go with VPN or root based solutions, or don't want to leave Private DNS, another option is to setup DNS forwarder on local network, preferably on default gateway if possible. Or your app must itself have capability to make DNS queries, not depending on Android's resolver.