How can I configure Android so that all Internet traffic goes through a VPN and is blocked if the VPN isn't activated, without rooting Android?
This is Always-on VPN, which is a stock feature since Android 7 (root not needed). You will see the "Always-on" & “Block connections without VPN", options in VPN settings.
If you are looking to have this feature on pre-Andoid 7 devices, you need to look for VPN apps that offer this. For example, this app offers this feature by name of "Kill Switch"
On Android 10 with AnyConnect VPN: Settings → Connections → More connection settings → Select the VPN client → Enable "Always-on VPN" and "Block connections without VPN":