As mentioned in comments, you are booting the phone into download mode. So technically your phone is turning on just fine but you are not doing it in the way that boots android.
Instead just press and hold only the power button until you see some flashing or the boot logo appears.
Download mode is like samsungs sort of equivalent to fastboot. It will simply sit there indefinitely until you send it something manually.
If you want to enter recovery, it's power + volume up instead.
If you can't actually boot android using the normal power button only boot, then you'll need to fix your install. You're in the best position if you already have a custom recovery flashed. If not, you'll have to do a more complete restore of the stock OS.