I know there is a good answer to this question already, but I feel that there could also be a detailed technical answer as well.
When you install a new app on your phone that uses push notifications, and give it permission to access the internet, it registers interest in it's push service website with the Google Push Notification service on your phone.
The Google Push Service will then register interest in updates with the push notification server for the app via Web Sockets. This will also occur automatically when you power on your phone when the app has been installed.
When an update occurs, the message is passed from the push notification server to the Google Push Service. Then it is passed to the interested app, which then wakes up if sleeping, or loads it's push monitoring code if it's not loaded to load the notification and notify you of the message.
When you power on your phone, and it registers interest all at once, you may get a flood of push notifications. These are all the notifications since the last time the app last notified the server as you read them. This is done standard in Web Sockets to bring you up to date in what you said you registered interest in.
Some apps may have a server that sends a generic push message to the Google Push App to cause it to wake up the actual app, and then the actual app retrieves the real push message. Not all services do this. This is done for security and privacy.
The reason why the Google Push Notification service is used as a middle man is simply because of the fact that it is always running, unlike almost every other app on your phone. It will always be there to receive a push notification. While you could circumvent it, it could mean that you could miss notifications if your app is not running when they come.