Lots of apps use this connection to do their work. As well as Google's own apps such as Gmail and Hangouts, third-party apps that use any of the following features via the Google Play services library need such a connection to work fully:
- license verification or in-app purchases
- push notifications using Google Cloud Messaging
- Signing in with your Google account or syncing or backing up your data to Google's servers
- Most Google Play Games Services features (e.g. online multiplayer and leaderboards)
Even so, the indicator isn't really meant as an indicator about Google's servers. Since Google's servers are always there, Android checks if it can reach them as a way of knowing whether it can reach the Internet in general. That way, it's easier for you to know what is wrong if you're on a captive Wi-Fi network; that is, the kind that redirects you to a login or terms & conditions page.