I've noticed on many phones in Canada and I think in the EU nowadays come with Google Music preinstalled. If Google Music is only available in the USA, then why bother putting it on the phone?
Google Music (or Play Music as it's called now) isn't just used for music purchased from the Play Music store, or for music stored in its cloud service, it can also play music stored locally on the phone, or on inserted SD cards.
My UK Galaxy Nexus has it pre-installed, even though the Play Music Store and cloud storage service aren't available here. It is just the latest version of the built-in Android Music player, and works perfectly well for playing MP3s saved on the phone, even if you never use any of the cloud services or the store.
You would wonder what else comes pre-installed. Some phones have been seen in Germany with software special to some Russian mail providers -- almost causing panic being suspected of a malware attack.
As it is, it is easier for manufacturers to maintain one installation type, which they can ship internationally. As long as they don't exaggerate, and keep the system image in a decent size, this should normally not bother anyone -- most of those apps are either hidden and only show up with a matching SIM card being inserted. Unfortunately that (hiding) is not true with some other apps as e.g. google music -- but still it serves the purpose of "production ease".
Every Android devices comes with many apps already installed.
On one hand, these are useful and enhance the functionality of your device. But on the other hand, like in your case, they are useless apps that use resources, aren't available on the user's country, and some times can't even be uninstalled and.
This is better clarified below, but it's all about deals that have been made...
What apps come with the device?
Almost every device comes with essential apps to allow the user to use the device instantly. Some examples are: Browser, Camera App, Email, Clock, E-mail, Music Player.
It's common to find apps to access the wireless carrier services, either to entice you to subscribe or to access information about your account.
Google Own Apps
Apps developed by Google may or may not be pre-installed in your device. This is decided by the device manufacturer or wireless carrier. One can always install or uninstall as desired.
Wireless carriers or device manufacturers, bundle sample apps as part of a deal made with vendors. Usually this app can't be uninstalled.