I have a Moto Droid, running Android 2.2 (Froyo), although this behavior also existed on 2.1. Whenever I plug something in the headphone jack, the Music app will start up (sometimes with a bit of a delay) and start playing something (generally Christmas music). More often than not, I'm plugging in headphones to use Google Listen to listen to podcasts... if Listen is already running, I may end up with both apps playing at the same time.
The Music app doesn't seem to have any settings that would affect this behavior. Is there some way, short of deleting all of the music so that there's nothing to play?