In gmail (on the web) you go to settings -> general. Under there is "My Picture".
Excerpt from Gmail Help:
- Click the gear icon at the top of any Gmail page.
- Click Mail settings.
- From the My picture section, click Select a picture. The Upload a picture window appears.
- Upload a new photo from your computer, or select a photo from Picasa Web Albums or your previous Google profile photos.
- Drag the selected region to crop your photo, or resize the region by clicking and dragging one of the region corners. Click Edit photo in Picnik to edit your photo.
- Click Set as profile photo.
You can select whether you'd like all Gmail users to see your picture, or only those who you've allowed to chat with you, using the radio buttons next to your uploaded photo on your Settings page.
- Visible to everyone means anyone who you email, or who emails you, can see your picture. If you have a public Google profile without a photo and you choose this option, your picture will be used on your Google profile and will be visible to others there.
- Visible only to people I can chat with includes users who have been given permission to see when you're online and to chat with you.
As of the time that I am writing this, I believe that the Gmail picture and the Google+ picture are still separate from each other, but Google+ profile picture, IIRC, defaulted to this image.
I want to add that ICS added the "Me" Contact. From what I know, this contact is tied to your Google+ account, and no one has seem to find a way to edit the info without deleting the contact and re-adding it.