Nowadays, there's a Delete button on the Play website:
So, the remainder of this post is just in case things change again at some later time, especially as in November 2012 Google has made things worse, but also kind of solved the problem for existing reviews by removing your name:
Google Play is now connected with Google+ to help you find reviews you trust.
New reviews will be posted publicly using your Google+ name and picture. Your name on previous reviews will appear as "A Google User".
Also note that someone reported Issue 16483: Android Market doesn't allow removing an existing application's rating and comment. Voting might help! Or maybe there's some alternative market app that interfaces with Google's store?
Despite what many write: clearing all text including the title does not clear the review for me; it just moved my old review to the top again, with a new date and an updated version number. In the Play app the "OK" button is disabled when clearing the title, so I guess the title text is mandatory. Indeed, just typing a title on the website does save the new review with an empty description. Still, that does show my name, rating and title.
To edit one's own review on the website, even if it's actually shown on the main page: first click "All reviews" or go to the tab "User Reviews", then find the "Edit Review" button next to "My Review". Note that different languages might show different reviews! (I cannot find my review in the Dutch Google Play, as I entered it using an English Play app?)
In the Play app one can swipe the stars to the left to get "unrated", but then the "OK" button is disabled.
Uninstalling the reviewed app does not seem to remove one's review. (Which makes sense, I guess.)
As true for new or updated reviews too: it takes quite some time for changes to propagate. Even once the new review is shown, refreshing might show the old one again. I guess there's multiple servers involved and it takes some time to synchronize. Be patient, for otherwise you might actually activate the old review again.
Old versions of the Market app reveal an option "Remove my review" by simply pressing the hardware menu button, when viewing an app one reviewed. That seems to have gone in 2.3.5 and I cannot find it in Play on 4.0.4 either.
I tried with an old Android 2.1 device, with Market version "2002306". That old Market app still works just fine, but not all apps are visible on that old device, and some show "The requested item could not be found" when selecting it.
Sniffing the network first shows some HTTPS request, probably authentication. Removing the rating gave me a POST to
POST /market/api/ApiRequest HTTP/1.1
This is a request like many others that people have been deciphering. Unfortunately the unofficial Android Market API does not support these rating requests.
Using "Manage apps" I removed all updates for the Play Store 3.5.19 on Android 4.0.4, getting me the version 3.4.7. No luck.
I also tried AppBrain, hoping it would somehow interface with the Google Play Store and would then allow for changing ratings. No cigar.