In my searching I stumbled across Revolutionary. It looked good enough, so I downloaded it and ran it. It prompted me to plug my phone in, and then it pretty much did the rest on its own. The result was that a custom recovery system was set up, from which it is easy to install ROMs and the like to the phone.
The Revolutionary docs mentions a zip file you need to install in order to root the phone. In order to install the zip file, I first downloaded it to my phone's SD card. Revolutionary only sets up the mentioned recovery system, so the official HTC Android 2.3 was still intact and running on the phone. I used that to mount the SD card on my computer and copy the zip file, but I could just as well have done it using Revolutionary.
After placing the zip file on my SD card, I shut down the phone, then held the volume down button and pressed the power button. This results in starting the phone's bootloader, from where one can start the recovery system (Revolutionary in my case). This greeted me with a neat menu containing the entry install zip from sdcard. I selected this and navigated to the zip file I downloaded. When the installation was complete, I rebooted the phone into Android, and my app drawer now contained the Superuser app. Root complete!