If my Android device is stolen, will the person who ends up with it be able to get my data by plugging it into a PC and using some trick? Is there anything I can do to prevent this or make it harder?
If you used full-disk encryption, then your data will be accessible until the device unmounted the encrypted disk (e.g. by rebooting) and discarded the decryption key. If you store your sensitive data in encrypted container, then your data will be accessible until the application closes the container and discarded the decryption key.
If you don't use any encryption, then your data is naked in the wild. Many thiefs are simply interested to sell the phone to make a quick buck and so will simply reformat the phone; but if the thief has a specific intent to steal your data (e.g. corporate espionage, identity thief, etc), then you can assume that they can read all your unencrypted data.
If your phone uses SD card (instead of internal memory), then getting to your data in the SD card is trivial. Getting data in internal memory took a little bit more effort, but is still relatively easy. A turned off device with full-disk-encryption is -- for all practical purpose -- a useless brick for your adversary.
In general computing, physical access to a device means all bets are off when it comes to security of data. A determined and knowledgeable thief is likely to either crack the protection, or at the very least copy the contents off the device to try and subvert encryption.
There are several security packages on the Market that can make this job a lot more difficult for the potential thief. These apps can remotely lock the device, wipe its data, and even attempt to "dial home" if the thief was careless enough to let it go online. This will deter most, but the very savvy ones.
Your best bet to safeguard sensitive data is to never have it on the mobile device in the first place. Banking apps that store your login credentials, entering credit card info into web forms on browser, etc. are all points of vulnerability.
In terms of damage control after the phone/tablet is stolen, it's always a good idea to have a list of accounts for apps that are installed, so that you can change the passwords asap. Email, social network apps, etc.
As the other answers already state: physical access to a device means all bets are off when it comes to security of data (from Chahk's answer). As this is already made clear enough, I will focus on some counter-measures with my answer.
One thing to make it harder was already mentioned in Lie's answer: encrypting the device will make it harder to access the data. But there are several more things. If you e.g. leave USB Debugging on, you make it easier for the thief to access your device, so it's a good precaution to keep that turned off.
Already named in several answers are several anti-thief apps available on the playstore. There's e.g. Cerberus Anti-Theft (one week free trial, then ~EUR 3) which gives you a lot of possibilities, remotely controlling your device to locate and track it, set off a loud alarm, record audio from the microphone, and even remotely wipe the phone (to destroy all stored data). Similar functionality is offered by Android Lost (available for free -- also see the video on that app's playstore page). Both are even using a "stealth-mode", so a thief can not easily find and uninstall them. A good thing about Android Lost is you even can remotely install and activate it if your phone already is lost -- as long it is still online and connected to your Google account.
So in short and summed up: Yes, a thief can get access to your data -- but there are means of making this at least very hard, or even limit the time frame he has for his "bad deed". Several things to consider on this background: