There is now a new attack that targets wireless networks and devices connected to them. Simply connecting to a corporate wireless network (one that uses EAP for security) is enough, if you are running a vulnerable version of Android. However, it is unlikely (don't quote me on this!) that they will be able to retrieve anything particularly sensitive from your Android device with this method. Maybe your wireless connection password.
You can use a detection tool (more info) to check if you have a vulnerable system OpenSSL lib on your device. Note that, as lars.duesing mentions, it's possible that specific apps are statically linked against vulnerable versions different from the system library.
According to this comment on Reddit, certain versions of Android are affected by this bug. Worse still, some browsers, especially the built-in one and Chrome, possibly use it and are therefore vulnerable.
Android 4.1.1_r1 upgraded OpenSSL to version 1.0.1: https://android.googlesource.com/platform/external/openssl.git/+/android-4.1.1_r1
Android 4.1.2_r1 switched off heartbeats: https://android.googlesource.com/platform/external/openssl.git/+/android-4.1.2_r1
That leaves Android 4.1.1 vulnerable! A quick grep on my access logs reveal there is a lot of devices still running 4.1.1.
Some other sources indicate that 4.1.0 is also vulnerable.
It seems the easiest way to fix it is to upgrade off that version, if possible. If you're lucky, your carrier will release a new version - but I wouldn't count on it. If not, you may have to investigate custom ROMs, possibly a downgrade, or rooting and manually replacing the library.
It is highly recommended that you resolve this issue. This bug can result in the theft of data, including usernames and passwords, from your browser by a malicious server.