It seems that the hardware erroneously regard the headset as attached at all times (to the 3.5" jack). This makes me unable to make calls without headset or speaker mode.
This is a hardware problem, but there is an app called toggleheadset2 that has a widget that allows you to override the hardware switch, toggling headset output off/on.
I've been having this problem for a long time and the only way to turn on/off headset/speaker was to reboot my Sony Ericson Xperia (waterproof edition). After a long period of testing and messing with settings, cleaning, etc, I realised it should be a sensor/software problem.
Note: I tried cleaning the plug multiple times but it's still not working.
SoundAbout works well but may not be needed. Plug in your headphones, power down the phone, power on the phone then remove the headphones. This worked on my GS3. on my GS3 the headphone icon was never on, but the phone was stuck in headset mode. Could not hear any calls, could not hear any music, UNLESS the headphones were plugged in. The speaker and Bluetooth icons were disabled so I could not switch to them.
I've been having this problem with my Samsung Galaxy Note III. It always thinks that headphones are plugged in. I even replaced the headphone jack and cleaned the contacts but the problem persisted. Soundabout did indeed get around the issue by rerouting audio. Setting media audio to speaker and phone call audio to earpiece worked well. However whenever I want to put a call on speaker, or actually use headphones it definitely got cumbersome having to open the app and change the setting. So I tried ToggleHeadset2 which is designed to instead fool the phone that headphones are or are not plugged in. The app was listed as incompatible so I got an apk for it instead, and it did nothing for me. Then I found HF Button Widget (<=KitKat). As the name suggests it supposedly only works on KitKat or below, and I'm on lollipop. But it actually did work for me.
So with this I get a widget which I use to toggle the headphones. I practically always have it at the off-setting, which makes the phone behave as it normally would with no headphones plugged in, e.g. I can use the native speaker toggle in calls to switch between earpiece and speaker. I very rarely use headphones which means the problem for me is completely solved in that sense. When I do want to use headphones I have a one-click widget to use. One minor annoyance is that the headphones icon always stay in the status bar.