Just want to add in schwiz's answer that it's just few lines of code needed to override the functionality of the back button. So it's not always reliable to close the app. So if the app developer doesn't want to close the app on pressing the back button, it won't get closed.
Although most good app developers popup an alert dialog on pressing back button on home menu of the app, asking if the user intends to exit the app or not, if you click on exit, it does close the app.
The correct answer could be this, the closure of the app is completely upon the app developer, if they wants the app to close on pressing the home button, they'll put the exit code in onPause(). If they want it on back button, they'll put it in the onBackKeyPressed. So it's not really in hand of the user.
The user can just avoid using the apps which doesn't provide the proper closing functionality, when not needed. The task managers are not a good way to close the app, since it can crash the app and doesn't give them a chance to save their state.
The OS does close the app whenever necessary, so you should not worry too much about it.
Edit:
There's an option to destroy the applications, as soon as you leave the app i.e. when you click the home key. You can go to Settings->Developer Options->Don't keep Activities. Although it's a developer option, but if you're too concerned about the apps running in background, you can use this option. But keep in mind that it's not recommended to be used by non-developers.