You can disable these apps. This way the apps will not be running and won't be consuming any resources. You have not mentioned your Android version, so here's how you can disable an app in Jelly Bean, if you have another version of Android just look around the setting may be somewhere else.
Settings -> Apps -> All -> Select the app you want to disable -> Disable
You have also not mentioned what app you are talking about. Remember, this only works for system apps, and they will still take up space on internal memory. If you have a downloaded app, you may have to uninstall it.