You have to uninstall them.

Android apps are able to register with the underlying Android system to receive certain events and service requests, and the system knows to start the app up if those events or service requests come in.

Unlike on a standard desktop apps "running" on Android are often not actually running at all (they are blocked until an event or update occurs ... or to refresh state from the network).  In general, Android apps are much simpler to suspend and resume than desktop apps, so the system can swap them out if the CPU or memory are necessary for another app without much complication.  So, other than having a clean list of running apps, there isn't much to be gained by killing apps.

That said, if a specific app is behaving badly and using too much CPU for the value you get from it, then uninstall it and/or ask the developer to fix their badly behaving app.  But really, if the app doesn't show up on the 'battery usage' meter, then you don't really have anything to worry about.

See [this question](https://android.stackexchange.com/questions/755/how-can-i-prevent-unwanted-apps-from-automatically-running-on-android) for some answers for more specific apps.