The Android Development document Tasks and Back Stack actually covers this pretty nicely. A relevant excerpt:
A task is a cohesive unit that can move to the "background" when users
begin a new task or go to the Home screen, via the HOME key. While in
the background, all the activities in the task are stopped, but the
back stack for the task remains intact—the task has simply lost focus
while another task takes place, as shown in figure 2. A task can then
return to the "foreground" so users can pick up where they left off.
Suppose, for example, that the current task (Task A) has three
activities in its stack—two under the current activity. The user
presses the HOME key, then starts a new application from the
application launcher. When the Home screen appears, Task A goes into
the background. When the new application starts, the system starts a
task for that application (Task B) with its own stack of activities.
After interacting with that application, the user returns Home again
and selects the application that originally started Task A. Now, Task
A comes to the foreground—all three activities in its stack are intact
and the activity at the top of the stack resumes. At this point, the
user can also switch back to Task B by going Home and selecting the
application icon that started that task (or by touching and holding
the HOME key to reveal recent tasks and selecting one). This is an
example of multitasking on Android.
Note: Multiple tasks can be held in the background at once. However,
if the user is running many background tasks at the same time, the
system might begin destroying background activities in order to
recover memory, causing the activity states to be lost. See the
following section about Activity state.
Summary: The task is moved to the background when you press Home and sits in memory, not really doing anything and retaining its state. This means that (generally speaking) you can switch back to the task and it will pick up where it left off. However, the Android system can - and will - kill background tasks if it needs to reclaim memory. When and if this happens is entirely up to the system, and is probably why you see inconsistent behavior when resuming.
If the task is destroyed by the system, it will have to be re-created when you launch it again. So unless the application author has taken measures to save the application's state during the destroy process, it will be lost (and saving exact state in things like games is fairly impractical).
Another good (but also verbose) document is the one covering the Activity Lifecycle (there's a nice flowchart if you scroll down a bit).