The other answers are correct, but fail to point out a couple differences:
Permissions
System apps are granted the ability to request certain system-only permissions that are never available to user apps. This is not the same as root/sudo/su permissions.
One example that I know of and use daily is the ability to reset the missed call notification counter. I have Go Dialer, which is a replacement for the stock dialer and contact app (how you make phone calls). When I miss a call, android lets me know that with a notification. However, Go Dialer is not allowed to reset this counter, even though it asks for that permission when you install the app. Android 2.2 and earlier let user apps reset it, but made this permission a system-only accessible permission for android 2.3
By moving my Go Dialer app to the system partition, it is allowed to reset this counter. Root is not required for the app (except, of course, to actually move the app to the System partition).
Another system permission is the ability to set the current gps location. Now, Android does have a preference where you can allow mock locations, so any gps spoofing app you install can declare a gps location. But if the app is in the System partition, mock locations can be disabled while still letting the app spoof the gps location.
Updates
System apps can be updated just like user apps, but the update is never integrated into the original rom. That is, system apps have the unique ability to roll back to the version that existed when the rom was first installed.
Move to SD
Android lets users move user
apps from internal memory to the sd card. System apps cannot be moved and always take up some internal space. Most (all?) devices, though, have internal user apps on a different partition, so uninstalling a system app would not give you any more user app space (other than the dalvik-cache being removed).
You will recover some space if you uninstall any updates to a system app. As noted, the updates are not integrated with the rom and are thus stored in user space. The updates are not allowed to move to the sd card either, but if removed, user space is recovered.
How it affects you
If you are a casual android user, you likely won't run into scenarios where system vs user app would make a difference. If you are an advanced user, you likely have root and can thus change a system app to user (and vice versa) if you ever run into one of these situations.