I know I can install my application in the
/system/app folder with root access. But is there any way to do it without root access?
Users are not allowed to write to the /system folder/partition. Any sort of exploit or method that would allow this partition to be writable is equivalent to root access on the device.
Some other examples besides root are: "Rooted recovery" which is a recovery partition that has more features than the standard Android recovery; an "Engineering" (HTC) or Unlocked (everyone else) bootloader; or a file-permissions exploit, which at that point you might as well put
SuperUser.apk on the phone anyway...