I have tried several roms on HTC magic: Stock 1.5, Sense 1.5, CM6.1 and YoshiMod GingerBread
I always Benchmarked the phone with Quadrant for each new rom, I found out that Sense was better than plain 1.5, then CM 6.1 was faster than Sense, but the faster for me was YoshiMod with GingerBread, had no troubles with it... CM6.1 was also solid though.

Check this post for info on YoshiMod GingerBread for HTC Magic http://android.stackexchange.com/questions/12662/install-ginger-yoshi-1-4-on-htc-magic-32a-which-radio-do-i-need