The description of the app, Battery Calibration, claims that after changing ROMs you should recalibrate your battery gauge. It has good reviews so it seems like the general concept is sound and trying the app at least won't cause harm.
What I'm wondering is what the battery statistics have to do with the ROM. My understanding is that the batterystats file contains a mapping of charge values in mV to %, so why does it need to be regenerated if a new ROM is flashed?