I have Nexus 5 with Lolipop 5.0.1. 
I set an alarm to get up in the morning for Office. The alarm did went off on time but there was nothing on the screen. The screen was off. I had set up my volume buttons to dismiss the alarm but that didn't worked either. I had to unlock my phone, open Clock, tap on Alarm and turn off alarm from there. 

Please help.

Cheers,
Deepak