Note: I have tried to make below a bit more generic than your specific Mac OS option, but the Mac is covered in the below answer, but this may help other Nexus 5 users. Full credit to the OP on XDA.
If you want to restore your Nexus 5 back to stock, you can flash a Factory Image. Note: This process will wipe your phone completely. Backup your storage before you continue. There are 2 methods, below I've given the "easy" method.
For this process, you will need to:
Install adb and fastboot drivers. To do this, use this program:
For MAC users:
*Note: MAC users will have to write "./" before every command in this thread. For example :
./fastboot flash recovery recovery.img
Linux users
http://104.imagebam.com/download/VxZG49wcq59qXJqsRAPIng/28626/286255946/Untitled.png
- Unlock your bootloader (In case you haven't already)
Easy Method:
Power off your phone.
Boot it in fastboot by pressing and holding Power Button + Volume Down at the same time.
Connect your phone to your PC through your USB cable.
Execute the flash-all.bat script (for Windows) or flash-all.sh(for MAC and LINUX) in the folder where you extracted the factory images.
!! For MAC: You'll have to edit the flash-all.sh, and add "./" before every fastboot command.
Wait for everything to flash.
After everything finished, select "Recovery" using the volume buttons.
When a small dead Android appears, press Power Button + Volume UP.
Select "Wipe data/factory data reset"
Reboot. Your phone should be stock.
If you want to Unroot completely:
The Nexus 5 has a thing implemented in the bootloader called a tamper flag. This shows if the bootloader was ever unlocked, kind of like the flash counter with Samsung devices. You can check it out by using this command
fastboot oem device-info
. If the tamper flag value is "true", then you'll have to reset it. The following steps will reset this flag, and lock your bootloader.
So, first, go here (Thanks @osm0sis for the wonderful script)
Under "Flashable Zips", you will find the Nexus BootUnlocker script. Download it.
Put the zip on your internal storage.
Download the latest TWRP recovery image for the Nexus 5 from here. Rename it to TWRP.img.
Now boot it. You do this by booting your phone in fastboot, and typing the following command:
fastboot boot path/to/TWRP.img
Replace path/to/TWRP.img with the path to where you placed the TWRP.img. For example: C:/Users/X/Desktop/TWRP.img
TWRP recovery should boot.
When TWRP finished booting, select "Install", then navigate to the Nexus Bootunlocker zip, select and install it. After it finished, use the back arrow at the bottom of the screen to go back, and reboot.
Now your bootloader should be locked, and your tamper flag should show "false".