You can downgrade to kitkat by flashing stock kitkat firmware using Odin (the samsung flash tool).
First, download Odin. Then download framework corresponding to your region from here
Install samsung drivers.
Next run odin on your pc and connect your phone while its in download mode (First go into download mode and then connect it. You go into download mode by turning off phone and then pressing and holding Power Key + Home + Volume Down
till it gets into download mode. That "Downloading... Do not turn off target !!").
Make sure Odin recognizes you phone, it should say something like "com..." in one of the boxes which would mean odin has detected your phone. If it doesnt, your drivers arent proper.
Next extract the firmware you downloaded earlier using winrar.
Then in Odin,
a) Click the PDA button and choose the file with .tar.md5 extension.
b) Click the phone button and choose the file with modem in its name (skip this step if file not found).
c) Click the CSC button and select the file with CSC in its name.
d) Click the PIT button and select the .pit file (skip this step if file not found).
Make sure that the "Auto Reboot" and "F.Reset Time" check boxes are selected in Odin. At the same time, uncheck/disable the "Re-Partition" option. Click it only if a .pit file is being used.
Then click "start" button and wait a while. It may take sometime. After process is done odin should show "success" and then you can restart phone (may take sometime at first boot. Wait atleast upto 20mins by watch). You will be back to stock android KitKat rom.
Hope this helps :)