Go to Settings→Network, tap WiFi, and watch out for your WiFi AP here. Tap it, select to edit it. Toggle the "advanced settings" to be shown.
By default, WiFi is configured via DHCP -- but for some reason unknown to me (and escaping my understanding), Android doesn't pick the name servers via DHCP, and instead sticks to Google's hardcoded ones (e.g.
126.96.36.199). So in this place, you need to switch to a static IP, set the default gateway correspondingly, and put your own name server's address (most likely again your router's IP) into the field for DNS1.
Save the changes, and you should be able to use your local hostnames. You could even try switching back to DHCP, and see whether it keeps your DNS1 settings.
To calm down upcoming worries: This only affects the WiFi AP you edited, no system-wide changes for all WiFi APs. In the worst case, you always can delete the AP and re-create it.