If the remote phone is off, or if it does not have network access, you won't be able to control it remotely. If the SIM card is not removed, you could control it via SMS if you set up an app to do certain security functions (with apps like AVG Antivirus). However, remote control from SMS is limited and you need to set it up in advance.
If you did not set up such security measures, there is no way to remotely access your device without network access. Since Android Device Manager doesn't work, we can conclude that the phone has no network access, or it is off.
At this point, although you cannot control your device remotely, you might be able to find out it's last reported location from Google's Location Timeline. This would be able to help you locate it if you did not turn off location history on your device.
If you do locate your device, please do not attempt to confront any suspect by yourself.
From Lookout Mobile Security:
If your cell phone is lost or stolen and you don’t have a mobile security app, the first thing to do is contact your network provider, who will be able to block your phone in order to stop anyone else from using it.
This is particularly important if you have a pay monthly contract, as you will be liable for any calls made (or expensive apps downloaded) before you report your phone stolen.
Most of the major US network providers allow you to suspend your service and request a new SIM online or by calling their customer service department.
If your cell phone has been stolen it's also important to notify the police, as insurance providers will usually need a crime reference number in order to process any claims.
If you use your smartphone to shop or bank, you may also need a police report to dispute any fraudulent charges made on your debit or credit card accounts using the stolen device.
Make a report at your local station, being sure to give them your device's International Mobile Equipment Identity (IMEI) number, which your network should be able to provide you with. This could help the police get your phone back to you if it were to be recovered.