I know that AndroidCommander works but its site is offline so I can only download version for windoze.
I ALSO want to do this if possible the other way round too (access PC files in Android not only Android files from PC)
How else can I do this?
ES File Explorer has this feature.
On the side Navigation bar you have the options that includes Favorites, Local, Library, and Networks. Expand the Network tab and click LAN. You'll see your LAN resources the same way you see them in Windows.
Click on the computer/share you want to browse. You'll have the option to see the items as icons or detailed listings.
Browsing Android Files from the Windows PC
To browse the items on the android from the PC, run an FTP server app on the android (There are many such as: FTP Server Ultimate).
On Windows Explorer you can type the full ftp link server name in the Explorer address bar to browse the files and folders just like the other local files and folders on Windows.
You can also type the command "explorer [ftp link]" in Windows 7 Search field to bring up the folders.
Ftp Link Example:
The numbers after the colon is the port. 2121 is the default port used by most of the android ftp server apps.