How to protect direct access to files

I notice that someone was trying to access some files in my asp.net website directly (such as .js .jpg .config), i.e., not through a .aspx page. 
While asp.net protects web.config file, is there a way to protect direct access to other files?
Help most appreciated.