Because content loaded from a web service via AJAX is not visible to search engines, I am looking for a way to detect a spider(google) and if a spider is detected, skip the method call in the web service. I don't want to waste all the resources (database calls mainly) that are used up when a search engine spider come crawling through the site.
Anyone have any ideas on this?
...
Go to the complete details ...