i have couple of pages and url which i do not want to be crawled by google crawler.
i know it can be done via robots.txt. i search google and found this way we need to arrange the whole things in robots.txt for disallow crawler but i am not sure does it right or not.
User-Agent: *
Disallow: /music?
Disallow: /widgets/radio?
Disallow: /affiliate/
Disallow: /affiliate_redirect.php
Disallow: /affiliate_sendto.php
Disallow: /affiliatelink.php
Disallow: /campaignlink.php
Disallow: /delivery.php
Disallow: /music/+noredirect/
Disallow: /user/*/library/music/
Disallow: /*/+news/*/visit
Disallow: /*/+wiki/diff
# AJAX content
Disallow: /search/autocomplete
Disallow: /template
Disallow: /ajax
Disallow: /user/*/tasteomatic
my question is
can i give the url like this way mean can i specify full url as disallow ?
Disallow: http://www.bba-reman.com/admin/feedback.htm
...
Go to the complete details ...