Dev Shed Forums – Prevent to crawl some pages
Date: July 26th, 2010 01:25 AM – spinxwebdesign – Untitled Post: You can blocks search engine spider through robots.txt file. In this file you can disallow that pages whichever you don?t want to crawl. For getting more detail you can …