Dev Shed Forums – Prevent to crawl some pages

Date: July 26th, 2010 01:25 AM – spinxwebdesign – Untitled
Post: You can blocks search engine spider through robots.txt file. In this file you can disallow that pages whichever you don?t want to crawl. For getting more detail you can refer this link http://www.robotstxt.org/robotstxt.html…
Intel® Atom? Developer Program Additional revenue opportunities: Build and sell app components.

See original post by hiltonm