What is robot.txt file?

What is the main role of robot.txt file in SEO …

In short, adding this (in the route directory) instructs robots/spiders not to crawl and index certain files.

This topic has been covered in other threads.

Please Search using the vBulletin Advanced Search and SitePoint’s [URL=“http://search.sitepoint.com/”]Advanced Search