Robots.txt
Robots.txt file does not improve your search engine positioning.
It provides robots with information concerning which files you will not allow to be crawled and indexed in the search engines.
When the search engine robot crawls your site it looks for the robots.txt file.
If it doesn't find one it assumes automatically that it may crawl and index the entire site.
This allows all robots to crawl all files.
User-agent: *
Disallow:
This Disallows all robots to crawl a folder called /cmsbuffet/ .
User-agent: *
Disallow: /cmsbuffet/
This web site validates your Robots.txt
http://tool.motoricerca.info/robots-checker.phtml
Here is the latest explanation from Matt Cutts:
Recently Launched Websites
K9-Toronto.Com
K9 Toronto brings the dog salon to Toronto dog owners, so your dog can get regular dog grooming in Toronto without the hassle. K9 Toronto professional dog grooming and dog walking services in Toronto take the stress out of these basic needs for both you and your pets.
Contact Us:
Phone: 647-477-2992Email: info @ CmsBuffet.Com
Helping You Achieve Web Visibility.


