Check the website robots file

[china-jewels]

Check the website robots file. Robots file can stop spider grab website unnecessary content, in order to reduce the spider web site to the difficulty of grasping, reasonable use of robots files can better make a web site was collected, but if meet not use robots of webmaster, may be the wrong writing files, leading to the spider can't normal on the web to grab, grab a long time less than content, spiders will think this website is cheating website, and give your site punishment. Especially beginners webmaster, in the use of robots files, don't know file was spelled correctly, can use baidu webmaster robots tool, this tool can help adsense generation robots files, Google keyword analysis tool can also detection robots document writing is correct. Because the robots document writing mistake that punishment of webmaster, need to use tools found robots mistakes, lest cause greater influence.