![]() |
#1 |
Registered User
Join Date: Jan 2017
Location: New Delhi
Posts: 24
|
Is it necessary to use Robots.txt file?
Hello friends- Is it necessary to use Robots.txt file? It can be affect on website ranking in search engine.
|
![]() |
![]() |
![]() |
#2 |
Registered User
Join Date: Feb 2018
Posts: 27
|
Yes, absolutely. Because Robosts.txt generator tells the search engines which pages to access and index on your website and which pages not to.
|
![]() |
![]() |
![]() |
#3 |
Registered User
Join Date: Jul 2017
Posts: 2,283
|
Warning: Don't use a robots. txt file as a means to hide your web pages from Google search results. If other pages point to your page with descriptive text, Google could still index the URL without visiting the page.
__________________
Villa Plots in Chennai Plots Sale in Kelambakkam Premium Villa Plots in Kelambakkam fastest selling villa plots in chennai Villa Plots in Kelambakkam OMR Residential Plots for Sale in OMR Premium Villa Plots in OMR Villa Plots in Kelambakkam Villa Plots Sale in Kelambakkam Plots Sale in Kelambakkam Residential Plots in Kelambakkam Land for Sale Near Chennai Plots for Sale in Chennai Premium Villa Plots OMR luxury villa plots in ECR Chennai |
![]() |
![]() |
![]() |
#4 |
Registered User
Join Date: Jun 2018
Posts: 508
|
Robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl & index pages on their website.
|
![]() |
![]() |
![]() |
#5 |
Registered User
Join Date: Aug 2014
Location: India
Posts: 816
|
This depends on us because, through robot.txt files, we can tell Google which website url you need to crawl and which page you need not to crawl.
__________________
It Consulting Services | Enterprise Applications | Business Solutions | Iot Applications | Iot | It Consulting |
![]() |
![]() |
![]() |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
The robots.txt file | rubberfender | Yahoo | 0 | 12-02-2012 07:32 PM |
Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |