What is the difference between indexed and crawling?
Two of the often used terms in the business are crawling and indexing. These two words are sometimes, if not often, interchanged with one used to represent the other. With their seemingly “interchangeable” nature, many of us may dismiss the difference and declare that both crawling and indexing share the same definition.
CRAWLING
Crawling takes place when there is a successful fetching of unique URIs which can be traced from valid links from other web pages. It’s like Pacman following all those dots and eating them, only that in the case of crawling, it’s the search engine robots that follow the links.
INDEXING
Indexing takes place after a crawled URIs are processed. Note that there may be several URIs that are crawled but there could be fewer of them whose content will be processed through indexing. The following reasons could be the causes of non-indexing of a previously crawled page:
1. A noindex directive in the page (<meta name=”robots” content=”noindex” />)
2. Duplicate content: a page that has the same content as with an indexed page may not be indexed.
|