How to Get More Natural Search Traffic With robots.txt
Increasing your link popularity is only part of the battle when it comes to increasing your search engine rankings.
There are some simple ways to improve these rankings and the robots.txt file is one of them.
What is robots.txt?
The robots exclusion standard, also known as the Robots Exclusion Protocol or robots.txt protocol is a convention to prevent cooperating web spiders and other web robots from accessing all or part of a website which is, otherwise, publicly viewable.
With this blog I created a robots.txt file so I could remove any junk pages and duplicate content from the search engines.
I have removed :
* Trackback URLs from being indexed because it was causing blank pages to be indexed.
* Comment and post feeds from the search results so that I would not have duplicate text indexed.
* Dated archives.
* Other elements for a better security.
Although I’ve made these changes a few days ago I already see a natural search traffic grow.
You can see my robots.txt file here.
Update: Sante wrote another great article about robots.txt on sitepronews that you must read.
Download Episode
Share and enjoy
Bypass 72 Minutes Limitation
