|
Article Home
Excluding pages from getting indexed
If there are pages or directories on your website that you do not want search engines to index you can exclude them using one of two different methods:
· Robots.txt
· META-robots code
Robots.txt
Robots.txt is a file that you place in the root of your web server. The file uses a simple syntax to exclude specific types of users - in this case search engine spiders - from parts of your website. You can either exclude specific search engine spiders or all spiders. To exclude all search engine spiders from all directories on your web server, you should write the following:
User-agent: *
Disallow: /
Note: This would disallow everything including your home page!
Learn more about how to write robots.txt files at SearchTools.com.
TIP:
We recommend that you validate your robots.txt file before uploading it. There is no way to predict how a search engine will interpret a robots.txt file with errors. You can use the free validator at Search Engine World.
META-robots
META-robots are small pieces of code you can place in the header of your HTML documents. You can use META-robots tags if you don't have access to your web server's root or if you want to exclude single pages on your website. You can read more about how to use the META-robots code at SearchTools.com.
Website
design company
 |
|
123webguru Articles
Outstanding website design
website design is no longer a choice for any business. It has become a necessity for every company to have a website design so that it can promote about ...
Need for E-commerce
The correct word for the abbreviation E-commerce could be something like Easy commerce and not electronic commerce. This is a boon of the internet technologies. ...
Selecting a Quality Web Host
"The page cannot be displayed." Have you ever encountered this message while surfing the net? Of course you have. We all have. Think it can't ...
To Be Successful One on Online
People have been running after the secret of online success and many have not yet found it, many have reached half the way. But each year Internet Marketing ...
Credibility of your website
After designing a website, a lot of strategies and planning goes into in order to make your website a success. The uphill competition in order to gain ...
Why need website design?
Nowadays, may be from a child to owner of a grocery shop owner having his own website which paused you pondering that what can be had out it. May be have ...
|
|
 |
|
 |
|
