How to use the meta robots tag

Marc0

Well-known member
Registered
Joined
Jun 6, 2012
Messages
890
Points
28
Hi there,

I disabled google and other search engines crawl and indexing my site by code in robots.txt

Code:
User-agent: *
Disallow: /
but in my html template I still put

Code:
<meta name="robots" content="index,follow" />
thus Google crawled and indexed my web pages.

just want to know How to use/control the meta robots tag for Google index or following my web pages?

Thanks in advanced!
 

Jack London

Well-known member
Joined
Jul 9, 2012
Messages
158
Points
18
Hello,

If you disable spiders crawl your web pages then you can use this robot tag:

Code:
<meta name="robots" content="index, nofollow">
Google spider will not index page, only look at page.

Code:
<meta name="robots" content="noindex, follow">
Google spider will not index/crawl pages, not look at other pages from this page.

Code:
<meta name="robots" content="noindex, nofollow">
You can switch between index/noindex and follow/nofollow depends on your requirements on web pages to control spiders crawl/index or not.

Hope it's useful to you.
 
Newer Threads
Replies
5
Views
3,426
Replies
2
Views
3,289
Replies
4
Views
4,905

Latest Hosting OffersNew Reviews

Sponsors

Tag Cloud

You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.

Top