How to use the meta robots tag

Marc0 · Dec 24, 2013

Hi there,

I disabled google and other search engines crawl and indexing my site by code in robots.txt

Code:

User-agent: *
Disallow: /

but in my html template I still put

Code:

<meta name="robots" content="index,follow" />

thus Google crawled and indexed my web pages.

just want to know How to use/control the meta robots tag for Google index or following my web pages?

Thanks in advanced!

Jack London · Dec 24, 2013

Hello,

If you disable spiders crawl your web pages then you can use this robot tag:

Code:

<meta name="robots" content="index, nofollow">

Google spider will not index page, only look at page.

Code:

<meta name="robots" content="noindex, follow">

Google spider will not index/crawl pages, not look at other pages from this page.

Code:

<meta name="robots" content="noindex, nofollow">

You can switch between index/noindex and follow/nofollow depends on your requirements on web pages to control spiders crawl/index or not.

Hope it's useful to you.

How to use the meta robots tag

Marc0

Well-known member

Jack London

Well-known member

Latest posts New threads

Latest posts

New threads

Latest Hosting Offers New Reviews

Latest hosting offers

Popular Contributors - Past 30 Days

Sponsors

Tag Cloud