How to check robots.txt on site?

Marc0

Well-known member
Registered
Joined
Jun 6, 2012
Messages
890
Points
28
Hi Friends..


If I would like to check whether a website has robots.txt, how would I check it?
 

psavsnacks

New member
Joined
Mar 11, 2014
Messages
4
Points
0
simple way to check robots.txt file....

type home page website url/robots.txt
 

TejasWebHosting

Member
Registered
Joined
Nov 9, 2014
Messages
15
Points
0
It is important to remember that just because you can browse the site, doesn't always mean everything is okay....

I would recommend using either Bing or Google's Webmaster Tools service to check it. They have a tool that allows you to confirm if it is visible or not with the click of a button.
 
Older Threads
Newer Threads
Replies
6
Views
3,761
Replies
16
Views
5,723
Replies
0
Views
2,253
Latest Threads
Replies
0
Views
130
Replies
0
Views
66
Replies
0
Views
60
Replies
1
Views
90

Latest postsNew threads

Latest Hosting OffersNew Reviews

Sponsors

Tag Cloud

You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.

Top