How to check robots.txt on site?

Marc0

Well-known member
Registered
Joined
Jun 6, 2012
Messages
890
Points
28
Hi Friends..


If I would like to check whether a website has robots.txt, how would I check it?
 

psavsnacks

New member
Joined
Mar 11, 2014
Messages
4
Points
0
simple way to check robots.txt file....

type home page website url/robots.txt
 

TejasWebHosting

Member
Registered
Joined
Nov 9, 2014
Messages
15
Points
0
It is important to remember that just because you can browse the site, doesn't always mean everything is okay....

I would recommend using either Bing or Google's Webmaster Tools service to check it. They have a tool that allows you to confirm if it is visible or not with the click of a button.
 
Older Threads
Newer Threads
Replies
6
Views
3,678
Replies
16
Views
5,568
Replies
0
Views
2,173
Latest Threads
Replies
0
Views
30
Replies
1
Views
12
Replies
0
Views
108
Replies
2
Views
120
Recommended Threads
Replies
1
Views
1,976
Replies
4
Views
2,309
Replies
17
Views
7,590
Replies
19
Views
6,420

Latest Hosting OffersNew Reviews

Sponsors

Tag Cloud

You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.

Top