I use this as my robots.txt file. is it right?

Thread starter Marc0
Start date Mar 7, 2013

Marc0

Well-known member

Registered

#1

I use this for my robot.txt
example:

User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Sitemap: http://www.domain.com/sitemap.xml.gz

as my wordpress site's robots.txt.

is it ok?

D

Deepak Kaushik

New member

#2

Yes, This is correct. Just a little bit you have to change in this coding in XML site URL remove ".gz" which is in last of the link.

W

WHUK

Member

Registered

#3

Nope, including the sitemap with .gz extension is correct. It's basically a compression used by the servers which is understandable by Google too. So shouldn't matter even if you remove it.

You must log in or register to reply here.

Share:

Facebook Twitter Reddit Pinterest Tumblr WhatsApp Email Link

Older Threads

How to check robots.txt on site?

Replies: 3

Views: 3,754

Dec 7, 2014

TejasWebHosting

What is email marketing? What are the types of email marketing?

Replies: 4

Views: 3,181

Apr 2, 2013

tayloralina

what is best tool create a logo

Marc0
Mar 7, 2013
Graphic Design

Replies: 12

Views: 5,463

Feb 23, 2014

Knightmedia

How we create a div in middile of any screen ?

Marc0
Mar 7, 2013
Website Design

Replies: 0

Views: 2,013

Mar 7, 2013

Marc0

how to create paging on php

Marc0
Mar 7, 2013
Web Programming

Replies: 1

Views: 2,595

Mar 7, 2013

Holly Nicole

Newer Threads

Facebook Advertisement?

Maxwell
Mar 7, 2013
Social Media

Replies: 6

Views: 3,775

Aug 16, 2013

Rufus Conn

What is 500 Internal Server Error ?

Replies: 8

Views: 5,325

Nov 11, 2016

24x7CSM

Difference in Static and Dynamic?

Maxwell
Mar 7, 2013
Website Design

Replies: 16

Views: 5,742

Oct 15, 2013

ChristopherBGregory

M-commerce Successful in India?

Marc0
Mar 7, 2013
Mobile Marketing

Replies: 0

Views: 2,262

Mar 7, 2013

Marc0

What type of website layout do you like?

Replies: 12

Views: 5,767

Oct 15, 2013

ChristopherBGregory

Latest Threads

R

How much bandwidth is enough for a growing website?

Ridoy
Thursday at 12:47 PM
Web Hosting Tutorials

Replies: 1

Views: 24

Today at 1:39 AM

FerdieQO

J

How does Jtti Server Defend against DDoS attacks?

jtti
Thursday at 6:53 AM
Web Hosting

Replies: 0

Views: 179

Thursday at 6:53 AM

jtti

J

What are the performance differences between DNS load Balancing and reverse proxy?

jtti
Wednesday at 7:20 AM
Web Hosting

Replies: 1

Views: 157

Today at 1:41 AM

FerdieQO

What Features Do You Value Most in a Hosting Provider?

hostiko
Wednesday at 5:36 AM
Web Hosting Reviews

Replies: 1

Views: 39

Today at 1:43 AM

FerdieQO

Free vs. Paid Hosting: Is It Worth It?

GreenhostCloud
Tuesday at 12:00 PM
Web Hosting

Replies: 2

Views: 81

Today at 1:35 AM

FerdieQO

Recommended Threads

Hello from Nidhish!

Replies: 6

Views: 2,348

May 14, 2015

brandonolguin

Hi from orangesweety

Replies: 7

Views: 2,947

Jun 5, 2015

orangesweety

Free or Premium DNS

JOED77
Aug 5, 2017
Domain Names

Replies: 0

Views: 1,582

Aug 5, 2017

JOED77

WTS 200 Big Pinterest Board Invites $99

Hawker
Feb 7, 2016
Services

Replies: 4

Views: 2,890

Jan 12, 2021

smmgoal

HC SSD VPS: 1 GB RAM - 10 GB SSD Storage - 1 CPU Core - $15/mo

Replies: 0

Views: 2,022

Apr 4, 2017

HostColor

Similar Threads

Crawl delay in robots.txt?

Replies: 5

Views: 2,859

Nov 13, 2018

royceD

The difference between Robots.txt and Robot Meta Tag

Replies: 1

Views: 1,846

Mar 19, 2015

elcidofaguy

How to set up a robots.txt to control Google spider crawl my site?

Replies: 2

Views: 2,357

Jan 27, 2015

Hassan

How to check robots.txt on site?

Replies: 3

Views: 3,754

Dec 7, 2014

TejasWebHosting

M

How to edit Robots.txt in blogger?

Replies: 2

Views: 2,815

Mar 29, 2013

zebrathemes

Latest posts
- Hello from Apon!
  
  Latest: David Beroff
  
  Today at 1:51 AM
  
  New Member Introductions
- HELLO ALL MEMBER
  
  Latest: David Beroff
  
  Today at 1:50 AM
  
  New Member Introductions
- Need Redesign My Website?
  
  Latest: David Beroff
  
  Today at 1:49 AM
  
  Website Design
New threads
- R
  
  How much bandwidth is enough for a growing website?
  
  Started by Ridoy
  
  Thursday at 12:47 PM
  
  Replies: 1
  
  Web Hosting Tutorials
- J
  
  How does Jtti Server Defend against DDoS attacks?
  
  Started by jtti
  
  Thursday at 6:53 AM
  
  Replies: 2
  
  Web Hosting
- J
  
  What are the performance differences between DNS load Balancing and reverse proxy?
  
  Started by jtti
  
  Wednesday at 7:20 AM
  
  Replies: 2
  
  Web Hosting

Latest hosting offers
- Save an incredible 90% off! The Best HostPapa Deal so Far: Unlimited SSL, 50 GB NVMe, 24/7 support
  Essentials Hosting provides reliable and affordable web hosting solutions!
  
  cancan
  
  Updated: Nov 20, 2024
- Affordable Dedicated Servers - Raspberry Pi & Intel N95 | From €12.90/mo - Unlimited Traffic.
  Affordable Raspberry Pi & Intel N95 Dedicated Servers with SSD Storage and Unlimited Traffic.
  
  Tinyservers.eu
  
  Updated: Nov 4, 2024
- SystemInPlace - Low cost Dual E5 384GB/512GB RAM DEDI. 30% OFF FIRST MONTH! 1gb/2.5gb/5gb/10gb
  Affordable enterprise grade dedicated servers at a price you can afford.
  
  systeminplace
  
  Updated: Oct 15, 2024

Popular Contributors - Past 30 Days

12
S
11
9
J
8
R
8
H
7
6
6
S
5
5
5
B
4
4
4
J
3

Sponsors

Tag Cloud

Top