SitePoint Sponsor

User Tag List

Results 1 to 3 of 3

Hybrid View

  1. #1
    SitePoint Member mrbetabm's Avatar
    Join Date
    Jul 2012
    Location
    Dhaka
    Posts
    3
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    How can I create a robot.txt file using google?

    I have a new website. I used wordpress platform. Now I want to use robot.txt file for avoid some page google crawling. Is it possible? Can anyone help me?

  2. #2
    Gre aus'm Pott gold trophysilver trophybronze trophy
    Pullo's Avatar
    Join Date
    Jun 2007
    Location
    Germany
    Posts
    6,097
    Mentioned
    219 Post(s)
    Tagged
    12 Thread(s)
    Hi there,

    It is indeed possible.
    Go to your WP dashboard -> Settings -> Privacy, there you have the option "Ask search engines not to index this site".
    As the name suggests, this will ask Google and co. not to index your (entire) site.

    If you want to block search machines only from indexing certain pages, you will have to create a "robots.txt" file and edit it by hand.
    This isn't very difficult however, so if you need a hand, just let us know.

  3. #3
    SitePoint Member
    Join Date
    Dec 2012
    Location
    Malaysia
    Posts
    16
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    by default WP has a virtual robot file

    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/


Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •