SitePoint Sponsor

User Tag List

Results 1 to 9 of 9
  1. #1
    SitePoint Wizard maartenvr's Avatar
    Join Date
    Jun 2002
    Location
    Canada
    Posts
    1,679
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    SE spider does not see my blog

    I recently updated a website and moved it to a new server.
    For some reason, the SE spider does not see all the blog posts at www.theshrimpfarm.com/posts/

    Any idea why? It takes a couple of the links, but there are over one hundred blog posts and it is not taking them.

    I used http://www.webconfs.com/search-engin...-simulator.php and the url I entered is www.theshrimpfarm.com
    Maarten

  2. #2
    SitePoint Evangelist
    Join Date
    Nov 2007
    Posts
    472
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Normally SE spiders not considering the IP so I hope server changing will not be the issue..All SE spiders not seeing or only Google?. have to verify more on it.. so plz give more detail about the issue

  3. #3
    Mouse catcher silver trophy Stevie D's Avatar
    Join Date
    Mar 2006
    Location
    Yorkshire, UK
    Posts
    5,888
    Mentioned
    122 Post(s)
    Tagged
    1 Thread(s)
    How do you know the spiders aren't finding them? Google is returning 475 pages on your domain.

  4. #4
    SitePoint Wizard maartenvr's Avatar
    Join Date
    Jun 2002
    Location
    Canada
    Posts
    1,679
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I guess I can't be 100% sure. Those old pages are from before I changed the design and switched to a new hosting account.

    It is just that both my sitemap generator, and webconfs are not seeing all the links in the /posts/ folder
    Maarten

  5. #5
    SitePoint Member
    Join Date
    Jan 2011
    Posts
    4
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    You post has been crawled by google,How are you telling that?

  6. #6
    Mouse catcher silver trophy Stevie D's Avatar
    Join Date
    Mar 2006
    Location
    Yorkshire, UK
    Posts
    5,888
    Mentioned
    122 Post(s)
    Tagged
    1 Thread(s)
    Google is returning 51 pages from within the /posts/ folder on your site. How many should there be?

    You can check what Google has on a site (or in a particular directory on a site) with the site: operator. Type site:domain.com into the googlebar and that is a much more reliable indicator of what Google sees than some third party junk\\\\SEO tool.

  7. #7
    SitePoint Enthusiast oseymour's Avatar
    Join Date
    May 2009
    Location
    New York City
    Posts
    39
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Not because your sitemap generator and webconfs can't find them doesn't mean google can't.

    Try using http://www.xml-sitemaps.com/, I use it all the time to generate xml sitemaps and it works great.

    Quote Originally Posted by maartenvr View Post
    I guess I can't be 100% sure. Those old pages are from before I changed the design and switched to a new hosting account.

    It is just that both my sitemap generator, and webconfs are not seeing all the links in the /posts/ folder

  8. #8
    SitePoint Wizard maartenvr's Avatar
    Join Date
    Jun 2002
    Location
    Canada
    Posts
    1,679
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I use xml-sitemaps, and this program can't find these pages. I have sent a support request to them.

    Google find these pages before my redesign.... the cached page will show the old layout.'

    Thanks for the help!
    Maarten

  9. #9
    SitePoint Wizard maartenvr's Avatar
    Join Date
    Jun 2002
    Location
    Canada
    Posts
    1,679
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    OK. What I have done is eliminated my blog from xml-sitemaps. And installed a sitemap plugin in Wordpress and submitted both to Google.... seems to work
    Maarten


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •