SitePoint Sponsor

User Tag List

Results 1 to 3 of 3
  1. #1
    SitePoint Enthusiast
    Join Date
    Mar 2005
    Location
    Orange County, CA, Unites States
    Posts
    95
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Why aren't my products on this page getting spidered?

    I own a store (in my signature) and I had an idea about a month ago. From time to time we discontinue products or the manufacturer decides to discontinue that product. We then remove the item from our site. When I say remove, I mean if you visit our site and try searching for the discontinued product you will not be able to find it. BUT... if a SE still has the link in it's database you can search for the product, click through from the search engine, and then view it on our site.

    Basically... you can't find the product if you look for it on our site... but if you click through from a search engine you can still see the product page. The page still exists and can be viewed... you just can't find it by searching on our site. Make sense?

    Anyways... after some time the SE removes the link to our site for the discontinued product because it can no longer find it on our site. This is bad though... as we lose incoming traffic. Sure... we can't sell that product any longer... but we want to change those product pages to point the visitor to other, similar products. Still with me?

    What I did was included a link at the bottom of every page that leads to a "Discontinued Product Page." This page displays links to every product in our database which is marked as "Discontinued." These products won't show up in regular search results on our site... but you can still view each page from our "Discontinued Product Page." You can view the page here:
    http://www.supplements101.com/store/dindex.asp

    My question is... why aren't SE's spidering the links from this page? Both Google and MSN have spidered this page "http://www.supplements101.com/store/dindex.asp," but neither SE has followed any of the links on that page. For example... if you run a search on google for one of the products I have listed on the discontinued page... this shows:
    http://www.google.com/search?hl=en&l...s+&btnG=Search

    Notice how it only shows the dindex.asp page? Why isn't it following that product link and spidering the product page (which would yield the same SE rankings we had before)?

  2. #2
    He's No Good To Me Dead silver trophybronze trophy stymiee's Avatar
    Join Date
    Feb 2003
    Location
    Slave I
    Posts
    23,426
    Mentioned
    2 Post(s)
    Tagged
    1 Thread(s)
    For starters that page has a ton of links on it and a PR of zero. That certainly doesn't encourage Google to crawl any deeper.

  3. #3
    SitePoint Enthusiast
    Join Date
    Mar 2005
    Location
    Orange County, CA, Unites States
    Posts
    95
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    So what would be my best move? Every page on my site points to that page. Should I submit that page to bunch of directories? Or will nothing I do get those links visited because there are too many for one page?


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •