I usually use simple submit to submit to most search engines when a site is done, then I'll go submit to lesser known ones by hand so to speak. Its never taken more than a day or 2 to get in any search engine listing for me until just recently. Its been 2 weeks and the only 2 listings I've got are voila, which is a french search engine, and yep.com No altavista no lycos no hotbot. I dont get what the problem is. And its not that my site isn't showing up in my searches, its not in their directory. Has anyone else experienced anything like this recently?
One of two things happened (with #1 being the most likely)
1) The search engines are going through a list of URLs to add and haven't been to your site yet.
2) You have submitted it too many times or you have performed actions against the engine's terms of indexing.
Who the heck is Gen. Failure and why does he want to read my hard drive?
Soon to come: Aspology.com
Well its not the latter I'm sure. But even so two weeks is a long time if it is the former.
It can be up to 6 weeks with some of the bigger search engines, I too have the same problem at times. Just wait, they will index it eventually http://www.SitePoint.com/forums/smile.gif
Methven Online - www.methven.org.uk
HotBot site submission ranges anywhere between three weeks and 60 days.
From Open Directory:
Once your site has been accepted into the Open Directory, it may take anywhere from 2 weeks to several months for your site to be listed on partner sites which use the Open Directory data, such as AOL Search, AltaVista, HotBot, Google, Lycos, Netscape Search, etc. We make updates of the data available weekly, but each partner has their own update schedule.
One more great search engine is Snap.com
Your site will be listed in 24 hours after submissions. They call it "Live directory or whatever." You just have to be a member to use this service. It's very easy to signup and it's free. It's like signing up for free email. Go check it out!
I was wondering about the dmoz partner sites. How do they run their engines ?
Do they download all the links from dmoz every so often ? Or do they just rip all the new links only ? How do they do that ?
On an unrelated issue. I was wondering, has anyone heard of "Code classification" for products and services ? Its supposed to be the next 'thing' in searching for suppliers. All products are to be classified for easy b2b transactions.
I was wondering if there were any easy method to create these codes, rather than with manual labour (which is what one organisation is doing).
It depends on the partner site..
The larger ones download the entire index on a regular basis, while smaller custom sites will just rip it on the fly.
Check out www.digitalmagician.com/projects/webography to see an example of ripping the data on the fly.
Wayne Luke - Sitepoint Moderator/Internet Media Developer
Digital Magician Studios - Making Magic with Web Applications