SitePoint Sponsor

User Tag List

Results 1 to 22 of 22
  1. #1
    SitePoint Member
    Join Date
    May 2011
    Location
    India
    Posts
    6
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Smile Is Google crawl/index blogs faster than websites?

    Lots of Newbies asks Why does Google crawl/index blogs faster than a normal site??

  2. #2
    Mouse catcher silver trophy Stevie D's Avatar
    Join Date
    Mar 2006
    Location
    Yorkshire, UK
    Posts
    5,881
    Mentioned
    122 Post(s)
    Tagged
    1 Thread(s)
    Google crawls sites faster when it thinks they are more important/popular, and when they have regular new content. Sites that are rarely updated won't be crawled as much, and sites that Google doesn't think much of won't be crawled as frequently either.

  3. #3
    SitePoint Addict
    Join Date
    Dec 2000
    Location
    Orange County, Ca
    Posts
    288
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Just from my experience I have seen Wordpress sites get crawled and indexed very fast. It is just anecdotal evidence, but for some reason they seem to love the Wordpress blog platform, perhaps it is the pinging service that helps.

    It might seem faster because blogs typically add one new page at a time, so the search engines do not have to crawl lots of new pages, but I have personally seen a new blog post published, crawled, indexed and ranking within 30 seconds. That might not be the case all the time, but it was pretty impressive when I did see it happen.
    Wordpress Auto Installer
    www.webproco.com

  4. #4
    SitePoint Member
    Join Date
    Jun 2011
    Location
    India
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    it could be due to the pinging service of wordpress rss.

  5. #5
    SitePoint Enthusiast
    Join Date
    Jun 2009
    Posts
    90
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Stevie D View Post
    Google crawls sites faster when it thinks they are more important/popular, and when they have regular new content. Sites that are rarely updated won't be crawled as much, and sites that Google doesn't think much of won't be crawled as frequently either.
    Completely agree. Every minute hundreds of people publish a new blog entry on blogsites like blogspot and wordpress so they have fastest crawl rate.. when you publish a post on these sites then you can see the next second Google crawl your post.

  6. #6
    Community Advisor ULTiMATE's Avatar
    Join Date
    Aug 2003
    Location
    Bristol, United Kingdom
    Posts
    2,160
    Mentioned
    46 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by revium View Post
    Just from my experience I have seen Wordpress sites get crawled and indexed very fast. It is just anecdotal evidence, but for some reason they seem to love the Wordpress blog platform, perhaps it is the pinging service that helps.
    As Stevie D has stated, Google will crawl a site if it has regular, potentially important content. A blog would fall into the category of a site being updated regularly with important information.

    I can say with some certainty that the script used to implement the blog will probably mean nothing to Google.

  7. #7
    Non-Member
    Join Date
    Dec 2010
    Posts
    65
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Yeah Google indexes blog faster than sites. Because in blogs you have ping service, you do a ping a lot of the times Google will come and crawl you. But in site you have to submit site maps and Because site map submission does not guarantee that Google will crawl the URLs on that list.

    So don't waste your time to wait for Google to come to you and index and crawl your site, instead of this do some link building tactics like article submission and social bookmarking to get indexing faster and increase your SERP.
    Last edited by Ted S; Jun 26, 2011 at 12:15. Reason: added spacing to make the post legible

  8. #8
    SitePoint Member
    Join Date
    Jun 2011
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    It really depends on how frequently you are upgrading your website content and how well your website is programmed. Blogs like wordpress are programmed for each of use and browsing and crawlers loves visiting those site.

    However, you must create and submit xml sitemap to Google and other search engines for crawlers to assist in crawling your website.

  9. #9
    Community Advisor ULTiMATE's Avatar
    Join Date
    Aug 2003
    Location
    Bristol, United Kingdom
    Posts
    2,160
    Mentioned
    46 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by piyush.dcs View Post
    It really depends on how frequently you are upgrading your website content and how well your website is programmed. Blogs like wordpress are programmed for each of use and browsing and crawlers loves visiting those site.
    This is 100% wrong. Google couldn't care less about how your site is developed. WordPress will be crawled at exactly the same rate as any other page of the same functionality. If you replaced WordPress with static files they'd be crawled at exactly the same pace.

  10. #10
    SitePoint Addict
    Join Date
    Dec 2000
    Location
    Orange County, Ca
    Posts
    288
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    It is true that Google does not care about what platform your site is published on, but the rate at which your site is crawled is roughly dependent on your page rank. So the higher your page rank the more likely your site will be crawled faster.

    Below is an interview from Matt Cutts from about a year and a half ago where he touches on the relationship between page rank and crawl rate.
    Matt Cutts: “Google Crawls According To PageRank”. A Must See Video For Bloggers
    Wordpress Auto Installer
    www.webproco.com

  11. #11
    JustAFanOfSlash mdamin76's Avatar
    Join Date
    Dec 2008
    Location
    Stage
    Posts
    434
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Google crawl blogs faster because of their automatic pinging system, but originally google is looking for fresh contents.

  12. #12
    Non-Member Jeff Collision's Avatar
    Join Date
    Oct 2010
    Posts
    225
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Normally whenever the blog is updated we regularly do ping blog service to specify different search engine that your blog has updated. So if you do ping a lot of time Google will come and crawl you but often its Google blog search

  13. #13
    Non-Member
    Join Date
    Jan 2011
    Posts
    18
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Generally Blogs are Updated Frequently comparable to Websites!! Another Reason could be CMS!! Wordpress based Blogs are indexed Faster as compared to other CMS based Blogs!!

  14. #14
    SQL Consultant gold trophysilver trophybronze trophy
    r937's Avatar
    Join Date
    Jul 2002
    Location
    Toronto, Canada
    Posts
    39,215
    Mentioned
    58 Post(s)
    Tagged
    3 Thread(s)
    Quote Originally Posted by mikest4u View Post
    Wordpress based Blogs are indexed Faster as compared to other CMS based Blogs!!
    o rly?

    faster than http://blogger.com blogs, which google owns ???



    rudy.ca | @rudydotca
    Buy my SitePoint book: Simply SQL
    "giving out my real stuffs"

  15. #15
    SitePoint Member
    Join Date
    Jun 2011
    Location
    India
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Yes i do have seen that blog posts get indexed faster than that of web pages itself. I will not give any credit to any automated pinging system. The reason is, blog publishing platforms are also used for building websites like sites in wordpress. In such cases do Google indexes those sites faster than others. Obviously not.

    I think the main logic is categorization of websites. When google bot finds a blog it categorize in a different way rather than general websites. These blog categories are crawled more frequently as blog can provide more fresh content than website itself.

  16. #16
    SitePoint Enthusiast
    Join Date
    Jun 2011
    Posts
    41
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I feel that blogs, specifically blogspot blog are fastly crawled as it is stored in google db and all of its template has standard structure for them to crawl easily. Also, other blogs has some standard template which make google crawler feel easy to crawl.
    But websites needn't have any definite structure and that might be the problem

  17. #17
    SitePoint Member
    Join Date
    Jul 2011
    Posts
    9
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Yeah. Blogs in blogger get indexing very fast. And in certain cases they get indexed even if they don't have any content at all. It happened with me 2-3 times. lol.. But certainly if you make a good site and even have only 1 article and go and make just 1 backlink from a high pr site you will get indexed pretty fast.

  18. #18
    Non-Member Johnletton's Avatar
    Join Date
    Jun 2011
    Location
    California, USA
    Posts
    38
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Stevie D View Post
    Google crawls sites faster when it thinks they are more important/popular, and when they have regular new content. Sites that are rarely updated won't be crawled as much, and sites that Google doesn't think much of won't be crawled as frequently either.
    I agree with you so it is said place the your website's link where those who have high index rate because when those websites get indexed then definitely your website is gonna index early.

  19. #19
    SitePoint Member
    Join Date
    Jun 2011
    Posts
    7
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    yah Google crawler read free blogs first, i mean word perss and blogger blogs after that it read domain blog.

  20. #20
    SQL Consultant gold trophysilver trophybronze trophy
    r937's Avatar
    Join Date
    Jul 2002
    Location
    Toronto, Canada
    Posts
    39,215
    Mentioned
    58 Post(s)
    Tagged
    3 Thread(s)
    wow, we sure have a wide range of (sometimes contradictory) opinions in this thread

    i think i'm gonna wait for an honest seo person (if that's not an oxymoron) to provide factual data on this topic
    rudy.ca | @rudydotca
    Buy my SitePoint book: Simply SQL
    "giving out my real stuffs"

  21. #21
    Non-Member
    Join Date
    Apr 2011
    Posts
    47
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Yes it do cause Google is always in search on new content and blogs are always with some posts but if your website is new with fresh content then within a weeks you will see your self indexed

  22. #22
    SitePoint Enthusiast zelthost's Avatar
    Join Date
    Dec 2009
    Location
    USA
    Posts
    37
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Yes it's right that, google always tries to maximize the quality, the relevance and the accuracy of our index, and we want to make a distinction between crawling and indexing because site map submission does not guarantee that Google will crawl the URLs on that list, it is very helpful to help them discover new URLs or to make canonicalization decisions

    But crawling and indexing is different, so if we do a ping a lot of time Google will come and crawl we but often its Google blog search because if we are doing those WordPress or web blogs or FeedBurner pings those pings are often those sort of things that are blogs. So the blog search can come and crawl we five minutes later. But then if you show up, you might show up in the blog search corpus not in the main web index corpus. So just because we get crawled it doesn't mean that we are getting any sort of index boost or anything like that. Google sort of try to rationally decide what is the best quality of data, how do they get that, sometimes its crawling stuff immediately like with blog search you have very fast, very real time sort of results.

    Thank You


Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •