Sitemap for over 10M pages

Hello,

what sitemap tool you would recommend me for site with over 10M pages. I think php generator tool couldn’t handle such quantity of pages. Tried to use xml-sitemaps.com tool with correct settings, but it’s never ends, getting such errors:

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 13681503 bytes) in /home/admin/sites/site.com/generator/pages/class.utils.inc.php(2) : eval()'d code on line 27

Can somebody share with your experience. Maybe some good python tool are on the net ?

thanks

you probably better of building something custom made. your major challenge would be execution time out. you could try using a background process for this.

You might want to review which pages you are including in your XML sitemap since there are so many. Some people like to include every page in there, but I do not think that is necessary. Bing has stated you should only include your useful content and not include your low/thin content pages.

If you do include all of your pages, take a look at the index ratio of URLs indexed to URLs submitted. I believe this is a good signal of what content Google believes to be quality. If you notice lots of URLs not being indexed, then see if you can increase the quality of content on those pages so they are indexed.

What about the Site Map Generation Tool by Google

Is that 10 million pages? If so, the answer is none.

If your website has original and informative webpages then congratulations, you must be either very tired or very rich - or both.

If your website is a content farm or just another example of auto-generated web fluff, then no further comment…