XML flavor of VBulletin?

[MattR]

I run a pretty hacked up version of VBulletin (the software that powers these forums as well) and I was thinking the other day about XML and PHP... so anyways:

There are better things we can be doing to improve performance than spending money on hardware for our databases.

I was thinking -- make the forums / threads / posts XML and then store all the other information in the database.

That would give the advantage of having lightning quick page display (since there aren't any worries about querying the DB each and every time a thread is viewed) and the ability to update them easily (via XML coolness). It also allows you to easily package things up for stuff like WAP (provided people would want to read posts and such from their WAP device).

Any thoughts?

[MattR]

Another thought is that maybe create an archive of posts / threads in XML (so that you can search them) every month or something to lessen the load of the post and thread tables.
So, you'd kick off a script every so often which would go through and rip out all posts / threads that haven't been touched in the past few weeks, write them to the XML files, and then kill them from the post and thread tables.

[TWTCommish]

I only know a little of XML - could you explain how this would work in more detail?

[MattR]

Well, you can almost think of XML as being a flat-file version of a database...

For instance, you could represent a thread as this:

<?xml version="1.0"?>
<thread id="11432423">
  <title>Do you like bacon?</title>
  <threadstarter>bob</threadstarter>

  <post id="21312">
    <userid>32423</userid> (or you could use <userid="32423"/>)
    <username>bob</username>
    <dateline>343242</dateline>
    <signature="1">
    <pagetext>This is the first post of the thread.  I am cool.  Yadda yadda yadda</pagetext>
  </post>

  <post id="21316">
    <userid>324322</userid> 
    <username>joe</username>
    <dateline>343299</dateline>
    <signature="1"/>
    <pagetext>Poop on that</pagetext>
  </post>
</thread>

Off of the top of my head, you'd set up a logical directory structure like this if you want to retain discrete posts / threads / forums:
/forum32/thread34234/post3243.xml
/forum32/thread34234/post3244.xml

or maybe:
/forum32/thread23323.xml (which would contain the same content that I made above)
/forum32/thread23233.xml

Then you'd call up PHP with the same parameters:
forumdisplay?forumid=32

would open the /forum32/ directory and loop through X number of threads depending on the settings and then you'd use PHP to parse the XML and turn it into the familiar thread display. I'm still fuzzy on PHP's XML parsing techniques but I think with some study I can pound something out (http://www.php.net/manual/en/ref.xml.php).

[Skunk]

alpha-board is a forum system which uses files similar to XML:

www.ukterrorist.com <-- check out the forums on there

It's a nice idea but in practise it greatly limits things like searching.

[TWTCommish]

I think I see what you mean, but doesn't that makes things slower and less dynamic? It make, say, reduce sever load in some cases, but vBulletin's dynamic attributes are what make it so appealing.

[MattR]

Originally posted by TWTCommish
I think I see what you mean, but doesn't that makes things slower and less dynamic? It make, say, reduce sever load in some cases, but vBulletin's dynamic attributes are what make it so appealing.

No, and no.

On the surface I would think it would be faster because you are reading from a flat file (a la a single HTML file) -- granted PHP has to parse it, but you don't have to do any database queries / join everything, yadda yadda yadda.

Less dynamic? I'm not sure what you mean by that. I can't think of anything that would not work in XML that vB does now.

[TWTCommish]

I suppose I need things explained to me a little more - are you saying it would all be MySQL based, but it would spit out XML code instead?

[MattR]

Sort of -- it would be like combining UBB and vB -- use the database for stuff that "should" be in there, such as things that change frequently, full-text search capability, etc. -- but remove things that do NOT need to be in there, such as forum, thread, and post information.

It's just a thought and I have no clue how it would work in a production environment -- but one of the things UBB has going for them is that they use HTML files for display purposes. This is GREAT from a server load / speed issue -- just check benchmarks of a PHP and a HTML file on a web server. You can show significantly more HTML files than PHP files in a given time interval.

So, store the post and thread information as XML files and then use PHP to output to the browser. Granted it’s a little slower than HTML files but not near as slow as running queries on the database.

Or you could use the XML files to output to HTML files for true HTML speed. You’d lose little hacks like the “ONLINE / offline” hack, but I can live without that.

There are some logistical things to work around (you’d still have the database for some global things I would think) but I feel that mixing flat and DB page generation would ultimately save the vB owner time and more importantly money because they don’t have to purchase more hardware for their database to eat up.