The first thing to check then is the load on the server. Your response times are terrible but that doesn't mean it's the server its self... are the load averages high, is memory maxed out, etc...
Next you'll want to head over to the vBulletin forums and read up on the mySQL and web server optimization tips. Simple settings from your max clients to your cache can have a dramatic impact especially if you're using InnoDB on the server side.
If your server is taxed and load is optimized the solution is of course more hardware, moving the DB to it's own box and so forth.
Assuming all of that turns out to be in order it gets tricky as you have to isolate what specific functions or queries are holding things up. This means disabling mods, enabling server level logs and sifting through individual scripts, potentially even kicking guests off the site.
I realize this is fairly broad but a slow site isn't a simple solve.