I'm currently setting up a marketplace style web site where buyers purchase directly from the third party sellers, I just provide a venue for the transaction. We'll be using Paypal Adaptive Payments which gives you some great options. Basically the customer uses your checkout system, but it can be configured in a few ways behind the scenes:
1) Parallel payment: the customer pays directly into the 3rd party vendor's paypal account. If they buy from multiple vendors, PAypal ensures the payment is split correctly and sent to all vendors simultaneously (identical tot he way Ebay works if you are buying from multiple vendors). Contract of sale is between customer and vendors, not you.
2) Chained Payment (probably the one you want): Customer pays you, but you can configure the payment to then automatically be sent to multiple third parties minus your commission. You can also automatically delay those secondary payments if required (i.e. once the customer confirms they are happy)
Using chained payments, you get your commission immediately, and control when the third parties get their money. However, customer is paying you so the buck stops at you - so make sure it's worth the extra hassle and adjust commission accordingly!
With option 1, you can create an additional party in the parallel payment, i.e. yourself and get your commission paid that way, but the customer will see this, which may be confusingy. otherwise you'll have to bill the vendors for your commission separately.
As for VAT, we discussed his with HMRC recently and all they care about is who the contract of sale is with (i.e. who the invoice is from). So with chained payments as long as you invoice the customer directly on behalf of the third party vendors (like Amazon do with amazon Marektplace), the VAT on the initial transaction is charged by the vendor, not the marketplace owners. You simply charge VAT on your commission to the vendor.
So customer pays £100 for a product via chained payments.
This £100 already includes the VAT From the vendor.
You are charging vendors 10% commission + VAT = £12
So you set up chained payments to forward the vendor £88 while you keep the £12 for yourself.
What I am not sure of yet is what happens when the customer requests a refund/chargeback in chained payments - does the the marketplace owner pay it and then recover it from the vendor(s) involved, or do Paypal claw it back from everyone in the chain? Hopefully the latter, but we're still looking into that.