Already half of the buyers don't pay for the auctions !
I will be happy to pay 5% of the sale price if you handle the entire transaction and keeping buyers and sellers safe .
Sedo and others charge 10% bcoz they take care of the entire transaction and keep both parties safe .
You could charge 10% like sedo and split the fees from (buyers / sellers), take care of the transaction and website files / domain transfer . Then it will be great ! . both seller and buyer feels safe .
if you can't start an escrow service of your own , try to get an affiliation with big escrow guys .
but without something extraordinay increasing prices simply to get rid of junk sellers is a bad idea .
you could moderate all the auctions with the help of moderators to get rid of all the junk listings !