I think I'm probably the king of turning down work. I typically decline two or three projects for every one I take on.
Some of it is economics; when it's clear that I'm talking to someone on a shoestring budget who's considering spending his $200 food budget on a hundred words of web copy, I know it's not going to lead to anything. It's also going to be a lot of trouble. I can usually afford to hang back and wait for someone with a healthy budget. I turn them away quickly and apologetically.
Most of the time, though, I either just don't have any interest in their project, or their situation is such that it would be far too difficult to leave them a happy client. If I look at the situation and see a high potential of an unhappy client - no matter what I do - I turn down the project. This could be because of their attitude on the phone, or because they've painted themselves into an SEO corner and they want me to magically get them out of it, or because they seem to expect me to craft perfect copy on no input. In the end, it's a vibe thing. If there's no real rapport in that first call, I almost always turn down the work.
I used to take on every project that came my way, and then I wised up. Since then I've worked quite a bit less, but have made a lot more money. Never hire a client you wouldn't go fishing with.
In any case, I usually just email them back (don't do it on the phone), thanking them for their time and interest, but politely turning down their project on the grounds that I don't believe that they would get full value from the relationship; I'm simply not the right writer for them. I assure them that there are plenty of good writers out there, point them to my website article on hiring copywriters, and wish them well and good luck.
Another option on the low budget prospect is to simply state that their project isn't economically feasible for you to take on (i.e., "I'd lose money"). That's what an attorney would do if you hired one to file a lawsuit with slim success chances and little collection odds.
One point: the only reason to recommend another professional is that you're too busy to take work that you'd really like to take on. Never refer troublemakers to your competitors; it's unprofessional and will destroy potential for valuable industry alliances later on. The problem child will find them on his own.
At any rate, never hire a client simply because you don't know how to turn down work. Trust me, it won't follow you, especially if you're professional about it. An annoyed, rejected prospect is far, far better than an angry, disappointed client.