I think as with most of these situations, it's rarely so black and white. I agree that it's your business so you have to set the ground rules, but equally you have to be flexible and sometimes adjust some of these rules for each client. In his situation you need to see if the client is willing to reconsider the terms you are currently working to as clearly they are not ideal for you. As others have said, if they are unwilling to budge, you need to get out of the relationship. But if you can renegotiate the terms of the project, with a structured spec, specific completion dates, payment plans and hourly rates to abolish any financial issues with scope creep, would you be willing to continue working with them, or do you feel you simply want to get out as they are frankly a terrible client to work with?
Ultimately, the choice is yours - you cannot be forced to work for anyone, but you may find it hard to collect on any owed money if you pull out of the project.
I think there's enough info in this part of the forum to get you on the right track, you should spend a couple of days reading through it, and check out John Tabita's articles here on sitepoint