I don’t think you should feel guilty. After all, they also gave you their word of honor to send you a fair amount of work to do. I think the most decent thing you can do is be honest about your desire to look for a better opportunity. You can however at least give them two weeks notice before bolting
That’s why we never give somebody a “discounted” rate when they promise “lots of work”. Typically, the customer who says they have lots of work for you, don’t.
It’s fair for you to change your rates at any time. At least get them back up to your standard rate. Give them some notice that your rate is changing. And then I’d also explain that any rush work is billed at time and a half. That should stop the emergency work from landing in your lap. And if it doesn’t, you’re making a good rate on it.
Really good points here ^
I get quite nervous whenever someone suggests that they have lots of work lined up for us. In 15 years it has never come through.
Also as noted above, rush work generally calls for a higher rate because now you have to drop everything to satisfy some insane deadline, work through the night or a long weekend… In this case its to clean up someone else’s mess. It seems your client has it all backwards. They want a discount on rush work 
Don’t lose the client, instead politely tell him your prices have increased.
if you are sure you don’t want them as a client then next time they send you a job explain that you have a big job on and that you wont be able to do theirs for the next three months.
I think it would be better to be honest and tell the customer that from now all work will be done on prepaid basis and only described changes will be done. I mean, first you describe the work required, the customer agrees, pays, after that you do the work. It is normal practice, it prevents those cases when customers say ‘please do this and that and that also for free’.
First you discuss the changes, set up the price, the customer pays, you do the job. What do you think about that?