Is there ever a time when you would not sign a non-compete agreement with a client? Say, if the money was very very low, or for any other reason? And do you sign non-compete agreements with all of your clients?
There is almost no reason I would sign a non-compete agreement. It’d have to be an exceptional case, not the other way around. You never know whether in a year you might have an idea that is a bit competitive with someone you once did client work for, or if client work might give you inspiration to start your own website or business in some area, or if you might end up working (even for a short time) for someone that competes with a past client. You’re locking yourself out of those opportunities.
I signed a non-compete with a certain software company when I worked there for a short time, and it really held me back until that contract term expired. I had to shelve a lot of ideas and if I wanted to, wouldn’t have been able to work for a number of other potential employers for a while.
You make an excellent point and I agree.
For what this guy would be paying me it would not be worth it.
How do you gracefully tell them no?
I would say, YES! I take the job but if you could change the no compete clause to a non disclosure agreement that would be fantastic.
I’ve been threatened in the past with the no compete clause. And I know the current company I work for would not let me go easily (they’re paranoid).
Live and learn 
What is the difference between the two?
One says you can’t disclose to others anything proprietary you learn while working for that client.
The other says you can’t perform any work which competes with that client.