content grabber

[jdulberg]

I've been using a content grabber to get daily updated mortgage closings from a financial website. It seems that they have somehow blocked out this functionality from their site. I was just wondering how this is possible and if there is any way of getting around it other than getting a new data source.

I am showing a link to the company on the website "data courtesty of ...." so people know where the data is coming from.

I tried the content grabber on other sites to ensure that its working correctly and it all looks good. Just doesn't work on some sites.

Any ideas are appreciated.

[drisley]

They can block it using .htaccess files under Apache. I don't know what software they use, though.

I'm not aware of a way to get around it. But, quite frankly, you should ask them before doing that anyway.

[tubedogg]

If they went so far as to block it, they probably don't want you doing it. Why would you continue to do this against their wishes?

Try asking them if you could work out a contract to allow this. It would require paying them something probably but at least you'd be doing it with a clean conscience.

[jdulberg]

I totally agree with you guys... its trying to explain it to my client that's the problem. He has the idea that if it works on one site, why can't it work on others.

Thanks.