Yes, true. That's exactly where our platform provides value that the data of the entire Internet can now be easily used, re-mixed, re-deployed without spending an arm and a leg.
A data platform like this is meant to be scraped, as the data are all public as APIs. So feel free to scrape as much as you want!
Less work for everybody.
Of course you have to pay a small fee to compensate the costs.
That's definitely a way to go. You are right. People's work should be respected by compensation. That's a tricky part that we are trying to solve.
But still, more than HALF of the Internet traffic is now by bots crawling and scraping everything they find useful. Everyone is doing it for their own good. It's a done deal and I doubt anything would ever change that people would simply stop scraping and totally respect other people's work. God knows if their work is REALLY their work in the first place if you know what I mean.
Scraping costs much Internet bandwidth, especially with thousands of different crawlers and scrapers harvesting the same popular sites, over and over again. The same piece of data are scraped by so many different people writing the same scrapers, run so many times. It's a tremendous waste of bandwidth, power, and human resources.
Our solution is to become the go-to place for data on the Internet, where people exchange data under some licenses or by compensations, so scraping itself comes to an end, with the same piece of data scraped only once but served thousands of times.
It's a win-win-win for everybody. However data source websites can opt out any time they want.
There are far more sites on the Internet, I agree, but sites being constantly updated with data / content valuable enough to be scraped, don't come more than 1,000,000.
How about offering a search of data? The data record displayed would have a link to the source? Like Google. That would make things better? I never thought of this. That's genius thank you!
What if our platform saves you 5 - 10 hours of work? What's it worth for you in dollars?
ISP offers infra access to the Internet, or human consumable content. But not data.
We are still investigating the possibility of this business model and only have some prototypes. Maybe you would be interested in giving your insights and suggestions? We would be very much honored and grateful!