There's always a legal risk in skimming people's content. Whether or not it's enough to get yous sued is an entirely different matter but Google, Yelp, TripAdvisor, they've all fought over rights, access and content. Is anyone going to care about your service? Hard to say.
If you offer tools to remove content / opt-out that may help, but it also may put you in a position of expected moderation, which can hurt.
If you use APIs from vendors to access official data they may give you the info under known terms. But not everyone does this.
You don't have to lose a case to lose... defending a suit is not cheap. At the same time there are services out there already aggregating reviews very well (see buzillions.com and most comparison shopping site). Thus it comes down to risk vs reward vs mitigating by doing something in the middle.
To be very clear, this is a forum for the discussion of a topic and theory around it, not practical legal advice. You'll find opinions -- mostly from people far outside of the legal world -- that are good for conversation but if you want actual tactical suggestions about what to do, hire an attorney.