Firstly, each page is ranked on its own merits against the exact search query used. While the overall quality of your site may be a small ranking factor when it comes to separating two results of similar quality, having a few slow pages on your site isn't going to have an adverse impact on your general search rankings.
The main question is, do you want your potential visitors to be able to find these slow pages directly from Google search? If the answer to that is yes, then let Googlebot crawl and index them. If the answer is no, for some reason, then exclude Googlebot.