This is a problem I encountered recently and the 404s should not affect SEO. Below is a reply I received on Google’s webmaster forums:
It is completely normal for Googlebot to revisit previously known URLs even if there is no known internal path to them. Eventually they will stop being crawled if there is now way to find a path to them.