You did not get bit by Penguin for using a person’s name as anchor text. Many large retail chains have 90% of their links as the business name (the anchor text says Amazon or amazon.com, for instance). Popular SEO experts like Rand Fishkin, Danny Sullivan or even Matt Cutts probably have the majority of links to their articles using only their name. Now, if that person was spamming forums or blogs with links, it would be another thing entirely but using the anchor text “James Smith” is not losing backlinks. If it was, you wouldn’t see them in the report!
Penguin penalizes over use of keyword anchor text that directs to a page optimized around those keywords, not people’s names or brand names. Although it is difficult to make out the dates from the image, it appears that it definitely was not Penguin that caused the backlinks drop (which Penguin wouldn’t do, anyway). Your drop in backlinks appears to have occurred in late April/Early May - three weeks before Penguin 2.0 was released. Penguin does not remove backlinks - it devalues them or even assigns a penalty to the really bad ones, but these links still show up in your profile (otherwise, how could you remove them?). You are not showing the more important graphs, which are traffic and SERP position reports.
Sometimes, you lose a ton of links when a website is taken down or de-indexed by Google. Also, if a ton of your links are comment links, you lose them in bunches as blog posts get moved further and further away from the home page. When they are new, a single link will show up in the post, the home page, “Recent Articles” or maybe even “Most Popular Articles.” Over time, it is entirely possible for that link to only show up as the single page link (the article gets bumped off the home page and “Recent Posts” page by newer articles and other articles may become more popular, moving the article (and link) off of a “Most Popular Articles” page).
Sorry, but there is no way to assess anything here based upon what you are showing us. The only thing for sure is that an algorithm change or even a manual penalty doesn’t affect the number of backlinks you have pointing at your website. It only changes the value of them in Google’s eyes. The only time an algorithm or manual penalty could affect the number of backlinks is if you had very bad websites linking to you that were de-indexed completely. That’s not something “James Smith” would have any influence over.
Now, if you saw a huge drop in traffic starting around May 22, it was Penguin that got you. In that case, you need to look at each and every link, one by one, and find the spammy ones. Dropping spammy irrelevant links in comments on blogs or creating a bunch of forum profile links would certainly be things to look at. Also, even if your comments were great, if the pages they were on got devalued by Google for being low quality, those links would be worthless or could even hurt you (but they’d still show up in your links report). For all you know, a website that you comment on frequently got nailed for selling advertising and/or not placing a no-follow tag on the ads or links that they sold. Nothing you did wrong there but definitely something that could hurt the value of any links you have on their now “untrusted” websites.
You can try to remove those by contacting webmasters, but you need to have really a lot of time for that 