I have to confess that I've not revisited the great resolution debate in recent times. I'm due for a brush up on the new wave of higher resolution screened devices and like anything in life, it all used to be simpler.
I have a background in graphic design and used primarily a print medium. We designed books, posters, ads etc and it used to be very easy to talk in terms of resolution because there was a maximum that any Press/Printer could achieve. You could talk to your Print company, get those specs and know the limits of the machinery and work to that. As specs improved there came a point where the perception of what was 'photo quality' started to mismatch the impressive numbers being thrown around. For most people looking at a print above about 300dpi looked almost identical to one of 1200dpi, you could say that the 300dpi version was good enough and the gains from there were small and appreciated only by a few.
Likewise screen technology had its limits, but the information to display on a screen (as opposed to paper) was much smaller. What looked sharp on paper at 300dpi, only took 72dpi on a screen, there simply wasn't any point in giving more information to photos on the web than was necessary, it was inefficient and just a waste.
Likewise in the days of dialup maintaining low file sizes was very important. People didn't want to wait all day for a photo that they may not even be interested in seeing. Optimising images became an important part of web design. One thing that was routinely done was to limit the dimensions to something appropriate, and also to remove unnecessary information stored in the file when it wasn't necessary for online. Typically these were things like thumbnails of the image that are used by the operating systems to quickly show you what a photo looks like without having to launch something like Photoshop. Saving for the Web in Photoshop has these sort of functions, it strips unnecessary data out and optimises the image for display on websites.
So where too now? With retina/higher resolution screens capable of displaying greater information and mobile devices running on limited data both being important considerations these days.
As a general rule which will always apply: you want enough image data to display your graphical content well, but not more than you need. Optimising images is important to Google, as is how they are cached and referenced in your website. Google is looking to see that the creators of websites balance these considerations of the users.
You can check any image file you have prepared against a tool like Smush.it to see how well you've optimised it.