With E-Commerce sites, products come and go and often times create 404 errors as a result of pages no longer existing or paginated category pages being cut down as a result.
Is a 404 response the best way to approach pages that come and go for products and paginated data?
No. You should never deliberately serve a 404 … it should only ever happen (1) if someone types in or clicks on a link to a non-existent URL, or (2) if you make a mistake and mess something up. Obviously, you don’t want either of these to happen if you can help it.
For products that are no longer available, put a redirect on the URL to a generic page that says something along the lines of “Sorry, that product is no longer available. Please choose a section from the links below to see our current catalogue range”.
Where the pagination is out of kilter, I would just re-map it to the last page (or possibly the first?) and if possible put a note at the top of the page along this lines of “Sorry - you asked for page 17 but we’ve only got 15 pages in this category”.
I agree that a 301 redirect would be an ideal resolution in the event that the page will never be served again because the site no longer supplies the product.
But in the event that “/category/page/2” is being redirected in some cases back to the first, and then sometimes should be returning 200 OK. Is Google going penalize these pages if they are changing between 200 and a 301 in the event of paginated pages/emergence of the old product?
If it’s a pagination issue, or some other temporary reason why you’re not returning the exact URL requested, use a 302 redirect to indicate that the page will probably come back some time later. That way, Google is less likely to index the redirect unless it’s in place for a long time.
Another thing to consider is that if your product lists are changing frequently, it may be worth thinking about whether to put a canonical tag on that sends people to the first page in the category. That’s assuming that the pages are in rank order, so the first page is where you’d expect most transactions to take place. That way, people are less likely to get confused by arriving at your site in the middle of the list … not as bad if the product they’re after is on a later page, but if they arrive at page 4 and since Google indexed it the product they’re after has moved up to page 3, there’s a much higher risk they won’t find it at all.