Thank you for that - it eventually led me to the answer! I realized I should compare the code WP was giving me for the working caption as opposed to the nonworking one.
The nonworking version (from WP 3.5.1) puts the caption (i.e., THIS IS MY CAPTION TEXT) outside of the caption brackets [ ] and at the end of the code, like this:
The working version (from WP 3.4.2) puts it in the brackets:
<p>[caption id="attachment_716"]<a href="my-link"><img class="blah-blah" src="my-image" /></a>THIS IS MY CAPTION TEXT[/caption]</p>
So to work around this for now, I will just manually move the caption text into the brackets. What I don't understand is why I should have to do this. Is this a WordPress bug or is it just me? I'm inserting a photo and creating a caption the same as always, but the resulting code is different now and doesn't work.
<p>[caption id="attachment_716" caption="THIS IS MY CAPTION TEXT"]<a href="my-link"><img class="blah blah" src="my-image" /></a>[/caption]
Edit: Plus now when I go in to make other changes to the page, it doesn't save my changes to the caption. Argh.