First go to the page/post of the photoset that needs captions to be changed. Once you are on that page, edit the page.
Go to the dropdown menu at the top of the lightbox and find
Uploaded to this post. This will show you all of the images that were posted on this page, which should include all of the photoset images.
Now, select the image you want to update the caption for.
On the left side of the page you will see a caption box. Fill this in or change the content to what you desire. Then, you must tab to the next box in order for the changes to take effect. Watch for a turning ball in the top right corner and wait for it to say “saved.” You can then close the window.