If you want to round the corners on your photographs or illustrations on your web site, one very hand tool to help you do this is the Rounded Rectangle vector tool in Photoshop. Here’s how to do it.
1. Open a photo and double-click the Background layer to make it editable. We’re unlocking the Background layer because we’ll be adding a mask to it later. (This is a photograph I took recently when a film crew was working on a period drama in a town near where I live, I converted to black and white with the tiniest hint of sepia).
2. Select the Rounded Rectangle tool from the Tools panel. The Rounded Rectangle may not be visible. Click and hold down the Rectangle tool icon until a pop-up menu appears and then choose the Rounded Rectangle tool.
3. In the Options bar, click the Paths mode button and change the Radius field to 50 pixels. There are several modes available when working with vectors in Photoshop but for this particular technique, make sure that the Rounded Rectangle tool is in Paths mode (the button looks like a square with a tiny dot on each corner with a pen in the center).
To control how round the image’s corners will be, type in a low number for a less rounded corner and a high number for a more rounded corner. I’m using 50 pixels in this case.
4. With your mouse over one corner of the image, click and drag diagonally to draw a box around it. When you release the mouse button, you’ll see a thing grey line that appears over your image – this is the path. If you need to move it, select the Path Selection tool (the black arrow below the Type tool in the toolbox) and drag it to where you want to put it.
5. In the Layers panel, make sure the photo layer is selected then add a vector layer mask by Ctrl + Clicking the Add Layer Mask icon at the bottom of the Layers panel.
Et Voila! Photoshop hides the area outside the rounded rectangle, leaving you with lovely smooth corners.
The great thing about using vectors is that you can resize the rectangle (Ctrl + T to free transform) and you will not lose the quality and smooth edges.