Design & UX
By James George

Making Complex Selections in Photoshop

By James George

Complex Selections in Photoshop

Making precise selections in Photoshop is an essential skill that every designer needs. As a print designer, you will find yourself using selections to remove objects from their background to place within ads. As a web designer, you might extract an image and place it on a website with no background. If you are a photographer, you might make a selection to remove blemishes or other unwanted features from an image. You can’t get around it; everyone uses selections, and if you make excellent selections, you’ll end up with excellent work.

So, what do you do when you have something that is extremely difficult to select with normal selection tools? As basic selection tools, you have the marquee tool, the lasso tools, the magic wand tool, and the quick selection tool at your disposal. These don’t always do the trick; you don’t have a truly precise tool to make intricate selections.

Even with the masking and channels, you can’t make a precise selection consistently. Sure, you can refine your selection, feather it, and spend a lot of time on it, but that method is neither easy nor consistent. Notice in the image below (which you can download from here), there is a woman on a subtle background. This might not be considered a terribly complex selection, but the hair is fairly difficult to extract onto its own layer.

You can try the quick selection tool, which does a great job of making a selection of the bulk of the image, but making a more refined selection is difficult with this method. The problem is the hair. The wisps of hair are too fine and tedious to select with standard selection tools.

In comes Refine Mask to the rescue. In the example, I made the selection using the Quick Selection Tool. This does a good job of selecting most of the image, but now we are going to refine our selection. In Photoshop, when you make a selection, the options menu for your selection tool should offer a button that says “Refine Mask.” Click that button to bring up the Refine Mask Dialog Box.

The Refine Mask menu is one of the best tools for refining your selection. You have many options for viewing your mask:

  • Marching Ants, which is your basic selection view.
  • Overlay, which is the same view as quick mask mode.
  • On Black, which gives you a lot of contrast in your view and looks just like a normal layer mask.
  • On White, which offers the same kind of contrast with the opposite background.
  • Black & White, which helps you distinguish your selection from the background when you are trying to see dark and light areas of an image that you want to add and subtract from your selection.
  • On Layers, which shows your underlying layers.
  • Reveal layer, which shows the normal background.

Each view has its own purpose, but the best views for contrast would have to be On Black, On White, and Black & White. This dialog box is handy, because you can easily toggle between each view by hitting the “F” key. You can see in the example below that our selection is good, but the wisps of hair are not included in the selection.

We can fix this by using the Smart Radius Slider. Just by using this slider, we can a more refined and precise selection. I bumped it all the way up to 250px and in doing so, have the result shown below:

This slider does a great job, but there are still a couple of hair wisps that aren’t included in the selection, making our image incomplete. We want our images to look as natural as possible, so we’ll need to include those. The Refine Mask dialog has a brush on the left that you can use to add to and subtract from the selection. Hold shift to add to the selection, and hold alt/option to subtract from the selection. If you need a different view, use the “F” key to toggle through the different views.

Our selection looks great and is much more detailed than before. As we cycle through the different views, we can see the hair that we wanted to include in our selection. Many would say that this is a very good selection, but now, if you go to the view called On Black, you can see a light outline around the woman’s body and in her hair. You can also see below that there is some ghosting around the hand. Use the brush to add these details back in.

This is called fringe. You could go through and erase it all or try to contract your selection, but it could cut into your image, which makes your selection worse instead of better. The best option in the Refine Edge menu is the bottom one called Output. If you select Decontaminate colors, you can move the slider to the right and remove the fringe.

You can also select how this is output. You can choose New layer with layer mask, new layer, new document, or new document with layer mask. If you still need to keep this in the current document, then choose to create a new layer with a layer mask. This gives you the ability to refine the layer mask even more. When you are done with your refinements, right-click on the layer mask icon and select Apply Mask.

With the image extracted, now you can place it over any background that you want. With your complex and professional level selections, you can convert them to masks and mask out and extract any object from its background. I added this background texture below the extracted image, and the result is shown below.

You might think that you need a lot of contrast for this to work, but in reality, you don’t. Here is a sample image of a dandelion, which you can find here. The image doesn’t have a lot of contrast, and the tones of the background and subject are very similar.

Make a rough selection of the dandelion. This doesn’t have to be exact, but try to select as much of the main image as you can. I used the quick selection tool, which took about 3 seconds to make this selection.

Then, I simply turned on smart radius and bumped the value all the way up.

To get rid of the extra content in the background, such as the green from the rest of the field, I used Decontaminate Colors and bumped the value all the way up. I exported the selection as a mask on its own layer and voila!

The Refine Edge and Refine Mask Settings can handle easy or tough selections, and it makes the process of making complex selections much easier. These techniques can make your Photoshop selections quicker, easier, and better.

  • This was an awesome and detailed post. Very helpful!

  • Wowww, thanks for such tutorial, very helpful indeed

  • Wow, such a great article. Photoshop is fantastic!! Keep the articles coming. Great Read <3

  • Very interesting tutorial. I haven’t used this before. I’m going to try it out.

    Even though it’s a good selection of the woman, though, it looks very unrealistic against the background where you’ve placed it. Certainly no one would be fooled for more than a second.

    I’d be very interested in a second tutorial going of from this and showing how to make it actually look realistic when you’re compositing the selection with the background.

  • The last one is really awesome! :-)

  • Wonderful – I remembered learning how to select feathery grass when I was in Design school, but could never remember how it was actually done. It’s bugged me for years, and it’s so easy! Thank you.

  • PaddyMac

    Nice post. Although I think you overlooked that her hands are now transparent in places!

  • this is really good photoshop tutorials

  • sec

    thanks :) very useful

  • nik

    sorry but did anyone see that the girl’s left hand got “burned’ in the process? that’s not very effective….

  • One could certainly work on the woman’s edge to make it perfect, which depends heavily on the kind of bg she’s supposed to be put on (light or dark). But while the dandelion outer selection is simply stunning and extremely difficult to do using standard selection tools, the inside of the flower shows that it was taken from another, light background. That spoils the trick. And that is why you just can’t Photoshop everything, and so if you want a well lit image of a dandelion on black background, and you can’t find one online… hire a photographer to create it for you :)

  • It’s high quality posts like this one that keep me coming back for more. I look forward to every new email from Design Festival. Thanks for the valuable lesson.

  • Hi – great tutorial – just curious – could you tell me what version photoshop you’re using. Doesn’t look like my CS3.

    Thansk you.


    Chris Silva

  • This is exactly what I was looking for. Thank you for making my search much more easier. Because I was really having a hard time trying to edit pictures of my friends and some of their hair I could not select. Thank you very much.

  • I like your blog, I will share it to my wife, she also like design.

  • Great Photoshop article. Really helped to explain the Refine Mask feature for me. Thank you very much. I look forward to trying this technique soon.

  • Thank you for the post, really helpful .

  • Absolutely brilliant! Thanks for all of the details. Your dandelion example is exactly what I’ve been looking for. That is so simple and so helpful.

  • Inbal

    It’s a good one!

    I still work with CS3…how do I do it with CS3 then?


  • Very practical software, this character is a big project

  • The girl in the tutorial is hot though

  • Jashim

    What a great tutorial.

  • pfcustodio

    A great tutorial. Outstanding.


  • It’s a good one!


  • Thanks so much for this post! I was trying to work out how to do this just yesterday and eventually gave up as I couldn’t work it out. Going to try again with your fabulous guidance! Thanks very much.

  • this is excellent tutorial, thanks.

  • Nice, nice & nice

  • Nice tutorial. That really would push me one level ahead, but I will need to get CS5 first.

  • very nice tutorial our intern got alot from this

  • nice tutorial thank you to enhance my knowledge about this topic.

Get the latest in Design, once a week, for free.