Skip to main content

Using Composite Photography to Create a Fantasy World

Lesson 14 of 31

How to Use Photoshop™ Brushes to Mask Out Images

Karen Alsop

Using Composite Photography to Create a Fantasy World

Karen Alsop

Starting under


Get access to this class +2000 more taught by the world's top experts

  • 24/7 access via desktop, mobile, or TV
  • New classes added every month
  • Download lessons for offline viewing
  • Exclusive content for subscribers

Lesson Info

14. How to Use Photoshop™ Brushes to Mask Out Images

Lesson Info

How to Use Photoshop™ Brushes to Mask Out Images

Okay zooming up on my rabbit here. As you saw before, the hair around here isn't really nicely cut out, it's a bit rough, so here is where I use my very special brush that I've created, it's a hair brush, I've got a couple of them. So in my brush set here, I'm only using brushes that I've got in my pack, although I think there's a couple of extra ones here, but most of the brushes that I use here will be from that pack and I'll let you know when I do that but this brush here is in the pack that comes with the class and it's a hair brush. I'll create a new layer so you can see actually how it looks. That's actually how the brush looks. What I've done is I've made it soft on this side, and kinda rough sort of hair on the other side, and this isn't to paint hair on, this is really to use to mask back hair. So you're making it look like there was hair there that maybe wasn't or you're masking back clip hair. Gonna go into the rabbit layer onto my mask, make my brush smaller, about the size...

of the hair so that the hair is sort of the same width and length as the hair of the rabbit, and using my brush, just paint in around the edges. And if I zoom up, 'cause that's hard to see, what that's doing is it's masking back and making it look like fine hair, so you see in between all those fine hairs. And you can go back and forth a bit. You'll notice one other little thing that I do here that you might wonder how am I doing it is I'm turning my pen around and it's turning the brush around. It took me a long time to figure out how to do this but it's actually a special Wacom pen, it's called the art pen, you have to buy it separately but it's amazing because you can turn on brush projection and then you can totally control the direction of your brush, so normally when you're in a particular brush, you have to do that manually, have to turn the rotation around manually like that, but the art pen, you can turn on brush projection and it only works with the art pen but it enables you to control the brush. And when I'm doing all this fine edging, especially in masking back and needing to have particular direction on my brush, it saves me so much time, so I highly recommend that pen. So I can turn this brush back this way, and I can press X, which switches my mask to white, so it's revealing so white reveals and black conceals and I can paint back again. So I usually go over it a couple of times, sometimes I even turn down my flow so that I can build up the hair area but if I go over that, I don't have to do it slowly, I can do it quite quickly and just go round the rabbit like this. And then back over, just to refine that hair area. When I zoom out, I know I'm working in very, very closely and if you're working just for the web and you're putting images up on the web, this sort of detail's not really going to be seen, but when you print, I usually print my images really quite large, up to (mumbles) size on fine art paper, on Canson Fine Art Paper, all of that detail, you can see it, so I'm pretty particular on just the littlest thing but it does depend on your output and what you're doing with it. But this for me, doing it this way gives me control over where I want the rabbit's hair and I can blend hair and grass and all sorts of things using the brushes, so I'm going to just go 'round the rabbit, mainly down the bottom 'cause it's the feet and body area that we're using predominantly. And just paint that out. So as I do that, I'm happy to answer any questions as I go round here. So first of all, just a reminder everyone, as Karen was talking about that all these brush that Karen is showing you, her set of brushes are included, that's a $45 value that is included when you do purchase this class, so that's really, really cool, thank you for that, Karen. No problem. Do you think that people can do this type of work without a tablet? That's a question from-- That's a really good question. It's definitely possible with a mouse but a big control that a tablet gives you is the pressure sensitivity. I'm not using pressure sensitivity at the moment, so if I was using a mouse, I would still be able to use this brush, I would have to rotate it manually as I showed you before but you could certainly do it. If you're doing a lot of this type of work though, the tablet is certainly a lifesaver, particularly with your arms and your hand and your wrist and everything as well. This is fine editing, this is like painting, so to have that control with a pen is much more natural than using a mouse, but I've seen people use a mouse, some of the best compositors maybe. Some of the photographers out there that have only just switched to a tablet that have been creating the most amazing pieces just using a mouse, so it's certainly not impossible. But if any of the brushes use brush projection, that's where you need the art pen, but you can turn off brush projection if you don't have the art pen. So just go up here. I'll zoom out, 'cause I just lost where I was up to, here we go. So this hair brush works well for human hair, works well for animal hair. So this is the other thing, if you're not using a green screen and you do need to cut out manually, this is more the type of approach that you would take is you would just mask around your subject and then you could use a brush to refine that edge. I'm not gonna spend as long on the top here because I probably will use a different head. I will go in and do that quickly. Any questions from you guys about the method here? All right so I'll leave the head there like that. One of the things with this pen or this brush that I've created is it's got a soft edge on the opposite side so you can sort of paint back on the opposite side where it's got a round edge to it, so it's like multi-use. Because I'm doing this all the time, I found that was a really cool way to work, instead of swapping back to the round brush, sort of having the round bit on one side and the brush on the other. It's all about time saving. Okay, so now we've got our rabbit there, cut out but without the head that I'm after, and I do need to hide Alice as well. So we can just see our rabbit. So next, I wanna go back to the head and then blend them in together so that we've got a rabbit who will be armless at this point in time because the arms are coming from the costume, but we'll have a rabbit that basically has the head, body blended, ready to be dressed. So back to Lightroom, so I do use Lightroom and Photoshop sort of together and work back and forth between the two. Okay so going back to my rabbit head. As I said before, even though I like that this one here is turned, that's not gonna work with the costume shot, because the costume shot was straight on so we do need the head to be probably more this one that we chose before. And it's really cute anyway so we'll choose this one. We will open that, edit in, open as a smart object in Photoshop. Okay. I'll drag this whole thing into my scene here because I am just going to mask this in very quickly using my brush instead of trying to cut around it because all I need here is the front of the face and the ears, so before I shrink this rabbit down, I'm going to create a mask, so by hitting down that button down the bottom which is the mask tool and that gives the smart object a mask, then I'll go into my brushes, I'll go into the default round brush, turn my hardness up to around about 80%. And switch to black and just quickly paint in around the edge and then I can go in and refine it once I've done that so the benefit of a mask as opposed to erasing is that you can go back and forth, it's non-destructive, it doesn't matter if you go too far, you can just go back by pressing X and going to white, so whereas if you use the eraser tool, you can't go back, you've done it then and you can't return, so definitely work with masks. And then you'll have much more control. If you've been using Photoshop for a very long time, sometimes it is hard to learn the newer tools, although mask's been around quite a long time, but as Photoshop brings out new tools and features, it's great to be able to work out what they are and try and figure out how they can help your work flow, 'cause they spend hours upon hours developing these features to enhance work flows, so every time a new version comes out, I work out what is it that's in it and how can that help me in creating my images? Okay so now I've got my rabbit head. It's a very quick cut out as I said before, I need to go in and refine it. I'll shrink that down. Now what I wanna do here is I still want some rabbit hair fur neck, because when I put the costume on, I wanna be able to blend it in the correct angle, so I'll hide that for a moment, here we've got the rabbit, he's sorta squished over, she's sort of squished over. So what I'm gonna do with this here is just add extra fur up above so that I can blend it. I don't really want the rabbit's head in this shot if that makes sense because I'm gonna use the head from the other one. Start sounding really bad when I'm doing that, cutting off heads and (chuckles). Yeah, shooting people, cutting things off, it's terrible, but I'm going to here add some fur up the top, and one way I can do that, you know normally, you can use this tool here, the clone stamp tool, but to do that on that particular layer, I would have to rasterize it, okay, and then I would lose that smart object ability. Instead, I wanna keep it as a smart object, so I'm going to create a new layer above. And I'm going to clip it, create clipping mask, to the rabbit layer below. What that does is it means anything I apply to this layer above only takes place on where that rabbit is, so if I just draw, that's only going to apply to where that rabbit mask is. If I unclip that, release clipping mask, it's over the whole thing. So if I create that clipping mask again, erase, if anyone didn't realize that there's been a razor on the bottom of your pen with your Wacom tablet, so very cool, just flip it over like a normal pencil. Okay so here what I wanna do is bring the fur up above, so if I go to the clone stamp tool and I check current and below at the top, what that's going to do is that's going to grab the fur that I pick up from below and apply it up above. If I just have it on current layer, that layer is blank so it's not going to pick anything up, but if I go current and below and then I hit alt or option on the Mac, take from down below here, paint it up here, so I'm bringing fur up. So that when I put the costume on, I've kinda got more fur to blend with my rabbit. Take it from here. Remember this is taking place on the layer above as well so I can mask back any areas that don't quite work. Don't really need the ears there, I just want fur. So I'll actually mask out the ears in a moment. That will do the job, it's gonna be underneath the costume so it doesn't need to be perfect. The ears on that mask below, I will just paint those out now, because I'm going to use the ears for my rabbit head. Okay so now I've got a very strange looking (chuckles) animal sitting there. It's very interesting, some of my half-finished images looking, okay, so rabbit head, here we go. My rabbit head, this is the rabbit I wanna use. So looking here, we've got to blend this area in, there's some whiskers there which we will probably wanna flow into the fur or flow over the costume. I'll probably mask them back after I add the costume because then I'll be able to see it, so for now, I'm going to just finally mask around the rabbit's head, again with that fur brush. All around the edge. And I feel it's a touch too obvious there, so I'm going to turn my flow down and blend it so soften up the edge more. Press X, go backwards in the process, so bring some more fur back. And turn my flow up for the rest of the edges. You can see how quick it is just to run the brush around the fur here, so much quicker than trying to cut around the edge or trying to refine the edge. And much, much quicker, especially if your animal is not against a solid background, so if your animal, you photographed out in the wild, or, wherever it is, you've got possibly similar colors behind the animal, and to use this method means you've got total control over the mask and you can create that fur and it doesn't matter what's behind the animal. So it's a very quick way of doing it, now you can see, because there's black behind my rabbit, and I don't want that black to show up, let's just for example say I wanted extra fur here. It's not gonna work on that side, I'll do it here. Okay can you see that black here? Say for example I wanted to expand out the fur. One thing I can do is create another layer above, too many. Create a clipping mask and do exactly the same thing. If I use my clone stamp tool and I grab some fur from over here and I paint over here... Can you see, it's expanding out the fur, but it's keeping the texture of the fur because I've masked that, so it's sticking to where I masked that fur. And I can turn that on and off separately. So if you've not got enough and you need to expand out your fur, that's how you would do it. You create another layer above, I'm going to do that again 'cause it's a couple of steps. So delete that. I'll go back with the mask, zoom up. Okay so it was like this. And I'm pretending I wanted extra fur on the rabbit's hair there but I don't obviously want the black area there, I want it to look like rabbit fur coming out. I create a new layer above. I create a clipping mask. Okay, if I release that, there is another shortcut to create a clipping mask too, it is alt or option and hold it in between the two layers and click that. And then, get my clone stamp tool, choose some hair from closeby so that the texture's the same, and stamp it over the area that you want the fur to be. And there you've got your extra fur. It's got all the texture of the fur. You're not painting on a static color, you've got all of those differences with the highlights and the shadows, it's very easy to do. So I'll turn that off because that was just an example. I'm going to mask back over here using the soft edge of the brush, because this part of the bunny, the ears is not really that furry. Little bit more on this side, a bit of fur. And going down here, a little bit here. So he's nearly done, he's looking pretty good. Okay zooming out. Okay, he's got weak shoulders going on at the moment, but we're going to leave the shoulders like that at the moment because we wanna see how the costume looks and the shoulders look in the costume and everything like that. He's almost ready to go with the costume, but are there any questions at this point? We have a couple questions that have come in and the first one is Old Red Eye had asked, can you have more than one toolbox of brushes in Photoshop, one for the default brushes and one for the ones that you're creating? That's a good question, I'm not sure if I know the exact answer to that. I have found that as you add brushes, then they go, they get added on so you have your default brushes, and you add more brushes and they keep going on to the bottom. You can rearrange them, but I do believe you have to load the brush set separately, so you could save a brush set, like I've saved this set of my brushes, you could save that as a brush set so all the ones you regularly use, you can turn off the other default ones and just load that brush set. And you could do the same if you wanted to reload, but someone else might have a better and than that, but as far as I know, that's how you do it.

Class Description

Karen Alsop is known for creating beautiful fantasy worlds through her unique compositing techniques in Adobe® Lightroom® and Adobe® Photoshop®. Whether you're a wedding, portrait, landscape or commercial photographer, this class will show you how to create beautiful and distinctive images you can offer your clients to expand your business.  

Join us for this class, and you’ll learn how to: 

  • Shoot with your composite in mind: lighting, posing and angles.
  • Choose background and subject images that will work best in the composite.
  • Learn Adobe Lightroom and AdobePhotoshop techniques to create a fantastical atmosphere.

Karen’s emphasis on creativity and imagination in her process has helped her to make a product that competitors have a hard time recreating. Karen’s beautiful, intricate work is not simply the result of vast technical skill, but rather is the careful integration of a number of elements. She puts subjects at ease and inspires them with artful direction; incorporates them into fantasy landscapes using Adobe Lightroom and Adobe Photoshop and then effectively prices and markets the final product.  

Software Used: Adobe Photoshop CC 2015


Judy Mitschelen

I've found many great instructors at CreativeLive and Karen ranks right up there at the top! With her relaxed, thoughtful manner of presenting, I was immediately hooked. Her organization, clear explanations and demonstration, and on target response to questions are superb. This course covers an amazing range of skills and tricks of the trade. Whether you're interested in getting better shots to work with, better workflow at the computer, or better output at the end, Karen covers it all.


Karen is very talented and a great teacher and I enjoyed every minute of the course. But what I found to be the best part was seeing what an amazing person she is. The video of compositing the disabled children to make their dreams come true had me in tears. It has inspired me to use my talents to help others and not learn photo manipulation for self enjoyment. God bless you Karen.

Endigo Rae

This was such an amazing class! Karen is so talented, inspiring, and such an amazing teacher. Very forthcoming and open about all of her techniques. I'm so looking forward to jumping into compositing, I feel like this is definitely something my soul desires to explore and Karen has made it so easy and accesible through her beautiful course! Thanks so much Karen and CreativeLive!