Skip to main content


Lesson 5 from: Introduction to Fantasy Compositing

Renée Robyn

buy this class


Sale Ends Soon!

starting under


Unlock this classplus 2200+ more >

Lesson Info

5. Masking

Mask the model and complicated headdress with a variety of techniques for smooth extraction.

Lesson Info


1 So let's start doing some fun stuff. 2 Let's get into masking. 3 So what I'm gonna do now is I'm gonna drag 4 in the background image that I created, 5 or well, not that I created, that I shot. 6 I photographed it when I was 7 in Scotland in the Isle of Skye. 8 And I'm gonna bring that over into this image 9 and start masking out this awesome 10 but extremely detailed headdress. 11 So first things first, 12 I'm gonna open up a background image. 13 So this is the background image I was 14 thinking about using. 15 I'm gonna replace the skyline here. 16 Sorry about the ears that are 17 in the corner of the view here. 18 My cat loves to watch me work in Photoshop, 19 so you're gonna see a little head pop up 20 every now and then, 21 Anyways, so this is our image. 22 It is 16 bit, 300 pixels, 23 and I'm not really gonna do much 24 to this other than I'm gonna 25 reduce the clarity just a little bit. 26 Now, the reason why I do that 27 is because this is a background image. 28 I'm not sh...

owing the ground the contact point 29 where she's gonna be touching the ground. 30 So I definitely always try to keep 31 that in mind when I'm working on this. 32 So I just drop it a tiny little bit. 33 I could blur the background, 34 but reducing the clarity oftentimes is enough. 35 So I'm just gonna go open image, 36 and here we go. 37 So I'm gonna select all by Control or Command + A, 38 and then I'm gonna go copy, Control + C 39 or Command + C. 40 Over here to our new, 41 all of our work here. 42 Control + Shift + V or Command + Shift + V. 43 Now what I'm gonna do here 44 is I'm gonna put all of this into a group. 45 Gonna call that contrast. 46 You can call it whatever you want. 47 You can call it like you know, puppies and stuff, 48 but make it something that makes sense, 49 Control + G. 50 So selecting all by holding Shift, 51 and then Control or Command + G, 52 frequency separation. 53 And here's our background image. 54 All right, so let's click up here. 55 I am not going to use all this image. 56 So this was shot with a wide angled lens. 57 And so normally I like to combine wide angled lens shots 58 with an image that I shoot in a studio 59 with a wider angled lens. 60 Now this is a compressed shot. 61 And so normally this would not work at all in my opinion, 62 because what happens is you get this distortion, 63 especially here on the edges of the image 64 where it's like bowing, 65 and the image of her obviously is quite compressed. 66 So how I work around that 67 is I'm gonna take this to the bottom, 68 and I know that I only want this to be part of the image. 69 So I want this to be kind of lower, 70 but I'm gonna stretch this so that 71 those edges, 72 those really distorted edges kind of just go away. 73 They're not quite as obvious. 74 And I would say about there, 75 'cause the rest of the image, 76 I kind of want to have like these clouds. 77 Sorry, my cat is fascinated right now. 78 I just, I wanna fill up the rest of this area with clouds, 79 because I want the background that she's going 80 to be going into to be kind of an afterthought. 81 So I'm just gonna click here, 82 and I'm gonna create a layer mask. 83 Actually, first things first. 84 So Photoshop has this really cool tool 85 called select subject. 86 So I'm going to, 87 there we go. 88 Make sure I'm not sitting on my background image. 89 Let's see here, where do we have, 90 well, let's just make a good layer to select from, 91 because if we pick this first layer here, 92 we're gonna wind up getting all of these hairs 93 and that string and stuff like that. 94 So we can delete this after. 95 Alt + Control + Shift + E, 96 merge everything up. 97 And then from here there is a select subject tool. 98 So sorry, I had to look around my cat, 99 my cat's head, move your head, honey. 100 Thank you, all right. 101 So right here it says a quick selection tool. 102 And then I'm gonna go select subject. 103 So this is only gonna work if you're actually 104 on the image itself. 105 If I was on the landscape image, 106 you see here select subject and select mask doesn't exist. 107 Also it was turned off. 108 But yeah, anyways, 109 it doesn't work as well. 110 So here we have our subject selected. 111 So I'm gonna go select and mask. 112 And basically all I'm doing here 113 is just smoothing it a little bit. 114 That's kind of the rest of it. 115 I'm gonna do the fine detail corrections manually. 116 Some people like to use the refine edge tool, 117 but with this headdress, 118 I actually tried it before doing this tutorial, 119 and it does not work. 120 It epically fails. 121 So these tools work really well 122 if you photograph on a contrasty background. 123 When we photographed these, 124 our only option was dark gray paper, 125 so that's what we shot it on. 126 So I'm just gonna hit okay, 127 so this is our selection. 128 It's not bad. 129 There's gonna be a bunch of stuff that we have to fix, 130 but it's also pretty good. 131 This used to be a lot worse than it is now. 132 So just gonna select onto the landscape image here 133 and hit that little mask. 134 And what this does is it is an inverse selection 135 of the mask. 136 So what I wanna do now 137 is I'm just gonna go Control or Command + I, 138 and that's going to invert the selection, 139 which is great. 140 And now because I am, 141 well, no, we'll fix the mask first 142 before we go in and fix anything else. 143 Now, how am I gonna know if my mask is any good? 144 Well, I'm gonna pull over another one of the cloud images. 145 Let's grab this guy here. 146 So these are one of the clouds 147 that I was thinking about using, 148 and this is a little more compressed. 149 I'm gonna bring up the shadows a little bit 150 and reduce the clarity a tiny bit 151 and just hit open image. 152 Same thing as before, 153 select all, copy, 154 paste. 155 And we're gonna move this up here. 156 And so again, I'm gonna have to stretch this a little bit. 157 I realize it's like a pretty big faux pas 158 to be stretching images, but you know what? 159 The re-rendering is getting pretty good. 160 I'm just gonna turn that down a little bit, the visibility, 161 just so I can see what's going on. 162 And to copy up this mask, 163 it's actually pretty easy. 164 Just go Alt, click and drag. 165 So if I was super stressed about this or whatever, 166 and I didn't wanna have like multiple images or whatever, 167 because I mean, once I do one mask, 168 obviously I'm gonna wanna copy it over to do the other. 169 So what I can do, 170 and I have done in the past 171 is I just take the whole, 172 oops, unlink the mask, 173 take this whole layer, 174 and I'll just stretch it just to fill the scene, 175 just so that I can see what I'm working on. 176 I'm just gonna delete this layer afterwards, 177 because it's definitely more distorted than I would like. 178 I'm holding the Shift key 179 to make the 180 shape change. 181 Okay, so this is what we are looking at. 182 So it's not too bad. 183 There's a little bit of haloing going on here, 184 that's super normal. 185 I am just gonna grab this smudge tool here, 186 and I'm just gently gonna push in. 187 I selecting on the mask layer itself. 188 And I'm just ever so carefully pushing in 189 that mask a little bit 190 along the back of the material. 191 So I could go in and do this like manually by hand 192 and like carefully go around all these edges, 193 but it's gonna take a ton of time and a bunch of work, 194 and I don't wanna. 195 So just gently 196 snugging that in a little bit. 197 Knowing that I'm gonna have to go in here like these spots 198 and fix them manually. 199 All right, let's grab our brush tool, 200 the letter B, 201 and let's turn this down a tiny bit, 202 just so that we can see what's going on underneath. 203 So obviously this mask has gone over some of her finger, 204 so I am just gonna get rid of that. 205 Get right under here in the corners. 206 Again, I'm super sorry about the fans. 207 They scream super loud. 208 So I'm not typically a fan of the big round brush, 209 because oftentimes what you get 210 is this like haloing effect around edges. 211 It's a lot of people, 212 when I am sent images or composites to critique, 213 it's usually pretty obvious that they're, 214 they are using a soft round brush, 215 and it always looks funky. 216 It never really translates perfectly well. 217 So I try to avoid using it as much as possible. 218 Now if you wanna make a straight line like that, 219 it's super easy, you're gonna click one spot, 220 hold Shift, 221 and then click the next. 222 So it just creates these really nice straight lines, 223 works great for parts of fabric like this. 224 And there's this little spot here on the finger. 225 Get rid of that, 226 and this little tiny dip down here. 227 Oops, wrong direction. 228 There we go. 229 Okay, now, all right, so up here at the front 230 of her dress, 231 there is 232 this. 233 So we'll just go through here, 234 and again, 235 fix this up. 236 And I'm not using 100% hardness on my brush. 237 I am adjusting that on my keyboard as well. 238 I reprogrammed my keyboard 239 in the preferences and the settings, 240 so that the buttons are all where I would like them to be, 241 so that I can do this really, really quickly 242 without having to go up here 243 and adjust my hardness settings 244 and soft and brush sizes and stuff. 245 You can do that all manually in Photoshop, 246 which saves a ton of time, 247 especially when you count how many hours you 248 wind up doing this for. 249 Okay, 250 missing that. 251 So we're starting to get into the area 252 where like textured brushes are probably 253 gonna start coming in handy. 254 Like I said, I'm not a huge fan of the soft round brush. 255 It's not really my friend. 256 Or even the 75% hardness round brush. 257 By the way, if you ever accidentally hit Caps 258 on your keyboard, the little X there shows up. 259 That took me years to figure out what was going on. 260 I would just like wind up rebooting my computer 261 whenever that was happening. 262 I was just like, oh my God, this is the worst. 263 Anyways, if that's happening to you, it's the Caps button. 264 And you're not the only one. 265 All righty, so that's done. 266 Now there's all these feathers. 267 Let's take a look back here. 268 So we have this little bit here, 269 I'm just handling the lower part of the image first. 270 I really wanna make sure, oops, 271 where is that coming from? 272 Oh! 273 Got it. 274 You ever look at something, 275 especially when you're dealing with a bunch of grays, 276 you're like, what is this spot? 277 Anyways, that's part of the background. 278 So we have to put, 279 we have to mask in some of the sky. 280 So this is like if you really get into fantasy photography, 281 you're gonna spend a bunch of time doing this, 282 like all the time. 283 You're gonna be masking out wig lines 284 and you know, synthetic elf ear, 285 prosthetics and horns and stuff. 286 And you're just gonna be constantly fixing this stuff. 287 And if you wanna do fantasy work, 288 you just kind of suck it up. 289 Just making sure I fix up that back there. 290 It's part of the green. 291 Now, how much of that is there? 292 Ah, see there's that little bit. 293 So turning on and off the layer 294 that you're gonna be masking from is super key. 295 All right, clean this up. 296 Do-do-do-do. 297 Don't paint over that area that I just masked. 298 What about this? 299 Goes like that there. 300 These tiny little details, 301 yikes. 302 But it looks so pretty. 303 Okay. 304 Yeah, that one's spot there is a little funky, 305 so I'm just gonna grab my smudge, 306 and then I'm gonna sharpen it a little bit, 307 so you can smudge and sharpen masks, 308 just so you know. 309 So if your mask edges are getting a little soft 310 or they're not quite in the right spot, 311 smudge 'em a bit, sharpen him a bit. 312 It works pretty good. 313 All right. 314 So, 315 I do believe we are good on the dress. 316 Now we get to do the super fun stuff, 317 which is all the feathers, 318 and there's a bunch of them. 319 So how we handle this, 320 first of all, you need your legacy brushes. 321 Legacy brushes are awesome. 322 Photoshop had them for years 323 and then they got rid of them for some reason, 324 but they only hid them. 325 So you go to this little gear here 326 and then you load your legacy brushes, 327 which is great. 328 And then basically what it does is it loads legacy brushes 329 in your default brushes. 330 And what I wanna do is I use these brushes a lot. 331 So in this case, normally with hair, 332 I use with splatter brushes, 333 but with 334 feathers, 335 I like to use these chalk brushes. 336 Oh my God, I'm on the smudge tool. 337 Yeah, so that's super fun 338 if you change your, 339 change your brush without changing your tool, that's great. 340 So let's try chalk 44. 341 There we go, that's better. 342 Yeah, always check. 343 Always check. 344 But yeah, so the reason why I like these 345 is that they have texture, right? 346 So these feathers have tons 347 of like rectangular angular textures to them. 348 And the things that will sell a really great composite 349 is paying attention to these teeny tiny little details. 350 Like this spot right here that I am totally going over. 351 I'll just invert that 352 and straighten it back out a little. 353 Because I mean, I think it was Joel Grimes 354 that said all you're trying to do is sell the fake. 355 Well, that's what we're trying to do here 356 is we're just trying to sell the fake. 357 Maybe I'll just grab a different brush here. 358 36, that's a little more like it. 359 There we go. 360 So this is gonna take a while, 361 so what I'm gonna do here is I'm just going to 362 record the screen and I'm gonna speed this up for you. 363 Otherwise, you're gonna be sitting here watching me 364 mask this out for like an hour and a half, 365 and nobody wants to watch that. 366 So I am going to start masking this thing up, 367 and I will speed this up for you, 368 so that however long this takes 369 is going to turn into a lot less time for you. 370 All right, 371 done 372 finally. 373 Okay, let's take a quick look here and see what we have got. 374 So this is all looking pretty good, 375 edges are not too bad. 376 The thing I don't like about auto select edges 377 is they get these kind of like, you know, 378 in comparison to here, 379 they get these soft edges and stuff, 380 but for the sake of it not being a million hours of masking, 381 we're gonna call it good just as that is. 382 So now I want to save this mask. 383 So I'm gonna go Alt, click and drag, 384 and I'm gonna name this mask. 385 So this means that this layer is exactly the perfect mask, 386 it's exactly how I want it to be. 387 So if going forward if I screw anything up, 388 this mask will always be there. 389 So I will not touch this mask, 390 I will not adjust this mask 391 unless there's something like really glaringly obvious 392 that I missed. 393 So from here, I'm gonna click and drag this other mask here. 394 Click, drag. 395 I would say replace layer mask, yes. 396 And I'm gonna turn that off. 397 Turn this on. 398 Visibility up. 399 So now we are ready 400 to start blending our backdrop pieces together.

Class Materials

Bonus Materials with Purchase


Ratings and Reviews

a Creativelive Student

An excellent workthrough of Renee's compositing process. Thoroughly recommended, with lots of really specific, pragmatic and useful tips, along with pointers on style and considerations for working with artists, models and the whole team - Great!

Alvin Lipscomb

She is awesome, This is a great lesson using Photoshop. Seeing the different brushes she used on her mask was an eye-opener and will definitely use for working on a mask's layer. Right there is a reason to watch this lesson. 🔥 Do yourself a favor and watch this lesson.

pat cash

love her I got all her workshops - she is intense and inspiring

Student Work