The Matte in the Nuker Mask
Before I start, I could not remember who and where did I get the advice to avoid relying the mask input in Nuke.
Just a nagging feeling when I’m new to Nuke and in the end, abuse the mask function a lot where applicable.
So the idea of masking in Nuke is similar to the Mask/Track Matte function in After Effects. If you understand how alpha works, masking should be straight forward.
Quick checklist for Masking in Nuke
- Masking uses the same method like how alpha works. White is opaque while black is transparent.
- Nuke allows you to choose your desire channel when applying a mask. You can use existing channel from upstream or if the node supports it, the Mask input.
- Be warned that masking is not recommended for some effects which I’ll explained more below.
How masking works
^The above GIF shows a straightforward process of using the mask input in Nuke. If you see a tiny arrowhead at the side of a node, 99.99% that it will be for mask input.
Choosing your channels
If you planning to reuse a set of masks frequently in your comp, I recommend creating a set of channels/layers and copy it to the node stream.
When not to use mask
As shown in the GIF, be very careful when you think of masking an effect like Defocus.
Soft edges in the mask will not blend the effect properly as it combines the the Defocus effect and the original plate. So instead of a proper transition of strong to softer blurring, we see the part of the blur effect being composite back onto the original plate.
If the node doesn’t support channel that drives the amount of effect (e.g. Defocus, Blur), look for alternative node instead.
Here’s the sample script use for this demo
set cut_paste_input [stack 0]
version 9.0 v5b43
StickyNote {
inputs 0
name StickyNote1
label "You can use this as\na form of garbage\nmatte through this method.\n\nJust make sure to label\nproperly if you work in a\nteam environment."
note_font "Verdana Italic"
note_font_size 16
selected true
xpos 258
ypos 97
}
add_layer {taukeke taukeke.mask none taukeke.triangle}
Roto {
inputs 0
output {taukeke.mask none -taukeke.triangle}
curves {{{v x3f99999a}
{f 0}
{n
{layer Root
{f 2097152}
{t x44700000 x44070000}
{a pt1x 0 pt1y 0 pt2x 0 pt2y 0 pt3x 0 pt3y 0 pt4x 0 pt4y 0 ptex00 0 ptex01 0 ptex02 0 ptex03 0 ptex10 0 ptex11 0 ptex12 0 ptex13 0 ptex20 0 ptex21 0 ptex22 0 ptex23 0 ptex30 0 ptex31 0 ptex32 0 ptex33 0 ptof1x 0 ptof1y 0 ptof2x 0 ptof2y 0 ptof3x 0 ptof3y 0 ptof4x 0 ptof4y 0 pterr 0 ptrefset 0 ptmot x40800000 ptref 0}
{curvegroup Ellipse1 512 bezier
{{cc
{f 8192}
{px x42140000
{xc1ee9640 0}
{x44450000 x43620000}
{x41ee9640 0}
{0 xc20256d4}
{x44528000 x438e8000}
{0 x420256d8}
{x41ee9640 0}
{x44450000 x43ac0000}
{xc1ee9640 0}
{0 x420256d8}
{x44378000 x438e8000}
{0 xc20256d4}}}
{cc
{f 8192}
{p
{{{x42140000 xc1ee9640}}
{{x42140000 0}}}
{{{x42140000 0}}
{{x42140000 0}}}
{{{x42140000 x41ee9640}}
{{x42140000 0}}}
{{{x42140000 0}}
{{x42140000 xc20256d4}}}
{{{x42140000 0}}
{{x42140000 0}}}
{{{x42140000 0}}
{{x42140000 x420256d8}}}
{{{x42140000 x41ee9640}}
{{x42140000 0}}}
{{{x42140000 0}}
{{x42140000 0}}}
{{{x42140000 xc1ee9640}}
{{x42140000 0}}}
{{{x42140000 0}}
{{x42140000 x420256d8}}}
{{{x42140000 0}}
{{x42140000 0}}}
{{{x42140000 0}}
{{x42140000 xc20256d4}}}}}}
{tx x42140000 x44450000 x438e8000}
{a bm x41400000 osw x41200000 osf 0 str 1 spx x44700000 spy x44070000 sb 1 ltn x42140000 ltm x42140000 tt x40e00000}}
{curvegroup Bezier4 512 bezier
{{cc
{f 8192}
{px x42140000
{0 0}
{x44a90000 x43ee0000}
{0 0}
{0 0}
{x44b54000 x43ee8000}
{0 0}
{0 0}
{x44b54000 x43530000}
{0 0}
{0 0}
{x44c16000 x439d0000}
{0 0}
{0 0}
{x44cb8000 x439d8000}
{0 0}
{0 0}
{x44bd8000 x432c0000}
{0 0}
{0 0}
{x44c78000 x42800000}
{0 0}
{0 0}
{x44bdc000 x42240000}
{0 0}
{0 0}
{x44b62000 x43010000}
{0 0}
{0 0}
{x44b5e000 x42240000}
{0 0}
{0 0}
{x44a84ccd x421d9998}
{0 0}}} idem}
{tx x42140000 x44b94e8c x435245d1}
{a osw x41200000 osf 0 str 1 spx x44700000 spy x44070000 sb 1 ltn x42140000 ltm x42140000 tt x40800000}}
{curvegroup Bezier3 512 bezier
{{cc
{f 8192}
{px x42140000
{0 0}
{x44930000 x43ba0000}
{0 0}
{0 0}
{x449ac000 x43ba0000}
{0 0}
{x42ac0000 x40800000}
{x448c0000 x43e60000}
{xc2ac0000 xc0800000}
{xc1b00000 x42480000}
{x44768000 x43a50000}
{x41b00000 xc2480000}
{xb9000000 x4257ffe8}
{x448ec000 x435e0000}
{0 xc2dc0000}
{x41f00000 xc2840000}
{x44834000 x43120000}
{xc1f80000 1}
{0 0}
{x44748000 x43140000}
{0 0}
{xc2ac0000 xc0000000}
{x4486c000 x42780000}
{x42ac0000 x40000000}
{xc1600000 xc2400000}
{x44964000 x43280000}
{x41600000 x42400000}
{x42e80000 xc1e00000}
{x448a0000 x439e0000}
{xc2e80000 x41e00000}
{xc1c15480 x38000000}
{x448a0000 x43ca0000}
{x42900000 0}}} idem}
{tx x42140000 x448a62e9 x43885d17}
{a osw x41200000 osf 0 str 1 spx x44700000 spy x44070000 sb 1 ltn x42140000 ltm x42140000 tt x40800000}}
{curvegroup Bezier2 512 bezier
{{cc
{f 8192}
{px x42140000
{0 0}
{x44558000 x43c30000}
{0 0}
{0 0}
{x44508000 x43e10000}
{0 0}
{0 0}
{x44668000 x43e30000}
{0 0}
{0 0}
{x446a0000 x431c0000}
{0 0}
{0 0}
{x445b0000 x431a0000}
{0 0}
{0 0}
{x44578000 x434c0000}
{0 0}
{x42780000 xc1c00000}
{x44348000 x43260000}
{xc2780000 x41c00000}
{xc2c40000 xc1a00000}
{x442c0000 x43c70000}
{x42c40000 x41a00000}}} idem}
{tx x42140000 x44513000 x43944000}
{a osw x41200000 osf 0 str 1 spx x44700000 spy x44070000 sb 1 ltn x42140000 ltm x42140000 tt x40800000}}
{curvegroup Bezier1 512 bezier
{{cc
{f 8192}
{px x42140000
{0 0}
{x430e0000 x43a20000}
{0 0}
{0 0}
{x43580000 x43ac0000}
{0 0}
{0 0}
{x43740000 x438b0000}
{0 0}
{xc2a80004 x427e6660}
{x43c56667 x43a13334}
{x421cccc8 x42ef3330}
{xc1900000 x41c00000}
{x44048000 x43b30000}
{x41900000 xc1c00000}
{0 0}
{x44180000 x432a0000}
{0 0}
{0 x40800000}
{x440c8000 x43200000}
{0 xc0800000}
{x41e00000 x42b80000}
{x43dd0000 x43880000}
{xc0c00000 xc0000000}
{0 0}
{x43d90000 x436e0000}
{0 0}
{0 0}
{x43be0000 x43660000}
{0 0}
{x42080000 x41000000}
{x43a2cccb x43853334}
{xc2080000 xc1000000}
{xc1c00000 x42600000}
{x43a90002 x42e1999c}
{xc22b3330 xc1a00000}
{0 0}
{x437b333a x428d999b}
{0 0}}} idem}
{tx x42140000 x43bb1b91 x437656a5}
{a osw x41200000 osf 0 str 1 spx x44700000 spy x44070000 sb 1 ltn x42140000 ltm x42140000 tt x40800000}}}}}}
toolbox {selectAll {
{ selectAll str 1 ssx 1 ssy 1 sf 1 }
{ createBezier str 1 ssx 1 ssy 1 sf 1 sb 1 tt 4 }
{ createBezierCusped str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ createBSpline str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ createEllipse str 1 ssx 1 ssy 1 sf 1 sb 1 tt 7 }
{ createRectangle str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ createRectangleCusped str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ brush str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ eraser src 2 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ clone src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ reveal src 3 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ dodge src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ burn src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ blur src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ sharpen src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ smear src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
} }
toolbar_brush_hardness 0.200000003
toolbar_source_transform_scale {1 1}
toolbar_source_transform_center {960 540}
colorOverlay {0 0 0 0}
blending_mode minus
lifetime_type "all frames"
lifetime_start 37
lifetime_end 37
motionblur_shutter_offset_type centred
source_black_outside true
name Roto2
selected true
xpos 280
ypos 327
}
set N82445c00 [stack 0]
Dot {
name Dot14
label "For this example, I created a\nnew channel called \"taukeke\"\nwhich I need to copy its layer\ninto the stream using the\nCOPY node for use later\nin the downstream"
note_font "Verdana Bold Bold Bold Bold"
note_font_size 16
note_font_color 0xffffffff
selected true
xpos 314
ypos 463
}
Dot {
name Dot8
selected true
xpos 314
ypos 574
}
push $N82445c00
Roto {
inputs 0
output alpha
curves {{{v x3f99999a}
{f 0}
{n
{layer Root
{f 2097152}
{t x44700000 x44070000}
{a pt1x 0 pt1y 0 pt2x 0 pt2y 0 pt3x 0 pt3y 0 pt4x 0 pt4y 0 ptex00 0 ptex01 0 ptex02 0 ptex03 0 ptex10 0 ptex11 0 ptex12 0 ptex13 0 ptex20 0 ptex21 0 ptex22 0 ptex23 0 ptex30 0 ptex31 0 ptex32 0 ptex33 0 ptof1x 0 ptof1y 0 ptof2x 0 ptof2y 0 ptof3x 0 ptof3y 0 ptof4x 0 ptof4y 0 pterr 0 ptrefset 0 ptmot x40800000 ptref 0}
{curvegroup Bezier5 512 bezier
{{cc
{f 8192}
{px 1
{0 0}
{x44576000 x44214000}
{0 0}
{0 0}
{x4461a000 x43ecc000}
{0 0}
{0 0}
{x44562000 x43ea8000}
{0 0}}} idem}
{tx 1 x445a6000 x44044aab}
{a bm x41400000 osw x41200000 osf 0 str 1 spx x44700000 spy x44070000 sb 1 tt x40800000}}
{curvegroup Bezier4 512 bezier
{{cc
{f 8192}
{px 1
{0 0}
{x44b2c000 x44638000}
{0 0}
{0 0}
{x44be0000 x44670000}
{0 0}
{0 0}
{x44c40000 x44180000}
{0 0}
{0 0}
{x44d18000 x44668000}
{0 0}
{0 0}
{x44de0000 x446c0000}
{0 0}
{0 0}
{x44cdc000 x44100000}
{0 0}
{0 0}
{x44e50000 x43880000}
{0 0}
{0 0}
{x44d50000 x43880000}
{0 0}
{0 0}
{x44c50000 x44040000}
{0 0}
{0 0}
{x44c64000 x436c0000}
{0 0}
{0 0}
{x44bb0000 x436c0000}
{0 0}}} idem}
{tx 1 x44ca62e9 x44121746}
{a osw x41200000 osf 0 str 1 spx x44700000 spy x44070000 sb 1 tt x40800000}}
{curvegroup Bezier3 512 bezier
{{cc
{f 8192}
{px 1
{0 0}
{x4498c000 x44348000}
{0 0}
{0 0}
{x44a4c000 x44390000}
{0 0}
{x42f00000 x41c00000}
{x448c8000 x44540000}
{xc2f00000 xc1c00000}
{xc2f80000 x40800000}
{x44878000 x44080000}
{x42f80000 xc0800000}
{xc1f00000 x42a80000}
{x44a04000 x43e60000}
{x41f00000 xc2a80000}
{0 0}
{x448a4ccd x43903333}
{0 0}
{0 0}
{x448dcccd x43686666}
{0 0}
{x42280000 xc2e40000}
{x44ac4000 x43e50000}
{xc2280000 x42e40000}
{x42f00000 xc1900000}
{x448c8000 x441a0000}
{xc2f00000 x41900000}
{xc2180000 x40c00000}
{x448ec000 x443f0000}
{x42180000 xc0c00000}}} idem}
{tx 1 x44961333 x440ee666}
{a osw x41200000 osf 0 str 1 spx x44700000 spy x44070000 sb 1 tt x40800000}}
{curvegroup Bezier2 512 bezier
{{cc
{f 8192}
{px 1
{0 0}
{x44488000 x44478000}
{0 0}
{0 0}
{x443a8000 x43400000}
{0 0}
{0 0}
{x44488000 x43440000}
{0 0}
{0 0}
{x44550000 x43ca0000}
{0 0}
{0 0}
{x44668000 x43d00000}
{0 0}
{0 0}
{x44748000 x43460000}
{0 0}
{0 0}
{x4483c000 x43520000}
{0 0}
{0 0}
{x445b8000 x44498000}
{0 0}}} idem}
{tx 1 x445bd000 x43c94000}
{a osw x41200000 osf 0 str 1 spx x44700000 spy x44070000 sb 1 tt x40800000}}
{curvegroup Bezier1 512 bezier
{{cc
{f 8192}
{px 1
{0 0}
{x43280000 x44510000}
{0 0}
{0 0}
{x43830000 x44568000}
{0 0}
{0 0}
{x43ca0000 x44180000}
{0 0}
{0 0}
{x43db0000 x445e0000}
{0 0}
{0 0}
{x441c8000 x44600000}
{0 0}
{0 0}
{x44210000 x430a0000}
{0 0}
{0 0}
{x44050000 x430e0000}
{0 0}
{0 0}
{x44050000 x43ff0000}
{0 0}
{0 0}
{x43dd0000 x43c90000}
{0 0}
{0 0}
{x43b20000 x44030000}
{0 0}
{0 0}
{x439d0000 x432c0000}
{0 0}
{0 0}
{x43420000 x43340000}
{0 0}}} idem}
{tx 1 x43ccaaab x44003555}
{a osw x41200000 osf 0 str 1 spx x44700000 spy x44070000 sb 1 tt x40800000}}}}}}
toolbox {selectAll {
{ selectAll str 1 ssx 1 ssy 1 sf 1 }
{ createBezier str 1 ssx 1 ssy 1 sf 1 sb 1 tt 4 }
{ createBezierCusped str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ createBSpline str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ createEllipse str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ createRectangle str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ createRectangleCusped str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ brush str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ eraser src 2 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ clone src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ reveal src 3 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ dodge src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ burn src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ blur src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ sharpen src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
{ smear src 1 str 1 ssx 1 ssy 1 sf 1 sb 1 }
} }
toolbar_brush_hardness 0.200000003
toolbar_source_transform_scale {1 1}
toolbar_source_transform_center {960 540}
colorOverlay {0 0 0 0}
blending_mode minus
lifetime_type "all frames"
motionblur_shutter_offset_type centred
source_black_outside true
name Roto1
selected true
xpos 305
ypos -128
}
Dot {
name Dot12
label "The MASK\nroto shape output\nas alpha"
note_font "Verdana Bold Bold Bold"
note_font_size 16
note_font_color 0xffffffff
selected true
xpos 339
ypos -54
}
Dot {
name Dot11
selected true
xpos 339
ypos 30
}
Constant {
inputs 0
channels alpha
color 1
name Constant1
selected true
xpos -367
ypos -262
}
ColorBars {
inputs 0
PAL true
name ColorBars1
selected true
xpos -221
ypos -384
}
Copy {
inputs 2
from0 rgba.alpha
to0 rgba.alpha
name Copy1
selected true
xpos -221
ypos -238
}
Dot {
name Dot9
label "I copy a white alpha\nchannel into this stream\nsince the ColorBars don't\ngenerate it's own Alpha\nchannel"
note_font "Verdana Bold"
note_font_size 16
note_font_color 0xffffffff
selected true
xpos -187
ypos -125
}
Dot {
name Dot1
selected true
xpos -187
ypos 30
}
push $cut_paste_input
Ramp {
output {rgba.red rgba.green rgba.blue -rgba.alpha}
replace true
p0 {100 -12}
p1 {100 908}
type plinear
color {1 0.417224 0 1}
color_panelDropped true
name Ramp1
selected true
xpos 65
ypos -214
}
Dot {
name Dot10
label "Just a regular\nramp aka gradient"
note_font "Verdana Bold Bold"
note_font_size 16
note_font_color 0xffffffff
selected true
xpos 99
ypos -129
}
Merge2 {
inputs 2+1
name Merge1
selected true
xpos 65
ypos 26
}
Dot {
name Dot13
label "The Merge node\nis masked using\nthe mask input\nconnected to the\nMASK roto"
note_font "Verdana Bold Bold Bold"
note_font_size 16
note_font_color 0xffffffff
selected true
xpos 99
ypos 127
}
Grade {
inputs 1+1
white {0 0.556425 1 1}
white_panelDropped true
gamma 5
maskChannelMask taukeke.mask
name Grade2
selected true
xpos 65
ypos 333
}
Copy {
inputs 2
from0 -taukeke.mask
to0 -taukeke.mask
channels taukeke
name Copy2
selected true
xpos 65
ypos 557
}
Dot {
name Dot7
selected true
xpos 99
ypos 659
}
ColorWheel {
inputs 0
gamma 0.45
name ColorWheel1
selected true
xpos 862
ypos -225
}
Dot {
name Dot15
label "ColorWheel is a great\nexample to demo the\nRED GREEN BLUE\nchannel as mask"
note_font "Verdana Bold Bold Bold Bold"
note_font_size 16
note_font_color 0xffffffff
selected true
xpos 896
ypos -66
}
Dot {
name Dot2
selected true
xpos 896
ypos 55
}
set N10566800 [stack 0]
Dot {
name Dot4
selected true
xpos 1013
ypos 55
}
Dot {
name Dot16
label "The RED channel is\nused here as the\nmask for Grade1"
note_font "Verdana Bold Bold Bold Bold Bold"
note_font_size 16
note_font_color 0xffffffff
selected true
xpos 1013
ypos 124
}
Dot {
name Dot3
selected true
xpos 1013
ypos 196
}
set N10566400 [stack 0]
Dot {
name Dot17
label "The BLUE channel is\nused here as the\nmask for ColorCorrect1"
note_font "Verdana Bold Bold Bold Bold Bold Bold"
note_font_size 16
note_font_color 0xffffffff
selected true
xpos 1013
ypos 270
}
Dot {
name Dot5
selected true
xpos 1013
ypos 360
}
set N9e5df800 [stack 0]
Dot {
name Dot18
label "The GREEN channel is\nused here as the\nmask for HueShift1\n\nSo you noticed the copy\npasta I mean paste\ndescription. :)"
note_font "Verdana Bold Bold Bold Bold Bold Bold"
note_font_size 16
note_font_color 0xffffffff
selected true
xpos 1013
ypos 429
}
Dot {
name Dot6
selected true
xpos 1013
ypos 517
}
push $N9e5df800
push $N10566400
push $N10566800
Grade {
inputs 1+1
multiply {{curve x1 0 x50 1 x100 0}}
maskChannelMask rgba.red
name Grade1
selected true
xpos 862
ypos 192
}
ColorCorrect {
inputs 1+1
saturation {{curve x1 1 x50 0 x100 1}}
maskChannelMask rgba.green
name ColorCorrect1
selected true
xpos 862
ypos 356
}
HueShift {
inputs 1+1
hue_rotation {{frame*2}}
maskChannelMask rgba.blue
name HueShift1
selected true
xpos 862
ypos 513
}
Merge2 {
inputs 2
also_merge taukeke
name Merge2
selected true
xpos 862
ypos 655
}
Dot {
name Dot19
label "The Defocus node uses the\ntaukeke.mask channel as the\nmask input."
note_font "Verdana Bold Bold Bold Bold Bold Bold Bold"
note_font_size 16
note_font_color 0xffffffff
selected true
xpos 897
ypos 754
}
Defocus {
channels rgb
defocus 30
ratio 2.65
maskChannelInput taukeke.mask
name Defocus1
selected true
xpos 862
ypos 851
}
