PostsNuke TipsDemo ReelGalleryResumeAbout
Coding in VFX and Animation Header

Coding in VFX and Animation

Huey Yeng

Coding in VFX and Animation Header

Coding to Improve Your Laziness

For the month of April, I’ll be covering about the use of coding/scripting to improve your productivity in common Digital Content Creation (DCC) like Maya, 3ds Max,  Houdini, Nuke and After Effects etc.

Typically the approach here is more like writing a script to automate the repetitive tasks.

Imagine setting up the render settings, replacing texture paths for every cuts in a production environment for a feature length film BUT having a small team of less than 10 people so manual approach is not practical.

So why work hard when you can be a more productive lazy artist with some coding!

Coding in VFX to Automagically…

The ultimate end goal for any codes (specifically scripts) that we’ll be writing is to let the computer perform the “samey” repetitive tasks.

In the screenshot below, I’ve written a MEL script in Maya which generate a UI Panel that contains many buttons that will execute the functions from an external scripts. The goal here is not only for my personal use but for my FX team members too so they can avoid doing repetitive stuff and spend more time lazing around.

Maya Script Editor and Custom UI Panel

While you can’t write a script to turn back time to tell your old self to learn programming and Nuke (hehe), here are popular examples of what scripting can do:

  1. Replacing all the textures path in a scene file that met a certain criteria
  2. Create folders if doesn’t exists when outputting a geometry cache
  3. Renaming objects names by adding prefix/suffix, version etc
  4. Converting directory path to UNC (Universal Naming Convention) for network rendering
  5. Assign randomize rock materials to dozens of rock geometry
  6. Output WIP in Quicktime format with the date/time as prefix name
  7. Lookup for a specific type of nodes and change the parameter values
  8. Setup the renderer settings with the appropriate values based on project specifications

Here’s an example of a MEL Script that I wrote for an actual project to export the characters geometry cache as FBX 2013 from Maya 2014 to Maya 2013:

// Copyright (C) 2016-2017 Huey Yeng - Developed at FLYSTUDIO
// 
// CH GeoCache to FBX for DV and various project
// For Maya 2014 only
// Last updated: 2017/01/05

// Check if local drive scene path is UNC or Drive Letter
string $user = `getenv "USER"`;
string $scenePath = dirname(`file -q -sceneName`);
$array = stringToStringArray($scenePath, "/");
$drivepath = $array[1];

// Generate Cache Directory Path after verifying UNC or local path
string $proj = "DV"; //Replace Project Name here (e.g. ZT, LF, HY, etc)
//$cacheDir = "";

if ($drivepath == "D" || $drivepath == "d") {
  $checkDirFX = $array[8];
  if ($checkDirFX == "FX") {
    global string $cacheDir;
    string $cacheDir = ("//" + $array[0] + "/D/" + $user + "/Project/" + $proj + "/Shots/" + $array[6] + "/" + $array[7] + "/FX/Caches/Maya/GeoCache/");
  } else {
    global string $cacheDir;
    string $cacheDir = ("//" + $array[0] + "/D/" + $user + "/Project/" + $proj + "/Shots/" + $array[6] + "/" + $array[7] + "/Caches/Maya/GeoCache/");
  }
} else {
  $checkDirFX = $array[7];
  if ($checkDirFX == "FX") {
    global string $cacheDir;
    string $cacheDir = ("D:/" + $user + "/Project/" + $proj + "/Shots/" + $array[5] + "/" + $array[6] + "/FX/Caches/Maya/GeoCache/");
  } else {
    global string $cacheDir;
    string $cacheDir = ("D:/" + $user + "/Project/" + $proj + "/Shots/" + $array[5] + "/" + $array[6] + "/Caches/Maya/GeoCache/");
    }
  }

//print $cacheDir;

// Create GeoCache directory if doesn't exists
$dirExists = `filetest -d $cacheDir`;
if ($dirExists == 0) {
  sysFile -makeDir $cacheDir;
}

// FUNCTION meshFBXexport - Apply Lambert to combined mesh and export at FBX 2013
global proc meshFBXexport( string $cacheDir )
{
  // Add/Remove your mesh array list here
  string $meshCombineList[] = {"cSAN", "wSPA", "cNER", "wRQN", "cVER", "wYMT", "wSNA"}; // Add your mesh object list here

  // Execute Assigned Lambert material and Delete History
  select -cl;
  for ($each in $meshCombineList) {
    string $meshSuffix = $each + "_mesh"; // The mesh object name with "_mesh" suffix
    if (`objExists $meshSuffix`) {
      select -add $meshSuffix ;
      
      hyperShade -assign lambert1;

      delete -ch; //-ch stands for construction history
    } else {
      warning("No " + $meshSuffix + " exists!\n");
    }
  }

  //FBXResetExport; // Reset existing user settings
  FBXExportInputConnections -v 1; // Boolean 0 = False, 1 = True
  FBXExportFileVersion FBX201300; // Set to FBX 2013 for Maya 2013 duh
  FBXExportScaleFactor 1.0;  // Set the scale factor. Typically Maya default unit is CM so leave this at 1.0
  FBXExportUpAxis y; // Maya default up axis is Y while 3dsmax is Z.
  FBXExportSmoothingGroups -v 1;
  FBXExportHardEdges -v 0;
  FBXExportTangents -v 0;
  FBXExportSmoothMesh -v 1;
  FBXExportInstances -v 0;
  FBXExportReferencedAssetsContent -v 0;
  string $fileName = $cacheDir + "CH_GeoCache_Mesh_v001.fbx";
  FBXExport -f $fileName -s;
  print("\nFBXs successfully written to "+$cacheDir);
}

// FUNCTION geoCacheFBX - Export CH/WPN Mesh GeoCache
global proc geoCacheFBX()
{
  //========================================================//
  //Merge cSAN and wSPA each as Individual Mesh for GeoCache//
  //========================================================//

  $start = `playbackOptions -query -minTime`;
  $end = `playbackOptions -query -maxTime`;

  // cSAN
  if( `objExists cSAN:ROOT` ) {
    global string $cacheDir;
    layerEditorLayerButtonTypeChange cSAN:GEO;

    select -r cSAN:L_pupil_render cSAN:L_iris_render cSAN:L_sclera_render cSAN:L_cornea_render cSAN:R_cornea_render cSAN:R_pupil_render cSAN:R_iris_render cSAN:R_sclera_render cSAN:eyelash_render cSAN:L_meat_render cSAN:R_meat_render cSAN:L_tear_render cSAN:R_tear_render cSAN:gumsLower_render cSAN:gumsShineLower_Spline_render cSAN:tongue_render cSAN:lowerTeeth_render cSAN:gumsUpper_render cSAN:gumsShineUpper_Spline_render cSAN:uppperTeeth_render cSAN:headGlass_render cSAN:headBack_render cSAN:head_render cSAN:hand_render cSAN:headwear_render cSAN:horn_render cSAN:nail_render cSAN:fitting_render cSAN:collar_render cSAN:underwear_render cSAN:robe_render cSAN:wear_render cSAN:muffle_render cSAN:skirt_render cSAN:pant_render cSAN:L_ankle_render cSAN:L_shoes_render cSAN:R_ankle_render cSAN:R_shoes_render ;
    polyUnite -ch 1 -mergeUVSets 1 -name "cSAN_mesh" cSAN:L_pupil_render cSAN:L_iris_render cSAN:L_sclera_render cSAN:L_cornea_render cSAN:R_cornea_render cSAN:R_pupil_render cSAN:R_iris_render cSAN:R_sclera_render cSAN:eyelash_render cSAN:L_meat_render cSAN:R_meat_render cSAN:L_tear_render cSAN:R_tear_render cSAN:gumsLower_render cSAN:gumsShineLower_Spline_render cSAN:tongue_render cSAN:lowerTeeth_render cSAN:gumsUpper_render cSAN:gumsShineUpper_Spline_render cSAN:uppperTeeth_render cSAN:headGlass_render cSAN:headBack_render cSAN:head_render cSAN:hand_render cSAN:headwear_render cSAN:horn_render cSAN:nail_render cSAN:fitting_render cSAN:collar_render cSAN:underwear_render cSAN:robe_render cSAN:wear_render cSAN:muffle_render cSAN:skirt_render cSAN:pant_render cSAN:L_ankle_render cSAN:L_shoes_render cSAN:R_ankle_render cSAN:R_shoes_render;

    cacheFile -fileName "cSAN_GeoCache_Mesh" -dir $cacheDir -startTime $start -endTime $end -format "OneFile" -cacheFormat "mcc" -points "cSAN_meshShape";
    
    select -cl ;
  } else {
    warning("No cSAN:ROOT exists");
  }


  // wSPA
  if( `objExists wSPA:ROOT` ) {
    global string $cacheDir;
    layerEditorLayerButtonTypeChange wSPA:GEO;

    select -r wSPA:meet_render wSPA:blade_render wSPA:mirror_render wSPA:gem_render wSPA:engrave_render wSPA:shank01_render wSPA:shank02_render wSPA:flower_render wSPA:skull_render ;
    polyUnite -ch 1 -mergeUVSets 1 -name "wSPA_mesh" wSPA:meet_render wSPA:blade_render wSPA:mirror_render wSPA:gem_render wSPA:engrave_render wSPA:shank01_render wSPA:shank02_render wSPA:flower_render wSPA:skull_render;

    cacheFile -fileName "wSPA_GeoCache_Mesh" -dir $cacheDir -startTime $start -endTime $end -format "OneFile" -cacheFormat "mcc" -points "wSPA_meshShape";
    
    select -cl ;
  } else {
    warning("No wSPA:ROOT exists");
  }

  // cNER
  if( `objExists cNER:ROOT` ) {
    global string $cacheDir;
    layerEditorLayerButtonTypeChange cNER:GEO;

    select -r cNER:face_render cNER:L_cornea_render cNER:L_pupil_render cNER:L_iris_render cNER:L_sclera_render cNER:R_cornea_render cNER:R_pupil_render cNER:R_iris_render cNER:R_sclera_render cNER:LT_eyelash_render cNER:LU_eyelash_render cNER:RT_eyelash_render cNER:RU_eyelash_render cNER:L_meat_render cNER:R_meat_render cNER:L_tear_render cNER:R_tear_render cNER:gumsLower_render cNER:gumsShineLower_Spline_render cNER:tongue_render cNER:lowerTeeth_render cNER:gumsUpper_render cNER:gumsShineUpper_Spline_render cNER:uppperTeeth_render cNER:coatA_render cNER:coatPartsA_render cNER:coatPartsB_render cNER:coatPartsC_render cNER:chestCover_render cNER:chestMetal_render cNER:innerHood_render cNER:inner_render cNER:zipperA_render cNER:zipperB_render cNER:innerParts_render cNER:L_hand_render cNER:accessaryA_render cNER:accessaryB_render cNER:ringA_render cNER:ringB_render cNER:innerHand_render cNER:R_hand_render cNER:handCoverA_render cNER:handCoverB_render cNER:pants_render cNER:beltMetal_render cNER:legCoverMetal_render cNER:legCover_render cNER:leg_render cNER:L_bootsMetal_render cNER:L_boots_render cNER:R_bootsMetal_render cNER:R_boots_render ;
    polyUnite -ch 1 -mergeUVSets 1 -name "cNER_mesh" cNER:face_render cNER:L_cornea_render cNER:L_pupil_render cNER:L_iris_render cNER:L_sclera_render cNER:R_cornea_render cNER:R_pupil_render cNER:R_iris_render cNER:R_sclera_render cNER:LT_eyelash_render cNER:LU_eyelash_render cNER:RT_eyelash_render cNER:RU_eyelash_render cNER:L_meat_render cNER:R_meat_render cNER:L_tear_render cNER:R_tear_render cNER:gumsLower_render cNER:gumsShineLower_Spline_render cNER:tongue_render cNER:lowerTeeth_render cNER:gumsUpper_render cNER:gumsShineUpper_Spline_render cNER:uppperTeeth_render cNER:coatA_render cNER:coatPartsA_render cNER:coatPartsB_render cNER:coatPartsC_render cNER:chestCover_render cNER:chestMetal_render cNER:innerHood_render cNER:inner_render cNER:zipperA_render cNER:zipperB_render cNER:innerParts_render cNER:L_hand_render cNER:accessaryA_render cNER:accessaryB_render cNER:ringA_render cNER:ringB_render cNER:innerHand_render cNER:R_hand_render cNER:handCoverA_render cNER:handCoverB_render cNER:pants_render cNER:beltMetal_render cNER:legCoverMetal_render cNER:legCover_render cNER:leg_render cNER:L_bootsMetal_render cNER:L_boots_render cNER:R_bootsMetal_render cNER:R_boots_render;

    cacheFile -fileName "cNER_GeoCache_Mesh" -dir $cacheDir -startTime $start -endTime $end -format "OneFile" -cacheFormat "mcc" -points "cNER_meshShape";
    
    select -cl ;
  } else {
    warning("No cNER:ROOT exists");
  }


  // wRQN
  if( `objExists wRQN:ROOT` ) {
    global string $cacheDir;
    layerEditorLayerButtonTypeChange wRQN:GEO;

    select -r wRQN:blade_render wRQN:handlelever_render wRQN:muffler_render wRQN:gold_render wRQN:handleBody_render wRQN:leather_render wRQN:grip_render ;
    polyUnite -ch 1 -mergeUVSets 1 -name "wRQN_mesh" wRQN:blade_render wRQN:handlelever_render wRQN:muffler_render wRQN:gold_render wRQN:handleBody_render wRQN:leather_render wRQN:grip_render;

    cacheFile -fileName "wRQN_GeoCache_Mesh" -dir $cacheDir -startTime $start -endTime $end -format "OneFile" -cacheFormat "mcc" -points "wRQN_meshShape";
    
    select -cl ;
  } else {
    warning("No wRQN:ROOT exists");
  }
  
  // wSNA
  if( `objExists wSNA:ROOT` ) {
    global string $cacheDir;
    layerEditorLayerButtonTypeChange wSNA:GEO;

    select -r wSNA:handCover_render wSNA:hand_render wSNA:innerHand_render ;
    polyUnite -ch 1 -mergeUVSets 1 wSNA:handCover_render wSNA:hand_render wSNA:innerHand_render;

    cacheFile -fileName "wSNA_GeoCache_Mesh" -dir $cacheDir -startTime $start -endTime $end -format "OneFile" -cacheFormat "mcc" -points "wSNA_meshShape";
    
    select -cl ;
  } else {
    warning("No wSNA:ROOT exists");
  }

  // cVER
  if( `objExists cVER:ROOT` ) {
    global string $cacheDir;
    layerEditorLayerButtonTypeChange cVER:GEO;

    select -r cVER:face_render cVER:gumsLower_render cVER:gumsShineLower_Spline_render cVER:tongue_render cVER:lowerTeeth_render cVER:gumsUpper_render cVER:gumsShineUpper_Spline_render cVER:uppperTeeth_render cVER:RT_eyelash_render cVER:RU_eyelash_render cVER:LT_eyelash_render cVER:LU_eyelash_render cVER:L_tear_render cVER:R_tear_render cVER:L_meat_render cVER:R_meat_render cVER:L_cornea_render cVER:L_pupil_render cVER:L_iris_render cVER:L_sclera_render cVER:R_cornea_render cVER:R_pupil_render cVER:R_iris_render cVER:R_sclera_render cVER:inner_render cVER:scarf_render cVER:L_hand_render cVER:R_hand_render cVER:L_glove_render cVER:R_glove_render cVER:coat_render cVER:coatButton_render cVER:shoulderButton_render cVER:L_sleeve_Button_render cVER:R_sleeve_Button_render cVER:pants_render cVER:beltBackle_render cVER:beltLeather_render cVER:L_upper_render cVER:L_lower_render cVER:L_boots_render cVER:L_bootspattern_render cVER:L_metalpattern_render cVER:R_boots_render cVER:R_bootspattern_render cVER:R_upper_render cVER:R_lower_render cVER:R_metalpattern_render ;
    polyUnite -ch 1 -mergeUVSets 1 -name "cVER_mesh" cVER:face_render cVER:gumsLower_render cVER:gumsShineLower_Spline_render cVER:tongue_render cVER:lowerTeeth_render cVER:gumsUpper_render cVER:gumsShineUpper_Spline_render cVER:uppperTeeth_render cVER:RT_eyelash_render cVER:RU_eyelash_render cVER:LT_eyelash_render cVER:LU_eyelash_render cVER:L_tear_render cVER:R_tear_render cVER:L_meat_render cVER:R_meat_render cVER:L_cornea_render cVER:L_pupil_render cVER:L_iris_render cVER:L_sclera_render cVER:R_cornea_render cVER:R_pupil_render cVER:R_iris_render cVER:R_sclera_render cVER:inner_render cVER:scarf_render cVER:L_hand_render cVER:R_hand_render cVER:L_glove_render cVER:R_glove_render cVER:coat_render cVER:coatButton_render cVER:shoulderButton_render cVER:L_sleeve_Button_render cVER:R_sleeve_Button_render cVER:pants_render cVER:beltBackle_render cVER:beltLeather_render cVER:L_upper_render cVER:L_lower_render cVER:L_boots_render cVER:L_bootspattern_render cVER:L_metalpattern_render cVER:R_boots_render cVER:R_bootspattern_render cVER:R_upper_render cVER:R_lower_render cVER:R_metalpattern_render;
    
    cacheFile -fileName "cVER_GeoCache_Mesh" -dir $cacheDir -startTime $start -endTime $end -format "OneFile" -cacheFormat "mcc" -points "cVER_meshShape";
    
    select -cl ;
  } else {
    warning("No cVER:ROOT exists");
  }

  // wYMT
  if( `objExists wYMT:ROOT` ) {
    global string $cacheDir;
    layerEditorLayerButtonTypeChange wYMT:GEO;

    select -r wYMT:swordBlade_render wYMT:swordGold_render wYMT:swordWraping_render wYMT:swordHandle_render wYMT:scabbardGold_render wYMT:scabbardBody_render ;
    polyUnite -ch 1 -mergeUVSets 1 -name "wYMT_mesh" wYMT:swordBlade_render wYMT:swordGold_render wYMT:swordWraping_render wYMT:swordHandle_render wYMT:scabbardGold_render wYMT:scabbardBody_render;

    cacheFile -fileName "wYMT_GeoCache_Mesh" -dir $cacheDir -startTime $start -endTime $end -format "OneFile" -cacheFormat "mcc" -points "wYMT_meshShape";
    
    select -cl ;
  } else {
    warning("No wYMT:ROOT exists");
  }
  
  global string $cacheDir;
  meshFBXexport($cacheDir); 

  confirmDialog
    -title "chGeoCache to FBX"
    -message "You've reached the end of the script. Please check Script Editor for additional warning"
    -icon "information";
}

Before We Get Started

Before one start to code, it is important to have access to the internet (especially Google and Youtube) when you need to lookup for a problem.

The way I work is to think up what I wanted to automate and start coding the script and execute it.

If there is an error, this is your cue to start look up for that specific error and see what others have to say.

The same goes if you’re trying to figure out a particular method like casting a list of arrays as a specific types. Try it out and obviously you’ll failed if this is your first time. Note the error message and look up with the proper keywords to help narrow down your results.

Having a rudimentary fundamentals in programming language is a plus as it will help in understanding the following:

  • Variables
  • Arrays
  • Functions
  • Syntax

Listen, listen, listen! Or Maybe Not

Depending on the software, it will have a listener which are a great tool in listening to the actions that you performed.

The following GIF example shows the Script Editor in Maya where it prints out the actions when I set up the Vray Render Settings.

Maya Script Editor Listener GIF

3ds Max does have a listener too although personally I think Maya does better in capturing more actions than 3ds Max.

Sadly, this is not true for compositing software like Nuke and After Effects (AE) although the nature of Nuke means majority of the actions/nodes are actually text scripts which you can copy to any text editor where else AE requires the user to code their scripts in Adobe ExtendScript.

Again, the listener will be your best buddy whenever you need to code… if said buddy is available to listen to your complaints.

Did You Know…?

…that coding a script is pretty straightforward despite the lengthy wall of text as seen above.

EXCEPT

When it breaks.

Troubleshooting the error and getting it to work to your requirement is the fun (and stressful) part!

For the next few articles, I’ll be covering practical example as seen in an actual production for Maya, 3ds Max, After Effects and probably Nuke although I haven’t try writing a Python or TCL script for it.

Remember that there are plentiful of great resources to jumpstart your coding skills on the internet and I’m focusing on actual use instead of a step-by-step tutorials.

Happy coding!

Recommended Links

General Coding Practice
https://learnpythonthehardway.org/book/ (A great tutorial for complete newbie to coding for general purpose and it’s Python which are supported by many DCC software!)

MaxScript for 3ds Max
https://www.scriptspot.com/bobo/mxs2/mxs_tut/lesson01.html
https://www.scriptspot.com/bobo/mel2mxs/mel2mxs.htm (For Maya MEL Script to Max MaxScript)

MEL Script for Maya
https://www.csit.parkland.edu/~dbock/Class/csc233/Lecture/MELFundamentals.html
https://github.com/BigRoy/mayaVrayCommandDocs/wiki (this is specific for Vray for Maya but super useful)
https://polygonspixelsandpaint.tumblr.com/post/102799282929 (explains the MEL whatIs command)

Social
GitHubGitHub
LinkedInLinkedIn
TwitterTwitter
PixivPixiv
VimeoVimeo
About
About meChangelog
© 2024 Huey Yeng
Back to Top
Dark BackgroundDefault Background