Tweaking Camera Tracker in NukeX

This tutorial explains the common functions in tweaking Camera Tracker in NukeX.

Tracking a shot using CameraTracker can be either easy or difficult depending on many factors such as the lack of usable tracking markers, motion blur, rack focusing/DOF, etc.

There are various 3D tracking software such as PFTrack, boujou, Syntheyes, 3DEqualizer and After Effects own built-in camera tracking, I personally prefer Camera Tracker as it integrates well in NukeX for seamless compositing (which Nuke supports true 3D compositing unlike AE approach of 2.5D).

What should I tweak in CameraTracker?

In a best case scenario, CameraTracker with the default settings can ideally gives a decent result of 3D camera tracking but let’s face it, most of the shots are often less than ideal for the default settings.

Before I begin, the following settings is taken from CameraTracker in NukeX 8.0. Don’t be surprised if the UI looks different if you’re using older version for special reasons.

CameraTracker Main UI

• Source: The shot that you want to track.
• Mask: You can pipe in a roto/matte that is use to mask out moving objects such as talents, vehicles or reflections/highlight. In some cases, you want to mask out mirror too as it will screw up with the tracking. Seriously.
• Range: Either Input, Global, Custom or Reference Images. By default it should be the input which you can change to your requirement.
• Camera Motion: This gets tricky as you need to know the type of camera motion that you’re tracking. There is Free Camera, Linear, Planar and Rotation only. You have to play around with this settings if the result is a less than desire voodoo magic.
• Lens Distortion: Unknown Lens is ought to be enough for my case. In some cases where you’re tackling with really wide angle lenses, you can try Undistort Input which will automatically assist in figuring the amount of distortion in the shot. Unless the shot is free of any lens distortion, choose No Lens Distortion.
• Focal Length: Again, Unknown Constant works really well for me. You can play around with this value if you have a known focal length or varying values due to zooming in and out.
• Film Back Preset: Since the current project I’m working on deals with shots taken with the Canon 550D, I simply chose the appropriate settings. Be warned that DSLR uses a crop film back size when recording video versus taking still.
• Film Back Size: Self explanatory. If you are unsure of the size, you can leave it at default value.

Analysis

• Track: Track the footage
• Update Track: If there is any adjustment to any settings after the initial tracking, you can use Update Track although I usually will start again from scratch.
• Clear Tracks: Clear all tracking data. If you ever noticed your Nuke script size is large, the tracking data from CameraTracker can be one of the culprit. Use this if you have redundant tracking data from several CameraTracker nodes.
• Solve: Solve the camera movement from the tracking data.
• Update Solve: Unlike Update Track, I often use this after deleting poor solved tracking data to minimize the error in the final camera movement.
• Clear Solve: If all things goes to hell, you can nuke it using this button.
• Error: Preferably try to keep it under 1.0 but since CameraTracker is a tool and not a human, be wary of the actual solved camera movement by viewing it in the 3D space.  You need to verify whether it look as close to the actual camera movement as you can get odd camera movement with a low error value.

CameraTracker Settings

Features

• Number of Features: The default is 150 and in my experience, it is the bare minimum if you want to at least get a solved camera movement. Raise it up for really complex shots or poor/low contrast shots (in the screenshot, I use 800 due to the lack of obvious tracking marker/objects).
• Detection Threshold: Increase this to lock onto a higher contrast pattern in the shot. It’s hard to give a decent default value so play with this until you get the desire result.
• Feature Separation: Lower number simply means the tracking method will clump together while higher number leads to higher separation. Again play with this value depending on the shot.
• Refine Feature Locations: Enable this if you have tracking marker or high contrast pattern which CameraTracker will lock on. In some scenario, if you have a corner of a building that is not overlapping other objects/buildings, it can help to increase the accuracy of your camera movement.
• Preview Features: Enable this to temporarily preview how CameraTracker is tracking the shot at the current frame.

Tracking

• Minimum Length: The default is 3 frames which is quite ok for most shots. Raise this for really slow camera movement and long duration shot. From my experience, I rarely adjust this settings.
• Track Threshold: Well I usually leave it at default value. Lower this value if you want CameraTracker to be more resilient in tracking a particular pattern in fast moving camera or blurry shots.
• Track Smoothness: Increase this if you want to get a smoother tracks in complex shots. Hard to comment on this since I leave it at default all the time.
• Track Consistency: This can be the huge game changer depending on the shot. Raise this if you want CameraTracker to process the tracks as consistent in position during the solving process. Particularly helpful when you have a solid flat wall with tracking markers that often ended up being non-planar in the final solved camera movement.

Solving

• Camera Motion: Since I often work with DSLR footage that is shot using a DIY steadicam rig, I usually leave this as Free Camera. Rotation only works beautifully when there is absolutely no or little parallax in the shot (which the other options will fail in tracking nodal panning shots). Linear and planar is self explanatory as usual. The former is for dolly shots using a special rig while the latter deals with movement in the X and Y axis aka no depth.
• Keyframe Spacing: I leave it at default value of 0.3 You can reduce it for fast camera movement shots to get a better result although I’ve yet to encounter such shots.
• Smoothness: Increase this to get a smoother camera movement… which I find it odd as we try our best to capture the original camera movement. Usually I leave it at 0.
• Reference Frame: Use this if you have a tricky camera movement by setting the frame number that covers the most variation in depth and parallax movement. This will tell CameraTracker to start the camera solving from that particular frame first before proceeding to other frames. Usually you need a really high number of features as enabling this can lead to a failed solve.

So what’s next?

In future updates, I’ll show examples of my work with selected shots that utilize CameraTracker.

In the mean time, here is a good overview of using CameraTracker by The Foundry.

As usual, there is plenty of video tutorials out there for CameraTracker and my post here is to share the common tweaks that I use to achieve the desire result I’m looking for.

Do not fret if you can’t get the result you’re looking for. Keep tweaking the damn values!