You control a puppet through the behaviors that are applied to it. Face tracking is an example of a behavior, and so is the automatic wiggling of vectored artwork. A behavior can have parameters for tuning it to your needs.
New puppets automatically get the following behaviors: Dragger, Eye Gaze, Face, Lip Sync, Physics, Transform, and Triggers. You can add and remove behaviors as needed.
A puppet’s behaviors are listed alphabetically, at each level of the puppet’s hierarchy, in the Properties panel.
You can add visual tags to layers and handles for easy identification and faster operation. Tags are also categorized based on their use, and you can view the tags as text or picture buttons.
Tags associated with layers and handles help you set up behaviors that control puppets.
You can select a layer and apply both layer or handle tags to it. When a handle is selected, only handle tags can be applied. When viewing tags as text, applied layer tags are blue and handle tags are yellow. When viewing tags as pictures, applied tags are blue.
The following behaviors are available:
- Auto Blink: Simulate eyeballs blinking
- Breathe: Simulate breathing
- Cycle Layers: Display layers in order
- Dragger: Offset a handle of a puppet by dragging with the mouse
- Eye Gaze: Control eye pupil movement separate from the rest of the face
- Face: Control head, eyes, eyebrows, nose, and mouth with your webcam
- Fader: Fade a group on and off via trigger
- Handle Fixer: Prevent movement of a puppet handle
- Head Turner: Switch between front, three-quarter, and profile views by turning your head using your webcam
- Layer Picker: Choose a specific layer to show or trigger
- Lip Sync: Control a puppet’s mouth with your voice
- Motion Trigger: Switch between groups based on the direction in which the parent puppet is moving.
- Nutcracker Jaw: Slide or rotate the lower jaw as you open and close your mouth or talk into the microphone.
- Particles: Control objects using physics (cannon, snowfall, etc.)
- Physics: Dangle and collide layers
- Transform: Apply basic transformations and opacity
- Triggers: Trigger artwork with custom-named triggers
- Walk: Make a character walk
- Wiggler: Make vector artwork wiggle
The behavior is now bound to the puppet. To see how a behavior works, either place the puppet in a scene, or open the scene in the Scene panel.
Some behaviors have parameters for controlling them. You can change the value of a behavior’s parameter after the behavior is applied to a puppet.
Behaviors with layer and handle parameters — such as Breathe, Dragger, Face, Head Turner, and Nutcracker Jaw — placed on different groups in a puppet’s hierarchy override the settings for the same behavior higher in the hierarchy. This capability allows you to have a behavior at the top level of the hierarchy define default settings (for example, a top-level Dragger behavior set to Return to Rest controls a character’s arm and leg group), and the same behavior applied on a group lower in the hierarchy define custom settings (for example, Dragger applied to the character’s tail set to Hold in Place).
If you need to add the same behavior to multiple selected groups, use the “+” button in the Properties panel. To view the applied behaviors for the puppet or a group, hover over its behavior icon. The list of applied behaviors appears in a tooltip.
You can remove all behaviors on a puppet or any group in it, or just at a specific group in the puppet hierarchy. To remove all behaviors from a puppet, follow these steps:
- Select the puppet track in the Timeline panel.
- Choose Puppet > Remove All Behaviors.
To remove all behaviors from a specific group on a puppet, follow these steps:
- Select a puppet in the Project panel (to remove the top-level behaviors), or a puppet group in the Puppet panel, that has behaviors applied.
- Choose Remove All Behaviors from either the Puppet menu or the “+” button menu in the Properties or Puppet panel.
Note: Any recordings associated with the removed behaviors are also removed from their scenes.
- If the parameter value at the track item level is different from (overrides the value) at the project level, a reset button (“x”) appears next to the parameter’s control. Clicking this button restores the parameter to the default value.
- For layer parameters (such as the Face behavior’s Blink and Mouth features), the parameter’s value is a set of matching puppet layers. For example, if a Smile mouth layer is found, the parameter’s value shows the path to that layer (Head/Mouth/Smile). If not found, the value is blank. You can use this information to diagnose if puppet layers are not being used as intended.
- For handle parameters (such as the Breathe behavior’s Breathe Handles; Handle Fixer, Nutcracker Jaw, and Dragger’s Target Handles; and Face’s Head Handle), the parameter’s value is a set of matching handles.
- Behaviors control the movement of handles, and the visibility of layers.
- Some behaviors have input parameters for the types of inputs or sources they use (for example, Camera Input for the Face behavior). These parameters don’t show a value, but do include an Arm for Record button to control whether the behavior uses that input’s data. If you have multiple behaviors that use the same type of input (for example, mouse clicks), you can arm one behavior’s Mouse & Touch Input parameter in one take, then the other’s parameter in a second take.
- Tool tips appear above layer, handle, and input parameters describing the layer and handle names the parameters need and the input sources a behavior uses. For the Face behavior, tool tips also appear above other parameters showing descriptions of the parameters.
- To turn a behavior off or on, click the eyeball next to a behavior’s name for a recorded take in the Timeline panel. This switch allows you to temporarily turn off a behavior without needing to delete it (and lose existing recordings), then later add it back again. Enable or disable behaviors to experiment with different settings.
If you don’t need to adjust or arm a behavior parameter during recording or playback, you can hide the behavior so the Properties panel shows only the controls that you need. Takes can still be recorded for hidden behaviors (for example, if you hide Triggers, then press keys while recording), so this capability is just to declutter the Properties panel. By default, the Handle Fixer and Cycle Layers behaviors are hidden.
To hide a behavior from appearing the Properties panel when a puppet track is selected:
If you use a set of behavior, especially one with specific settings, you can copy them between puppets or groups in a puppet. You can also copy them between puppets or groups in different projects.
The parameter values for the behaviors to copy come from the puppet level (i.e., when the puppet is selected in the Project panel or opened in the Puppet panel). Modified parameter values for the behaviors on a selected puppet track cannot be copied at this time.
Behaviors that were renamed retain their custom names. Also, behaviors are added, even if an instance of the behavior already exists.
This behavior automatically triggers a layer, such as eyelids to blink or lights to flicker. The blinking can occur at regular intervals or with some randomness. You can use this behavior to have a character blink randomly, but it can work with the Face and Triggers behaviors if you also want to control blinks with the webcam or keypresses.
To specify the layer to blink, assign the Left or Right Blink tag to it.
Note: This behavior isn’t applied by default to puppets, so add it first to see its effect on a puppet.
controls the number of milliseconds (1/1000ths of a second) to show the layer before disappearing.
controls the regularity of the blinking frequency, with 0% blinking at regular intervals and higher percentages varying the rate.
This behavior scales the chest of a character as if it were breathing in and out. For example, try animating the Wendigo puppet in the Start workspace, which shows breathing and head turn movements.
To specify the part of a puppet to expand and contract, assign the Breathe tag to a specific handle or to groups to affect its origin. If
This behavior isn’t applied by default to puppets, so add it first to see its effect on a puppet.
Breaths per Minute
controls the rate of breathing (scaling) between minimum and maximum scale amounts.
Max Scale and Min Scale
control the range of scaling around the Breathe handle. If there aren’t handles near the Breathe handle, you might need to reduce this range (for example, from 95% to 110%).
Offset and Direction
control the distance and angle that the Breathe handle slides along as it’s scaling. You can even use these parameters to make a character’s shoulders shrug.
This behavior displays each of the puppet’s layers in order, which can be useful to repeat a pattern like a splash animation or blinking lights. Used with Triggers, the cycle of layers can be triggered by pressing a key.
Apply this behavior to the puppet containing the sequence of layers to cycle.
If you want the cycle to be controlled by pressing a key, select the puppet layer in the Puppet panel, and then enter the key in the Trigger section of the Properties panel.
See Dojo Joe in the Character Animator examples download for a working example that you can modify.
This behavior has the following parameters:
- Start controls when to start displaying the layers — the available options are When Triggered and Immediately.
- Layer Order controls how the layers are used — Top to Bottom (the topmost layer is the first one shown and the bottom layer is the last) or Bottom to Top. If you are assembling the cycle in Photoshop by pasting layers (which get added on top of existing layers), selecting Bottom to Top is easier than reversing the layer order in the artwork file.
- Advance Every controls the duration to display each layer before switching to the next one.
- Cycle controls the number of times to repeat the layers, Continuously or Once.
- Forward and Reverse controls whether the layers play in reverse after reaching the last layer.
- Hold on
LastLayer controls whether the last layer in the cycle stays visible while the trigger is active. It can be used with Forwardand Reverse to, for example, hold on the last layer until the key is released, then play in reverse.
- On Trigger End controls playback when the trigger key is released. Stop Immediately stops the cycle on key release, whereas Let Cycle Finish completes the cycle, including any reverse playback (if enabled). If the cycle was started by a trigger in a swap set, other triggers in the same swap set will be prevented from triggering until the cycle is finished.
- Continue After Pause controls how the cycle is resumed from a Pause-layer (
layerthat has the Cycle Pause tag applied). When Triggered is used if you unpause the layer with the trigger key. When Trigger Released is used if you unpause the layer by holding down and releasing the trigger key.
This behavior allows you to drag a region of a puppet away from the rest of it (for example, to wave its arm). It is applied by default for new puppets, but works only if a location on the puppet is set up for control via mouse or touchscreen.
Assign the Draggable tag to a specific handle or to groups to affect its origin. If imported artwork has a guide or a layer with the word "Draggable" in its name, the Draggable tag is applied to the corresponding handle automatically.
The Dragger tool in the Puppet panel can create Draggable-tagged handles without needing to modify the original artwork file.
Drag near the location in the Scene panel. The nearest Draggable handle location on the puppet moves to match the relative changes in mouse position while dragging.
The Dragger behavior records each Draggable handle that you move as a separate take group, so that multiple performances for a specific handle compose together, and don’t affect performances for other handles that you drag. By grouping Dragger takes by dragged handles, you don’t need to use multiple Dragger behaviors to capture multiple dragged handles. The Timeline panel shows each Dragger takes grouped by handle name as “Handle (handle-name)”.
If you have a touch-enabled display, you can control Draggable handles by touching the display. Multiple handles can be controlled at the same time. The following actions can be performed at the handle location:
- Move: One finger down, then dragged in any direction. This gesture is equivalent to dragging a handle with the mouse.
- Scale: Two fingers down, then dragged apart or together, spreading or pinching the fingers.
- Rotate: Two fingers down, then either one swept in a rotating motion around the other or both swept in a rotating motion and same direction around the midpoint of the two fingers.
- Mouse & Touch Input controls whether the behavior processess mouse actions such as clicks and drags or touchscreen gestures.
- After Move controls what happens after you stop dragging the Draggable handle. Return to Rest (the default) makes the handle settle back to its resting pose, whereas Hold in Place keeps the handle at the spot where you left it.
- Return Duration controls when Return to Rest is used for dragged handles, the handles return smoothly to their resting locations. You can control how long (in seconds) it takes to return to rest with the Return Duration parameter.
Each Draggable handle can have different After Move and Return Duration settings. For example, you can use Hold in Place to pose one draggable hand of a character, switch to Return to Rest and drag the other hand, then change the Release Duration to be longer and then drag a necklace.
This behavior uses the webcam, mouse, arrow keys, or touch-enabled display to control the movement of a puppet’s pupils for finer control and recording of eye gaze. You can dart the puppet's pupils directly to the nine common positions shown below.
Organize and tag layers similar to those for the Face behavior. However, you only use the Head, Left Eye, Right Eye, Left Pupil, Right Pupil, Left Pupil Range, and Right Pupil Range tags.
To pause pupil movements (when the Camera Input parameter is armed), hold down the semicolon (;) key.
Use this capability
to, for example, have the character glance from side to side, holding the stare at each side. When you release the key, the pupils smoothly move to the currently tracked pupil positions. To slow down the transition, increase Smoothness.
100% is the default transformation for each parameter, but you can decrease it to 0% to dampen the transforms or increase above 100% to exaggerate them.
- Mouse & Touch Input controls whether the behavior processes mouse or touch-display drag actions for eye gaze. When armed, clicking or dragging with the mouse or touch will cause the puppet's eye pupil layers to look at the mouse/touch location. The location is defined relative to the center of the scene, which corresponds to the puppet's eyes looking straight ahead. You can dampen the offset by decreasing Mouse & Touch Strength.
Tip: Temporarily disarm behaviors controllable via the mouse (for example, Dragger or Particles) if you want to control eye gaze with the mouse.
- Camera Input controls the movement using the input from a webcam.
- Keyboard input: Control the movements using arrow keys as follows:
- Single arrow
key:points the pupil in that direction.
- Pair of arrow keys: points the pupil in a diagonal direction. For example, up and left, up and right down and left, or down and right.
- Snap eye gaze: Select this option when using camera or mouse to restrict pupil movement so that it darts to one of the nine common positions shown earlier. This controls the responsiveness of the pupil to your own pupil movement. When controlling the eye gaze with the arrow keys, the pupils always dart even if snap eye gaze is not enabled.
- Smoothing controls how much to dampen jittery pupil movements in front of the webcam or due to nonideal lighting conditions causing pupil tracking points to move unexpectedly. The default value does some smoothing, but you might want to decrease it if you prefer to have your puppet’s pupils react instantaneously to quick motions, including rapid eye movements. Note that smoothing has an effect even when snap eye gaze is enabled.
- Camera Strength controls how far pupils can move when controlled using the camera input.
- Mouse & Touch Strength controls how far pupils can be offset when controlled via the mouse or touch.
- Keyboard Strength controls how far the pupils move when controlled using Keyboard.
- Minimum Snap Duration specifies the minimum duration of a snapped eye gaze position before the pupils can move. This parameter can be used to prevent overly jittery eye gaze movements when snapping is enabled.
You can also blend together Eye gaze recordings (takes).
When the Camera Input and Microphone Input buttons in the Camera & Microphone panel are disabled, they override (disarm) the Arm for Record setting for Eye Gaze behavior. This allows you to view, while stopped, the results for a selected puppet with recorded takes for these behaviors without needing to deselect the puppet first.
This behavior uses the face-tracking results from your webcam to control the position, scale, and rotation of named facial features in your puppets.
Organize and tag layers as described in Body features.
A puppet’s eyes can blink in two ways — swapping to different artwork for a closed eye or sliding opposing eyelid layers together. The former gives you more control over the look of a closed eye, especially if it cannot be represented as two adjacent layers, and is easier to set up. However, no scaling occurs for partially closed eyes. The latter gives you continuous movement/scaling of eyelids, but requires more set up.
Assign separate Left Blink and Right Blink layer tags within the Left Eye and Right Eye tagged layers. When the Face behavior detects a closed eye, the Eye artwork is swapped for the Blink ones.
The vertical distance between these handles determines how far to close and open the eyelids (i.e., Character Animator tries to move the top/bottom eyelid layers together to simulate closing of the eye).
As you raise or lower the eyebrow of your puppet, you can also tilt them for more expressiveness. You can tilt the eyebrow inward at the low point to emphasize a
To adjust the intensity of eyebrow tilt when raised or lowered, adjust the Raised Eyebrow Tilt and Lowered Eyebrow Tilt under Face behavior.
Note: Both these options are relative to the drawn orientation of the eyebrow at the rest pose, with 100% or -100% tilting the eyebrows vertically.
The Move Eyebrow Together option under the Face behavior is set as default in Character Animator. This means that the eyebrow movement happens in sync with each other. You can deselect the option for independent control of the eyebrows.
Character Animator captures your facial expressions from your webcam and animates the puppet based on your facial movements.
For some configurations with both internal and external webcams, the internal camera might not be the first (default) camera, or the intended external webcam might not be the next video input, so you might need to switch to the intended webcam. Also, sometimes you might need to reset or retrain the face tracking to the current position and orientation of your face so that the initial appearance of a puppet is as intended.
Choose Switch to Next Camera from the Camera and Microphone panel until the intended webcam is active in the panel, or choose Switch to Default Camera to reset to the first webcam.
If you have multiple webcams or your webcam isn’t the first video source found (for example, if you have a video capture device), cycle through available video sources to choose the intended one.
If the number of video sources changes during or between sessions, you might need to reselect the intended source.
- Increase direct lighting on your face.
- For eyelid and eye gaze direction tracking, move closer to the camera so your face appears bigger in the frame.
Look at the center of the Scene panel, place your face in what you consider a default, rest pose, then click Set Rest Pose in the Camera and Microphone panel or press Cmd/Ctrl+P.
In case they no longer follow your facial features, try moving your head within the field of view of the camera, double-click in the Camera & Microphone panel, or push the red dots towards your face using your hand.
If your facial movements in front of the webcam are uneven or if lighting conditions cause facial tracking points to move unexpectedly, you can try compensating for these movements by having captured camera information smoothened over time. To smooth out facial movements, increase the new Smoothing parameter’s value in the Face behavior. The default value does some smoothing, but you might want to decrease it if you prefer to have your puppet react instantaneously to quick motions, including rapid eye blinks. Mouth replacements are not affected.
To ensure that a punched-in performance (starting a recording during playback) for the Face or Lip Sync behaviors captures the initial frame of the performance, deselect the Pause During Playback option in the Camera & Microphone panel menu.
A character can have multiple Head-tagged
As you press the key trigger to show a head, you can then turn your head to trigger the different views.
See Wendigo in the Character Animator Examples download for a working example that you can modify.
You can produce pose-based movement from the webcam automatically, emphasizing key poses you make with your head and facial features using the Face behavior. Adjust the Pose-to-Pose Movement parameter to control how much to pause the head and face motion. The higher you set the parameter, the more the system will hold the key poses. Setting the parameter lower will cause the key poses to change more frequently.This parameter does not affect lip sync. Use the Minimum Pose Duration parameter to specify the minimum amount of time that a key pose will be held. This parameter only has an effect if the Pose-to-pose movement parameter is greater than 0%.
Tip: Increase Smoothing to 60% or more to ease the transition between key poses. Lower smoothing values can produce jarring jumps between poses.
As you move or rotate your head or make different facial expressions (smile, mouth wide open, etc.), the puppet in the scene should follow.
Audio from the microphone can be used to show different visual representations of your voice, via the Lip Sync behavior (described in Lip Sync: Control a puppet’s mouth with your voice).
In After Effects CC, extract face measurements data from the video, as follows:
a. Import the video footage into a composition.
b. Draw a closed mask around the face, open the Tracker panel, then set Method to Face Tracking (Detailed Features).
c. Click the Analyze buttons to track the mask to the face.
d. Set the current time indicator to the frame representing the rest pose for the face, then click Set Rest Pose.
e. Click Extract & Copy Face Measurements.
The face measurements data on the system clipboard is converted to a Camera Input take on the selected puppet.
Hold down the semicolon (;) key. You can use this capability to, for example, have one character stationary but talking to another character that is moving in the scene. When you release the key, the puppet’s head smoothly moves to the currently tracked head position and orientation. To slow down the transition, increase the Smoothness.
The semicolon key is usable when the Face behavior’s Camera Input is armed.
- Camera Input allows you to control puppet using your face movements on the webcam.
- Smoothing controls smoothing out of jittery or uneven facial movements captured using the webcam. See Smoothening facial movements for more information.
- Head Position Strength, Head Scale Strength, and Head Tilt Strength control the movement, scaling, and z-axis rotation of the head.
- Eyebrow Strength controls the vertical movement of the eyebrows.
- Eyelid Strength controls the movement and scaling of the eyelids. At 0%, no blinking triggered by the webcam occurs, which is useful if you want to control blinking automatically using the Triggers behavior (with eyelids assigned to a key).
- Mouth Strength controls the movement and scaling of the mouth.
This parameter affects only shape-based mouth expressions. Visemes controlled by the Lip Sync behavior are not scaled or affected.
- Parallax Strength controls parallax movement of the eyes, nose, and mouth.
- Blink Eyes Together controls the synchronization of eye blinks (for Blink layers only). Uncheck this option to control each eye blink separately (for example, wink).
When the Camera Input and Microphone Input buttons in the Camera & Microphone panel are disabled, they override (disarm) the Arm for Record setting for Face behavior. This allows you to view, while stopped, the results for a selected puppet with recorded takes for these behaviors without needing to deselect the puppet first.
Assign the Fixed tag to a specific handles or groups to affects its origin. If the imported artwork has a guide or layer with the word “Fixed” in its name, the Fixed tag is applied to the corresponding handle automatically.
The Pin tool in the Puppet panel can create Fixed-tagged handles without needing to modify the original artwork file.
When the puppet moves (for example, via the Face or Dragger behavior), the bendable areas stretch and deform. For example, you can restrict the movement of a character’s torso and lower body by placing a “Fixed” handle at the waist (like a Puppet pin in After Effects or Photoshop), but have the upper part of the body (Head handle) controlled by the Face behavior.
See Headless Horseman.psd or Red Monster.ai in the Character Animator Examples download for working examples that you can modify.
The Stick tool (available in the Puppet panel) allows you to create rigid segments in the warpable mesh of a puppet’s rubber sheet. The area around the segment cannot bend, but the segment can stretch or shrink. For example, you can create sticks (segments) for a character’s upper arm and lower arm, leaving a gap at the elbow so it can bend there.
To create a Stick handle in the Puppet panel, click at the place above a layer to specify one end of the stick, then drag to the other end of the intended stick. A handle with a line through it is created.
This behavior switches between groups, for example, different views like the front, quarter, and side/profile of a character, as you swivel your head left or right.
Specify the controllable views by tagging the layers with at least two of the following:
- Left Profile
- Left Quarter
- Right Quarter
- Right Profile
The number of provided views determines the distance your head needs to swivel. If you provide Left Profile, Front, and Right Profile, Front is triggered when you look straight at the camera, and the profile views triggered when looking to either side. If all five views are provided, you need to turn farther to the sides to trigger the profile views.
- Upward and Downward layers trigger only when you are facing forward.
- This behavior is not applied by default to puppets, so add it first to see its effect on a puppet.
This behavior has the following parameters:
- Camera Input allows you to control puppet using your face movements on the webcam.
- Sensitivity controls how far you have to swivel your head around the y-axis to switch to the quarter and profile layers. Decrease the value if the switching is happening too quickly, or increase it if not fast enough.
As you turn your head, face tracking accuracy for your eyes, nose, and mouth decreases, so you might want to increase Sensitivity to still have good control over facial features, or reduce Eye Gaze Strength for the Face behavior applied to the profile views.
You can create multiple head-tagged group, each with its own set of views (and Head Turner behavior). For example, you might have one set of views by default, but then use a keyboard trigger to switch to a different set of views.
You can trigger a specific layer in a puppet or group using the Layer Picker behavior. Following are the various ways of choosing the layer – by index number, percentage, microphone, keyboard, mouse, or touch-enabled display, or a combination of these. Each layer can even have Cycle Layers applied for more complex triggering of layers.
This behavior isn’t applied by default to puppets. You can add the behavior to the specific group of a puppet whose layers you want to pick from. Only the layers and groups in that puppet are chosen, layers within groups are ignored.
The layer to pick is a combination of multiple input controls. The controls that are available are index number, the percentage of the number of layers available, via loudness captured by the microphone, via arrow keypress, and via dragging via mouse or touch-enabled display. The sum of all layer numbers chosen by these controls determines the actual layer to trigger. You can use one or more controls that are best suited for your needs.
Layer Picker parameters:
• Audio Input: controls the layer picked based on how loud you talk into the microphone. Use the Audio Sensitivity parameter to adjust the value (defined below).
• Keyboard Input: controls the layer picked by pressing the arrow keys. Use Left or Up Arrow to decrease the layer number and Right or Down Arrow to increase the number.
• Mouse & Touch Input: controls the layer picked by dragging horizontally in the Scene panel with your mouse or using touch-enabled display. You can adjust this parameter with the Mouse & Touch Strength parameter (described below).
• Index Offset: controls the layer picked by selecting its layer number (position within the group or puppet). This parameter has a minimum value of 1 (first layer).
• Percentage Offset: controls the layer picked by selecting a percentage of the range of layers. This parameter has a minimum value of -100% and a maximum of 100% so that you can, for example, select a higher-numbered layer with Index Offset, but reduce it with a negative Percentage Offset.
• Audio Sensitivity: controls how much influence the audio volume has on the layer picked. This is used when Audio Input is armed.
• Mouse & Touch Strength: controls how far you need to drag horizontally to change the layer picked. This is used when Mouse & Touch Input is armed.
• Keyboard Advance: controls how you press an arrow key to advance the layer number. On Hold (set by default), allows you to hold down the arrow key to repeated increase or decrease the layer number. On Tap, requires pressing the arrow key for each change in layer number.
• Out of Range: controls how to handle layer numbers that are outside the range of available layers. Limit to First/Last (set by default) restrict to the first and last layer, showing the first layer if the number is negative or the last layer if the number is larger than the last layer’s index. The loop repeats the range of layers.
This behavior produces lip-synced animation if the original artwork contains visemes (visual representations of your mouth as you make different sounds) and you talk into the microphone. You can also process audio in the scene to generate lip sync data for a puppet.
The Lip Sync behavior has a Keyboard Input parameter that, when armed, allows you to display specific visemes by pressing the first letter of the viseme layer’s name for example, A for the Ah viseme, D for D, W for W-Oo, etc.). You do not have to add keyboard triggers manually to those layer names.
A viseme is a generic facial image that can be used to indicate a particular sound. A viseme is the visual equivalent of a phoneme or unit of sound in spoken language. Visemes and phonemes do not necessarily share a one-to-one correspondence. Often several phonemes correspond to a single viseme, as several phonemes look the same on the face.
Within Character Animator, there are three shapes that are determined by the shape of your mouth in the webcam. These only show up if no audio is detected (no one is talking). Neutral is the most common to see and should be your default “rest” mouth.
The other 11 mouth shapes, called Visemes, are determined by audio. Visemes are visualizations of key mouth positions when saying common phonetic sounds. Character Animator listens for 60+ specific sounds and translates them into visemes.
If you name and structure your Mouth group like this, Character Animator automatically recognizes and tags these mouth shapes upon import.
• Lock the top jaw. Keeping the top row of teeth in a consistent place helps things look smoother.
• Cycle Layers behavior can be added to a mouth group to add a few frames of transitional animation when that sound is picked up. The mouth opening to Ah or W-Oo is a common application.
• You can add additional mouth shapes (sad, yell, etc.) into your Mouth group and show them through keyboard triggers.
• These are examples of a frontal view, but quarter and profile views can follow the same general guidelines.
Try making a “boooooo” sound to see if the mouth reliably stays on the “W-Oo” viseme, and a “la-la-la-la-la” sound to see if the viseme with the tongue appears (assuming your artwork included it).
A scene named after the puppet is created, and the scene is opened in the Scene panel.
You can change the audio hardware references. To change them, select Edit > Preferences > Audio Hardware.
As you talk, the audio signal is analyzed and a matching viseme for your mouth is displayed. When no sound can be detected or the microphone is disabled, control falls back to the Face behavior (if present) analyzing the video signal (your mouth expressions captured by the webcam) to possibly trigger the Smile or Surprised mouth shapes.
Choose Timeline > Compute Lip Sync from Scene Audio.
The Compute Lip Sync from Scene Audio command analyzes the scene audio and creates a Lip Sync take only where the scene audio overlaps the selected puppet track items. Muted tracks are not computed. Visemes are automatically generated for the audio and are displayed below the Lip Sync take bar.
Computing Lip Sync from scene audio may take time depending on the duration of your audio.
When the Camera Input and Microphone Input buttons in the Camera & Microphone panel are disabled, they override (disarm) the Arm for Record setting for Lip Sync behavior. This allows you to view, while stopped, the results for a selected puppet with recorded takes for these behaviors without needing to deselect the puppet first.
Visemes are represented as adjacent bars below the Audio Input track bar and the gaps between these bars are moments of silence in the audio. Each bar represents a separate viseme. Each viseme bar
has the viseme name displayed on it for easy recognition. Zoom in to view these names.
To zoom into the timeline, do any of the following steps:
- If you want to timeline zoom in/out around mouse, place pointer over the specific point in the timeline and hold down Alt-vertical-wheel(Win) or Option-vertical-wheel(Mac).
- If you want to timeline forward/backward in time, hold down Shift-vertical wheel (Mac/Win) or Horizontal wheel (Mac/Win).
If the Timeline panel is zoomed too far out to see the Lip Sync viseme bars, they change to a diagonal-lined pattern to convey that viseme information can be edited if you zoom in.
To select visemes or silences, do any of the following steps:
- To select a viseme, click the left-edge of the viseme bar. The edge turns white when you select it.
- To select a silence, click its left edge. The cursor changes to a horizontal arrow when you hover over the edges.
- To select multiple visemes, press Shift+click(Win) or Cmd+click(Mac).
- To retain the existing selected visemes and silences as you drag and select them, hold down Shift (Win) or Cmd/Ctrl (Mac) before starting to drag.
To adjust the timing of a viseme or silence, do any of the following steps:
- Drag the viseme bar horizontally from the edges.
- Drag the edge between viseme bars or silences horizontally.
The left edge of the viseme or silence moves earlier or later in time. You can drag the left edge of the viseme bars or silences across other visemes to replace them.
To delete visemes or silences,
- Select Edit > Delete or press Delete.
When you delete a
There are multiple ways in which you can edit
- Left arrow or right arrow: Selects the previous or next
visemeor span of silence in time.
- Up arrow or down arrow: Moves the selected
visemeearlier or later by one frame in time.
- The first letter of the
visemename, for example, A for Ah, E for Ee, W for W-Oo: replaces the selected visemeassociated with that letter.
- Forward slash key (/): Splits the selected
Note that Deleting a selected
Right-click the viseme to be replaced and choose a new viseme from the context menu. To replace a viseme with a silence, choose Silence from the context menu.
The letters in parentheses are sounds. For example, use D viseme for sounds like n, th, and g.
Tip: Though you can play back or scrub through time, or deselect the Puppet track item to view the result, you can also disable microphone input in the Camera & Microphone panel to make changes to a viseme and see the results on the character immediately.
- To add a viseme where there is silence, right-click in the silence and choose a new viseme from the context menu. This adds the viseme and removes the silence.
- To add a viseme anywhere in time, click the viseme and hold down Alt(Win) or Option(Mac) and right-click to choose a new viseme from the context menu.
Depending on placement, the inserted viseme leaves silence after it. To make the inserted viseme fill the rest of the span of silence, press Alt/Option-left click to open the Visemes popup menu.
To split a
- Hold down Alt(Win) or Option(Mac) as the cursor changes to a razor and click the area you want to split. The razor indicates the area to be split.
- Select and drag the
playheadto the area to be split and select Edit > Split, or Cmd+Shift+D (Mac) or Ctrl+Shift+D (Win).
You can cut or copy and paste lip sync takes from one puppet or project and use it in another by following these steps:
- To retain the original
visemesand copy the lip sync take, choose Edit > Copy (Cmd/ctrl+C). To cut the original viseme, choose Edit > Cut (Cmd/Ctrl+X).
- Select the puppet track item in the timeline where you want to paste the copied
viseme. This could be in the current project or another one. Position the play head where you want to paste the lip sync. To paste, choose Edit > Paste (Cmd/Ctrl+V).
Note: If you copy multiple Lip Sync takes, they are pasted in order of their selection.
You can take Character Animator
The Motion Trigger behavior switches between groups based on the direction the parent puppet is moving. For example, as a character moves to the right, switch to a profile view of the character running with motion trails behind it. The movement of the character can either be direct (you dragging it across the scene) or indirect (the puppet is attached to a hand on an arm that is swinging because the torso is moving).
To specify the motion views, attach any of the following tags:
- At Rest
- Moving Left
- Moving Right
- Moving Up
- Moving Down
- In Transition
- These directions are based on your view of the scene in the Scene panel (i.e., a puppet moving to the left side of the scene would trigger the Moving Left layer, if present). Diagonal movement chooses the dominant direction, but falls back to the other direction if a layer for the dominant one does not exist.
- This behavior is not applied by default to puppets, so add it first to see its effect on a puppet.
Move the parent puppet whose group has this behavior applied. Movement can come from any other behavior (Face, Dangle, Dragger, Transform, etc.).
This behavior has the following parameters:
Speed Threshold controls the threshold that the parent puppet must move in pixels per frame before a layer is triggered. Speeds below this threshold
Minimum Duration controls the minimum number of frames to trigger a layer before triggering a different layer, which can reduce flicker/jittery switches between layers when moving at a slower speed.
This behavior moves the lower part of the puppet’s mouth as you open and close your mouth in front of the webcam or talk into the microphone. This can be a simpler way of making a puppet talk without needing to specify separate artwork for the different mouth shapes and visemes used by the Face, and Lip Sync behaviors. If Face, and Nutcracker Jaw behaviors are both on a puppet, the Face behavior can still control the rest of the face, but Nutcracker Jaw controls just the mouth.
This behavior isn’t applied by default to puppets, so add it first to see its effect on a puppet.
If imported artwork has a guide or layer with the word “Jaw” in the name, the Jaw tag is applied to the corresponding handle automatically.
See Nutcracker Annie.psd in the Character Animator Examples download for a working example that you can modify.
- Camera Input allows you to control puppet using your face movements on the webcam.
- Audio Input controls the audio input from the microphone.
- Camera Flappiness controls the offset of the lower jaw based on your mouth in front of the webcam. 100% is the default maximum offset from the rest position, but you can change the value to affect this maximum offset.
- Audio Flappiness controls the offset of the lower jaw based on the amplitude of your voice in the microphone. 100% is the default maximum offset from the rest position, but you can change the value to affect this maximum offset.
- Movement controls how the lower jaw moves. You can choose from the following options in the Movement parameter:
- Position is for vertical movements.
- Rotation Clockwise is for the jaw to pivot clockwise. This is appropriate for right-profile puppets.
- Rotation Counterclockwise is for the jaw to pivot counterclockwise. This is appropriate for left-profile puppets.
When the Camera Input and Microphone Input buttons in the Camera & Microphone panel are disabled, they override (disarm) the Arm for Record setting for Nutcracker Jaw. This allows you to view, while stopped, the results for a selected puppet with recorded takes for these behaviors without needing to deselect the puppet first.
Particles behavior creates multiple instances of a puppet and treats them as particles within a simulation (for example, spewing as if fired from a cannon, or falling like snowflakes). This behavior isn’t applied by default to puppets. Add it first to see its effect on a puppet. You can also apply it to a group of puppets to have the group emitted as particles from the parent puppet.
The Physics behavior must be applied at or above the group that has Particles behavior applied. This allows Particles to use the settings for the physics system, and particles to move. If you are opening an older project, or a puppet with only Particles behavior applied, add the Physics behavior to use the settings.
Particles applied to a group render above all groups of a puppet. See Snowflakes.psd in the Character Animator Examples download for a working example that you can modify.
- Particle Mode controls the type of particle motion to generate — Snow makes particles fall from the top of the scene, Point and Shoot fires particles from the puppet’s position toward the clicked location, and Cannon fires particles in a specific direction.
For the Snow mode, you might want to set Emitter Opacity to 0% or position the puppet off the edge of the scene (via the Transform behavior) so all you see are the falling particles.
- Mouse & Touch Input controls whether the behavior processess mouse actions such as clicks and drags.
- Keyboard Input controls whether the behavior processess keyboard presses.
- Continuous Mode (for Point and Shoot and Cannon particle modes) controls if particles fire automatically, without clicking the mouse.
In Point and Shoot mode, the pointer must be above the Scene panel for particles to fire.
- Particles Per Second
controlsthe birth rate.
- Direction (for Cannon mode) controls the angle that particles fire from the center of the puppet.
- Velocity controls the rate of fire.
- Spread controls the width of the area from which particles are birthed. For Snow mode, if you want particles to fall across the full width of the scene, set this value to 100%.
- Randomness controls the variability in
directionthat particles fire. Use in combination with Spread for different looks.
- Lifespan controls particle duration (in seconds), from birth to death.
- Honor Emitter Transforms controls whether emitted particles are affected by the scale, skew, and rotation of the emitter (for example, water droplets that spray from an elephant’s trunk). When unchecked, emitted particles are upright but is still affected by the emitter’s position and the Particle Scale parameter.
- Particle Scale controls the size of emitted particles, relative to the normal size of the puppet.
- Emitter Opacity controls the visibility of the original puppet from which particles are emitted. Set to 0% so that all you see are the emitted particles. The Transform behavior’s Opacity can additionally affect emitter opacity.
- Particle Opacity controls the visibility of emitted particles.
- Fade Particle Opacity controls whether the visibility of emitted particles reduces over their lifespan.
Note: Unless you want particles to move along an arc, set Direction (of emitted particles) and Gravity Direction in
Gravity Strength, gravity Direction, Bounce Off Scene Sides, Bounciness, Wind Strength, and Drag parameters are now a part of Physics parameters. Squash and Strech parameter is no longer available.
The Particles behavior can emit particles that are
Physics allows layers in a puppet to collide and bounce against each other and the sides of the scene. For example, a character can walk through a pile of leaves, with the feet pushing them out of the way. You can even have a structure of nested groups act as a chain or produce ragdoll animation. You can even have a structure of nested groups act as a chain or produce ragdoll animation. This behavior lets a puppet warp (hang and sway) as if it was pulled and pushed from some point on it. Handles on the puppet act as particles that get dynamically simulated, causing the artwork on them to move and deform. For example, the long hair, earrings, or wings on a character that sway or oscillate like a spring as the character moves around the scene.Note: Collisions occur only between independent groups of a single puppet, and between puppets that are in the same scene.
You can apply Physics at multiple group levels for finer control of stiffness, wind, and gravity settings.
Note: The Physics tag is ignored if you apply it to a puppet’s origin handle, either manually in the Puppet panel or if the artwork had a layer with “dangle” in its name.
Note: Collisions occur only between independent groups of a single puppet, and between puppets that are in the same scene.
Assign the Physics tag to the handles that you want to control by physics. The puppet dangles from the other handles, such as Origin or the ones with the Fixed tag. If the imported artwork has a guide with the word "Dangle" in its name, the handles created from those guides get the Physics tag automatically. Layers with the word "Dangle" get the Physics tag applied to their origin handles. However, the Physics tag applied to any origin handle is ignored.
The Dangle tool in the Puppet panel can create Dangle-tagged handles without needing to modify the original artwork file.
Make sure that some parent puppets can be controlled by the Dragger, Face, or Transform behavior so that the dangling puppets can react to the motion.
Tip: Use Hinge attach style to allow the puppet to pivot smoothly around the attachment point.
Dangling still uses Dangle-tagged handles and the Dangle tool, but to operate collisions, tag the
layers as either Collide or Dynamic. These handle and layer tags are in the Tags > Physics section of the Properties panel.
Follow these steps to make an independent group dangle:
- Add one or more handles tagged as Dangle to the group.
- Reposition the group’s origin to where the dangling movement should be anchored.
- To make independent groups collide or move around the scene, apply the Collide layer tag if you want the group to be
collidablewith other Collide-tagged layers. The layer does not move unless you also use the Dynamic layer tag, but allows other collidableobjects to bounce off it.
- Apply the Dynamic layer tag if you want the group to move around in the simulation. You might want to also apply the Collide tag so that the dynamic layer doesn’t ignore other layers that you want it to bounce off of.
If the Collide or Dynamic tag is applied to a layer or non-independent group, it uses the mesh associated with the nearest parent-independent group in the puppet structure. Also, when setting up a hierarchy of groups for ragdoll movement, change the Attach Style for the groups to Hinge to make each group pivot as expected. When the Dynamic tag is assigned to a group that contains Dangle handles, the group will not move rigidly but can still dangle.
Move the puppet on which the dangled artwork is attached. For example, move your face (if the Face behavior is controlling the puppet) or drag in the Scene panel (if the Dragger behavior is controlling the puppet).
The Physics behavior has gravity controls that affect both dangling groups and colliding layers, and parameters specific to dangling and colliding actions. You can control them using the following parameters:
Gravity Strength: controls the amount of force or pull of gravity.
Bounce Off Scene Sides controls if emitted particles bounce when hitting any side of the scene.
Bounciness controls how much particles bounce. If lowered to 0% and Bounce Off Scene Sides is checked, they’ll slide along the edges.
Wind Strength controls the influence of the simulated wind on the Dangle handles.
Wind Variation controls the randomness of wind direction and strength. Increase this value to make the wind feel more alive.
- Spring Stiffness: controls the rigidity of the regions controlled by the Dangle handles. You can set the Spring Stiffness value up to 200%. Some puppets require lowering the default value (for example, down to 20%) to get more elasticity.
- Wind Strength: controls the influence of the simulated wind on the Dangle handles.
- Wind Direction: controls the orientation of the wind, pushing Dangle-tagged handles in this direction
- Wind Variation: controls the randomness of wind direction and strength. Increase this value to make the wind feel more alive.
- Start: controls when the collision simulation begins, either When Triggered by a parent group that has a trigger is run, or Immediately upon the start time for the puppet in the scene. Tip: If you want a simulation to settle before showing it (for example, a pile of leaves settling on the ground before being pushed away by another object), set Start to When Triggered, and then press the trigger key when you want to see the settled animation.
- Shape: controls how precise collisions should be. Fast approximates the shape of layers to be either a rectangle or circle (so detailed layers might not appear to touch when colliding
),but is faster to process the simulation. Contour uses the outline of the layer for more accurate collisions,but can take longer to process the simulation.
Weight :controls the density of layers that collide with each other.
- Friction: controls how much colliding layers decelerate when sliding against each other.
- Bounciness: controls how much colliding layers bounce off each other.
- Bounce Off Scene Sides: controls whether colliding layers can bounce off the edges of the scene.
- Return Strength: controls how fast layers that are dynamic layers
returnsto their original position after being pushed away by another colliding layer. Set to 0 or low values to prevent or slow them from returning.
Make a portion of your puppet dangle independently
This behavior allows you to adjust the anchor point, position, and rotation of a puppet in the scene. It is useful when you want to assemble multiple puppets in a scene or move a puppet across it.
- Anchor Point X and Anchor Point Y control the location of the puppet’s center. Rotation, scaling, and skewing occur around this point.
- Position X and Position Y control the location of the puppet relative to its original rest location.
- Scale controls the overall size of the puppet, but you can also adjust horizontal and vertical scaling independently with the Scale X and Scale Y parameters.
- Rotation controls z-axis rotation.
- Opacity controls the visibility of the puppet.
- Group Opacity controls how the opacity of layers behaves. By default, Group Opacity is not applied to the layers. You set the opacity for each layer individually, which can cause overlapping layers to appear darker. Check Group Opacity if you want uniform opacity across the puppet.
Character Animator CC uses a triggering system that replaces the older Keyboard
Beta 6 and earlier projects and puppet files that use the obsolete Keyboard Triggers behavior will continue to work, but if you would like to use the new system with older puppets and projects please see See Converting Obsolete Keyboard Triggers to Triggers.
The Walk behavior allows a puppet to walk across the scene by controlling the legs, arms, and body of the puppet. The behavior simulates some common walking styles such as - strut and prance.
You can create an animated walk cycle with the behavior. A walk cycle is a series of poses played on loop, which creates an illusion of a walking puppet. Walk cycles can convey different moods and emotions to enhance your animation. For example, long bouncy steps represent a happy walk.
This behavior assumes that the character is two-legged and two-armed, and is viewed in profile. But, it is possible to apply it to additional legs.
Upon applying the behavior on the puppet, the legs move through a looping series of poses to complete a walk cycle. The feet are planted at ground level and the arms swing in opposite direction of the legs while walking. Walk behavior can also adapt to long and short legs for a smooth walk motion.
Tip: The Start workspace includes a template puppet - Walkbot, which has been created using the Walk behavior. You can use this template puppet to view the behavior
For more information on different workspaces, see Workspaces in Character Animator.
When you open existing projects that use Walk behavior, you may notice that the position of the puppet moves vertically. This is done to make the different walk styles keep the feet on the floor as you switch between them.
Setup a profile view puppet with the legs and arms as independent groups to avoid unwanted overlapping. All the independent parts to the parent puppet (profile view puppet) staples automatically to prevent limbs from detaching. For best results, use Hinge or Weld attach styles. Next, you can add sticks to the legs and arms to keep them straight and prevent bending.
This behavior is not applied by default to puppets, so add it first to see its effect.
The puppet is made of a number of independent parts attached with the parent puppet. In the Puppet panel, identify the following locations on each leg and on the body of the puppet.
- Left Elbow
- Left Wrist
- Right Elbow
- Right Wrist
- Left Knee
- Left Ankle
- Left Heel
- Left Toe
- Right Knee
- Right Ankle
- Right Heel
- Right Toe
Use the correspondingly named handle tags in Properties panel to identify the locations. To find the handle tags, follow these steps:
You can create a basic walk cycle with a minimum set of tags. Before you create tags, you need to create handles, which the tags are associated with.
To create a handle, follow these steps:
For foot movement, use either Ankle or Heel tag and for leg movement, use either Waist or Hip tag. If you have a left-facing character walking to the left by default, also add the Knee tag. For the leg and arm handles, be sure to set the same ones on both left-facing and right-facing puppets, for
If you have separate left-facing and right-facing views, make sure to tag them correctly.
To add tags, follow these steps:
- In Puppet panel, select the left-facing view.
- In Properties panel, click the triangle icon next to Tags.
- Under Views section, you can view the layer tags.
- Select the Left Profile layer tag for left-facing puppet and the Right Profile layer tag for right-facing puppet.
After you tag the puppets, switch to the Scene panel to see the puppets respond to the changes. With the Start parameter set to Left & Right Arrow keys, when you press Right-Arrow key, right-facing puppet is displayed, and when you press Left-Arrow key, left-facing puppet is displayed. If you tag only one profile puppet or no profile puppet, and press the opposite arrow key, the puppet walks backwards. For example, if you tag only the left-facing puppet, the puppet walks backwards when you press the Right-Arrow key.
By default, the puppet legs walk in place to help you preview the walk cycle of the puppet and make changes. You can make the puppet move using the Walk parameters
To make the puppet move, do any of the following:
- When Start is set to Immediately, increase the Body Speed value. Set to 100% to avoid the feet from appearing to slide along the ground. When the puppet moves out of view, click the Refresh button in the Scene panel to restart its walk.
- When Start is set to With Left & Right Arrow Keys, press Left-Arrow or Right-Arrow key. Pressing either key ramps up the walk from the character’s rest pose to a walking motion, and releasing the key returns the character to rest; this is equivalent to changing the Strength parameter when pressing and releasing the key.
Tip: You can apply a draggable handle to the puppet’s top-level origin handle if you need to reposition it
within the scene.
Walk behavior has the following parameters:
- Start controls when the puppet moves. The available options are — Immediately (when it first appears), and With Left & Right Arrow Keys (when those keys are pressed).
- Start/Stop Easing controls how long (in seconds) it takes to ramp up to full speed after pressing the arrow keys and ramp down from full speed after releasing them. Use a short duration to simulate a character building up energy into the walk cycle and slowing down to a stop.
- Style controls the type of leg movement. The available options are — Walk (basic walk), Slump (upper body slouched forward), Strut (stiff gait with high knee movement), Prance (similar to a strut but with funkier upper body movement), Sneak (stealthy motion), Run (faster/longer stride), and Head Bang (rocking backwards and forwards).
- Stride Length controls the horizontal spacing of the leg and arm movement. For best results, set values between 80% and 120%.
- Step Speed controls how quickly the legs move through the walk cycle. At 100%, the character makes two steps per second. At 0%, the legs of the character do not move. Negative speed produces
- Step Phase controls where the legs start in the walk cycle. When Step Speed is set to 0%, adjust Step Phase manually to control the stepping.
- Body Speed controls how quickly the body moves. At 100%, the feet of the character feet do not appear to slide against the ground. At 0%, the body does not move, which is useful if you want to simulate a character running in place or moving above a scrolling background. If your puppet walks off the
scene,clickthe Refresh button at the bottom of the Scene panel to reset the position of the puppet.
- Arm Swing controls the range that the arms swing relative to the range of movement for the legs.
- Arm Angle controls the orientation of the arms at the rest pose with 0° (angle control pointing up) placing the arms straight down and 90° (angle control pointing to the right) placing the arms straight out in front of the character.
- Elbow Bend controls the amount of bending of the arms at the elbows.
- Strength controls the overall influence of the walk cycle relative to the drawn, rest pose of the character, with 0% the body is at the rest pose (with no movement), and at 100% the body moves through the normal walk cycle.
- Toe Bend controls the amount of roll or bend of the foot at the toes as the heel rises from the ground. You can adjust the parameter between 0% to 100%. 0% means no bend while 100% means maximum bend. The parameter is set to 50% by default. The bend of the foot is controlled by the toe-tagged handle. You might want to position the handle slightly away from the tip of the toe. Note: the placement of handles and sticks should be consistent between the right feet and left feet for best results.
Tip: Add multiple Walk behaviors with different walk styles, speeds, and phase to produce more complex motion.
This behavior, when applied to Illustrator artwork, wiggles the artwork’s paths automatically to give a little life to the puppet. It does not affect Photoshop artwork, or Illustrator artwork whose skin has the “Render As Vector” option disabled.
This behavior isn’t applied by default to puppets, so add it first to see its effect on a puppet. However, for very complex Illustrator documents with numerous paths, performance can be slow with Wiggler applied.