Removing an unwanted object or area from a video can be a time-consuming and complex process. With the Content-Aware Fill feature, you can remove any unwanted objects such as mics, poles, and people from your video with a few simple steps. Powered by Adobe Sensei, this feature is temporally aware, so it automatically removes a selected area and analyzes frames over time to synthesize new pixels from other frames. Simply by drawing mask around an area, After Effects can instantly replace it with new image detail from other frames.
The tool gives you the option to help get the fill to blend seamlessly with the rest of the image. The Content-Aware Fill panel contains various options to help you remove unwanted objects and fill transparent areas.
- Fill Target: This is the preview of the area that Content-Aware Fill analyzes. The transparent area is outlined in pink.
- Alpha Expansion: Use this to increase the size of the area to fill. Content-Aware Fill does not require precise masking, and may offer better results when the area includes pixels outside the object being removed.
- Fill Method: Choose the type of fill to render:
- Object: Removes an object from the footage. It fills the transparent area by taking pixels from the current and surrounding frames. It removes an object from the footage. It estimates the motion of the scene behind the object and uses this to find appropriate color values. For best results, use this to replace moving objects, like a car on a road.
- Surface: Replaces the surface of an object. It works similar to Object as it takes pixels from surrounding frames but uses the motion estimated in the comp under the transparent area. For best results, use this for static and flat surfaces, like a stain on a shirt, or a sign on a building.
- Edge Blend: Blends surrounding edge pixels. It fills the transparent area by sampling pixels at the edges of the transparent area and blending them together, and renders fast. For best results, use this to replace static objects on surfaces that lack texture, like text on paper.
- Range: Choose whether to render the fill layer for only the work area or the entire duration of the composition. Setting it to work area restricts Content-Aware fill from pulling in content outside of the work area.
- Create Reference Frame: Creates a single-frame fill layer frame and opens it in Photoshop. Use reference frames to help Content-Aware Fill learn what the fill layer should look like. For example, if you have a video with complex background and Content-Aware Fill is unable to give you desired results, use the tools in Photoshop such as clone, and patch to create a better result on the reference frame. Once done, generate a new fill layer. Content-Aware Fill transfers the pixels from the reference frame to new frames in the fill layer. For some shots, you can create multiple reference frames at frames where the lighting or camera angle changes. Note that you can also use any-other process to create single-frame layers to guide content aware fill. It also respects content in those layers as a guide. This is a shortcut to do that.
- Generate Fill Layer: Creates a new fill layer. Analysis and rendering progress are displayed at the bottom of the panel. Content-Aware Fill prioritizes analyzing and rendering the frame under the Current-Time Indicator (CTI). While the fill is being rendered, you can move the CTI to a different frame to prioritize that frame, which can help you decide whether the results look correct before the entire fill layer is generated.
The Content-Aware Fill panel includes multiple features to enable precise removal of unwanted objects from your vidoe. But all these features may not be useful in every situation.
Use any available method to create transparent areas in your composition. For example, draw a mask around an object or area of your composition that you want to replace, and set it to Subtract mode. To learn how to create masks, see Create masks.
Use the Pen tool and draw a mask around the boat. For best results, draw the mask close to the object but leave some space around it for After Effects to analyze the frames better. You can use the following steps to draw a simple mask:
1. In the Timeline panel, select the layer and select the Pen tool.
2. Click key points along the shape that you want to mask. To close the mask, click again on the start point.
3. Adjust the mask point by using the Selection tool. Click and drag points to adjust them.
4. If you want the mask to have refined curves, click around each mask point to adjust the curve.
5. Once the masking is done, select Subtract.
6. Since it is a moving footage, track the mask so it moves with the boat. Right-click the mask layer, and select Track Mask. To learn more, see Mask Tracking.
The panel displays multiple options. For a detailed list of the panel options, see The Content Aware Fill panel. Use the following settings:
- Set the Fill Method to Object.
- Set the Range to Entire duration.
This fill method is best used on footage with moving camera as it replaces the object entirely with what is behind it. It fills the transparent area by taking pixels from the current and surrounding frames, and outputs a seamless looking frame.
There can be footage where Content-Aware Fill alone may be unable to achieve the desired results. For example, a footage with water or areas where there is varied light and texture. In such situations, you can take advantage of the Adobe Photoshop tools such as Clone Stamp to finetune your footage.
Make sure you have Photoshop installed on your machine.
In the above example, we remove the girl from the footage where the background has varied light and texture.
To make a Reference Frame, after you create a mask, select a frame where the object is largest in your shot. In this example, use the first frame. Click the Create Reference Frame button to send that frame (with its mask) to Photoshop. A reference frame is a user-painted single-still clean plate to teach the Content-Aware Fill algorithm what pixels to fill in your masked area.
Paint in the fill in any way that works for you. For this example, use the Clone Stamp tool to get finer results. After you make the edits, save the footage. Reopen After Effects, and the Content Aware Fill panel places your reference frame on a layer below your original shot. This layer updates with your saved paint job from Photoshop, and fills the object mask for that frame only with your Photoshop Reference Frame.
Another tool you can use is the Photoshop Content Aware Fill.
To learn about the different Photoshop tools, see Remove objects from your photos with Content-Aware Fill, and Retouch and repair photos.
To open the Content-Aware Fill Settings dialog, click the menu icon (three bars) in the title bar of the Content-Aware Fill panel, and choose Content-Aware Fill Settings.
- Output Depth: The color depth, in bits per channel (8-, 16-, or 32-bpc), used to write the fill layer sequence files. By default Output Depth inherits the project's color depth.
- Output Location: Choose where After Effects After Effects saves the fill layer sequence files. By default the files are saved to a folder named Fills in the same folder as the project file.
- Path Type: Project Relative saves the files to a location relative to the project file. Absolute specifies the full path to the folder.
- Path: The path to the folder. For Absolute, this is the full path to the folder. For Project Relative, this specifies the path relative to the project file. A single dot (.) in the path represents the folder where the project file is located. Use two dots (..) to go up or down one level.
- Create Photoshop Reference Frame: Saves reference frames as a PSD file, and opens it in Photoshop. Enabled by default. If disabled, saves the reference frame as a PNG (8- or 16-bpc) or EXR (32-bpc) file and opens it in the system default application for those file types.
- Create Photoshop Sequence for Output: Saves fill layer sequences as PSD files. Disabled by default. If disabled, fill layer sequences are saved as PNG (8- or 16-bpc) or EXR (32-bpc) files.
- Auto-manage Unused Fill Footage: When a new fill layer is generated, you will be asked if you want to delete fill layer footage not being used in a composition. Disabled by default.
To manually delete fill layer footage that is not being used by a composition, click on the menu icon (three bars) in the title bar of the Content-Aware Fill panel, and choose Delete Unused Fill Footage.
- You can select an area in a footage using using any method that creates transparency, including:
- Eraser tool
- Keying method such as Keylight method
- If you have 4K footage in an HD comp, use footage that is already scaled down via transcoding for better performance.
- Remove more complicated items in sections. For example, you can remove a person and their shadow separately more effectively than removing both at once.
- Smaller masks take less time to process over larger ones.
- The feature might give different results for a complex moving texture like water or areas where there is varied light and texture.
- If the mask is feathered, Content-Aware Fill blends in the filled region with the original content based on this alpha value (semi transparency). This is very useful when you use the panel, it causes a noticeable boundary between the filled region and the original content
- Generate Fill Layer renders sequences of images that are placed in the Timeline. These files may take up considerable space on the hard drives, depending on the kind of footage, and duration of your sequence.
- In certain cases, Content-Aware fill alone may be unable to remove object with precision. To improve results, create a reference frame and take it into Photoshop, and use the tools there to fine-tune the results.
- At the time of shooting a footage, consider the background and the foreground of your composition to reduce editing time during post-production.