FaceFusion is an excellent tool for swapping faces, colorizing, or enhancing the quality of photos and videos. It's completely free and can be run on your own machine (requires sufficient computing power).
The settings and notes in this guide are based on FaceFusion v3.4.1.
- FaceFusion documentation:
- The official GitHub repository.
- The official documentation.
- The error codes reference. Note that when errors occur, nothing appears in the terminal. To see the error code, run
echo "Exit code: $?"immediately after.
- Vast.ai documentation:
- My custom versions: facefusion-3.4.1 and facefusion-3.3.2. These versions disable filters and add several new features.
- Face detection parameters
- Face Swapper Parameters
- Face enhancer parameters
- Face masker parameters
- Avoid using reference photos with glasses when swapping faces.
- Use 4-6 reference photos with different angles and expressions for best results.
- Instead of splitting videos for testing, use the "Trim Frame" option (processes only the selected segment) and "Preview Frame" (displays the generated result at each frame).
- Models are stored in
.assets/models/
- Temporary files (uploaded references and targets) are stored in
/tmp/facefusion/on Linux systems.
- Output fps of 35 is sufficient—higher values aren't necessary.
- To preserve emotional expressions, use the
expression_restorerprocessor.
Modify
facefusion.ini. Below are just the settings that need values in that file.Note: Configuration may vary depending on your machine's power. The settings above are optimized for systems with powerful GPUs.
With this custom
facefusion.ini (some parameters are prefilled), we can use following command to run the facefusion without using its gradio UI,Using FaceFusion with Vast.ai
Based on excellent work by Singulainthony, this section adapts his approach with a few of my own notes.
I've modified the code with my own improvements. The GitHub repository for version 3.4.1 is available here. In this custom version, I completely disabled the filters, added
--share-gradio (useful on remote servers like Vast.ai to access the app via a public URL, but unnecessary locally).Remember to stop or remove unused instances to avoid significant charges!
- Use this template (for version 3.3.2, no customization) or my template for version 3.4.1 (with customization).
- Open the instance application, create a new terminal, and run the following commands (you can review the gist file to see what happens behind the scenes):
- After the installation completes, close the current terminal, create a new one, and run:
- You should see output similar to:
⚠️ The free Gradio URL remains active for only 7 days. Read this official documentation to learn more about this public URL and how to create your own custom URL.
When working with a remote machine like Vast.AI, there are several drawbacks:
- The connection can be extremely slow through the Gradio tunnel.
- We can't monitor the upload process, and all uploaded files are only stored temporarily.
For these reasons, it's more efficient to upload files directly from your local machine to the Vast.ai instance and then run the FaceFusion CLI.
First, we need to connect to the instance via SSH.
- Check your local machine's public key
- On the Vast dashboard, go to Keys and paste your public key. This key will be added to all new instances you create, but it won't work with your current instance.
- If the previous step doesn't work, manually copy your public key to the current instance's
authorized_keysfile:
- SSH to the instance. First, check the instance's IP and port in the "Instances" page by clicking the key icon at the bottom bar of the instance.
- To upload a file from your local machine to the instance:
Tips for Working with Vast.ai
- Performance
- With RTX 5090, 2 processors enabled and the configurations described in the previous section, processing speed reaches about 5 frames/s for HD videos or up to 20 frames/s for medium-resolution videos.
- With RTX 4080 using the same settings, processing speed is about 2.5 frames/s.
- With a local Mac M4 Chip, processing speed is about 1.5 frames/s.
- For heavy files, avoid using the download button in the FaceFusion UI as it downloads via the Gradio public URL, which is very slow. Instead, open Jupyter notebook, navigate to the download folder, and download from there—it's much faster!
- To empty the trash when you've removed files:
- FaceFusion's cache in Linux is located at
/tmp/facefusion/. Each project has its own folder; userm -rf folder_nameto remove specific folders.
- Generated videos are typically very large (GBs). Use the following command to reduce file size while maintaining quality:
- Let's use QuickTime Player or QP (on Mac).
- Quickly split a clip at the current frame (
cmd+Y) as many times as needed, then usedelto remove any unsuccessful splits.
- You can also use QP's built-in trim feature (
cmd+T).
- For merging different video files, use QP → Edit → Add clip to the end. QP produces smaller file sizes compared to
ffmpegor iMovie, which create much larger files.
- Use the Left/Right arrow keys to move forward or backward frame by frame.
- Avoid changing the playback speed in QuickTime Player while editing videos, as this will result in poor frame quality.
All commands in this section require ffmpeg to be installed. If you encounter a "no file found" error, first run
setopt NULL_GLOB.- Split a video into segments of 200 ms each
- Often, FaceFusion cannot process videos that have been modified by certain applications (like QuickTime Player). You can use the command below to "fix" these videos. This process also significantly reduces file size—for example, a 1GB
.movfile can be compressed to a 50MB.mp4while maintaining the same resolution:)
If you want to create an alias to quickly use the command, put below in
.bashrc or .zshrc- Convert
.m3u8(streaming video) to a.mp4
- Resize all images to max 768px in the current folder (replace current ones)
- Compress all images in the current folder
- Compress and resize video to 480p
Compress and resize all videos in the current folder to 480p