Third Party Integration

ComfyUI

The comfyui-sync-lipsync-node is a custom node for ComfyUI that generates lip-synced videos seamlessly within its workflow, making audio-video synchronization simple and efficient.

1

Clone and Set Up ComfyUI

First, clone and set up the ComfyUI repository. Choose one of the following methods:

$# Clone ComfyUI
>git clone https://github.com/comfyanonymous/ComfyUI.git
>cd ComfyUI
>
># Set up your virtual environment and dependencies as needed
3

Install the Sync Lipsync Node

There are 3 ways to set up the lipsync node:

$# Inside the custom_nodes folder
>git clone https://github.com/synchronicity-labs/sync-comfyui.git
>cd sync-comfyui/
>pip install -r requirements.txt

If you didn’t follow Method A or B, you can install through the ComfyUI interface:

  1. First, run ComfyUI: python main.py
  2. Open the ComfyUI GUI in your browser
  3. Click Manager in the top right corner.
  4. Select Install via Git URL
  5. Paste the URL: https://github.com/synchronicity-labs/sync-comfyui.git
  6. It will prompt you to restart the terminal - do so and follow the new link
4

Start ComfyUI

After completing Method A or B, go back to the main ComfyUI directory and start the application. If you used Method C (Manager GUI), ComfyUI is already running during installation—restart it if prompted:

$cd /path/to/ComfyUI/
>python main.py

A link will appear in the terminal — open it in your browser to access the ComfyUI GUI.

5

Configure the Workflow

Choose one of the following methods to set up your ComfyUI workflow:

Set up the workflow by manually connecting nodes in the ComfyUI interface:

  1. Locate the Sync Nodes: On the left sidebar, go to the Nodes tab and search for Sync. You will find these exact nodes:

    • sync.so lipsync – video input: For video input configuration
    • sync.so lipsync – audio/tts input: For audio or text-to-speech input configuration
    • sync.so lipsync – api key: For Sync API key configuration
    • sync.so lipsync – generate: For processing the lip-sync generation
    • sync.so lipsync – output: For handling the final output

  2. Connect the Nodes: Connect the first three nodes to the generate node as inputs, and then connect to the output node:

    • Connect video input to generate
    • Connect audio/tts input to generate
    • Connect api key to generate
    • Connect generate to output
  1. Download the workflow JSON from this link
  2. Drag and drop the JSON file onto the ComfyUI canvas

Configure Inputs:

  • Video and Audio: You can provide either a URL or a local path as input. For local files, they should be in the ComfyUI repository that you are using.
  • Loading Local Files: If you want to load a video or audio from local storage, connect the input audio/video nodes to LoadVideo/Audio nodes.
  • API key: Enter your Sync API key in the api key node. You can get your API key from the API Keys page.
6

Generate Output

Click the Run button to start the lip-sync generation process. The system will process your video and audio inputs to create synchronized output.

Output Handling:

  • The lipsynced video will be automatically saved along with the job_id in a JSON file for your reference.
  • Manual Saving: The video can also be manually saved by specifying a desired path and video name in the output node.
  • UI Download: You can download the generated video directly from the UI by clicking on the 3 dots on the generated video and then clicking on download.

Error Monitoring and Job Tracking:

  • Any errors will show up in the terminal where ComfyUI is running.
  • For every generation, the job_id and status will be automatically saved in /path/to/ComfyUI/output/ as a JSON file.
  • This allows you to track the progress and status of your lip-sync generations.

Features

  • Seamless Integration: Works directly within ComfyUI’s node-based interface
  • Flexible Input: Supports both URL and local file paths for video and audio
  • Five-Node Workflow: Separate nodes for video input, audio input, API key, generation process, and output for clear workflow separation
  • Interactive Node Information: Hover over nodes to get additional information about their functionality
  • Multiple Output Options: Automatic saving, manual path specification, and direct UI download
  • Comprehensive Monitoring: Real-time error display in terminal and automatic job tracking with status files
  • Local File Support: Easy integration with LoadVideo/Audio nodes for local file processing

Requirements

  • ComfyUI installation
  • Python virtual environment
  • Sync API key
  • Video and audio files (URL or local paths)

Troubleshooting

Installation Issues

  • Dependencies: Ensure all dependencies are properly installed using the requirements.txt file

  • ComfyUI Manager Security Error: If you encounter the error “This action is not allowed with this security level configuration” when using “Install via Git URL”, follow these steps:

    Error Message:

    ERROR: To use this feature, you must either set '--listen' to a local IP and set the security level to 'normal-' or lower, or set the security level to 'middle' or 'weak'. Please contact the administrator.
    Reference: https://github.com/ltdrdata/ComfyUI-Manager#security-

    Solution:

    1. Navigate to: ~/comfy/ComfyUI/user/default/ComfyUI-Manager/config.ini
    2. Change security_level to normal-
    3. Change network_mode to local
    4. Run ComfyUI with: python main.py --listen 127.0.0.1

Runtime Issues

  • API Key Problems: Verify your API key is valid and you’ve added a valid payment method
  • Input File Issues: Check that input files are accessible and in supported formats
  • Local File Access: For local files, confirm they are within the ComfyUI repository directory
  • Error Monitoring: Check the terminal where ComfyUI is running for real-time error messages
  • Job Status: Review the JSON files in /path/to/ComfyUI/output/ to check job status and troubleshoot failed generations
  • Node Connections: Ensure all nodes are properly connected (video input, audio input, and API key nodes to the generation process node, and generation process node to the output node)