IV2V & TV2V - transfer body movements from reference video with IC-Union-Control lora
IV2V & TV2V - transfer body movements from reference video with IC-Union-Control lora
Works best for less action packed motions, and close ups.
Despite the example video above, for fast tik-tok routines or dance/fight scenes, it might not be as good (but will also do a Dev model workflow with more steps that might do this part better).
Works great for slower body motion transfer though, and close ups.
In the workflow you can choose between LTX audio, custom audio input, or using the audio from the input video.
And there is one workflow with SD Pose and another with DW Pose.
SD Pose will soon be natively supported in ComfyUI, will update the workflow to reflect that as soon as the PR is added ;-)
The more close the input image body position and body size is to the first frame of the input video the better the results.
Will make a workflow that uses Flux Klein to automagically re-pose the input image to that of the first video input frame, but that will come later.
But you can easily do so yourself for now with Qwen Image Edit or Flux Klein
(images used in the example video above are random picks from PromptHero website. Credit to the image creators)
Feel free to play around with it ;-)
https://huggingface.co/RuneXX/LTX-2.3-Workflows/tree/main/Control-reference