Multithreading (for AnimateDiff)

[Idmon]

I've been using this plugin for a while now and love it.

Now the last couple of weeks I've been tinkering with AnimateDiff. I noticed that the plugin works with AnimateDiff but the issue is that it sequentally swaps each frame, which can take a while with 16+ frames. For the 30XX - 40XX users you could easily swap 10 - 15 frames simultaneously just like how other Inswapper / Roop-based projects do it such as FaceFusion.

They usually implement it with a Python Threading library where the user can provide how many threads they wanna distribute the tasks to.

Multi-threading would offer massive speed gains not only for AnimateDiff but also if one would like to create batches of images that all needed to be swapped. I'm not familiar with A1111 inner-workings and if this solution is possible within that pipeline.

[Gourieff]

Good idea, I will investigate how to implement it

[Idmon]

import threading

# Class - creates a thread and returns value when joined
class ThreadWithReturnValue(Thread):

    def __init__(self, group=None, target=None, name=None,
                 args=(), kwargs={}, Verbose=None):
        Thread.__init__(self, group, target, name, args, kwargs)
        self._return = None

    def run(self):
        if self._target is not None:
            self._return = self._target(*self._args,
                                        **self._kwargs)

    def join(self, timeout=None):
        Thread.join(self, timeout)
        return self._return

# Separate swap function for threading usage
def swap_face_thread(image, args):
    result, output, swapped = swap_face(
        args.source,
        image,
        source_faces_index=args.source_faces_index,
        faces_index=args.faces_index,
        model=args.model,
        enhancement_options=args.enhancement_options,
        gender_source=args.gender_source,
        gender_target=args.gender_target,
        source_hash_check=args.source_hash_check,
        target_hash_check=args.target_hash_check,
    )

    return result, output, swapped
    
# Proccess batch of frames
def process_batch(images, p, gpuThreads):
    all_threads = []
    for i, image in enumerate(images):
        while len(all_threads) >= int(gpuThreads):
            # order dependant
            result = all_threads.pop(0).join()
            p.init_images[i] = result

        # launch new thread
        thread = ThreadWithReturnValue(target=swap_face_thread, args=(image, args))
        all_threads.append(thread)
        thread.start()

Here's some pseudocode that might give an idea how to implement.
Haven't tested it yet as I've never dove into A1111 extensions, just copied code from an older project.
See which threading library fits within the context of a custom A1111 extension. If post-images are executed to run script one by one, then threading might not be ideal

[g0kuvonlange]

Was this being implemented? Would be an amazing feature tbh!

Right now the faceswapping is indeed a little bottleneck to go over it frame-by-frame