Flux Image-To-Image pipeline

[dsocek]

FLUX Image-to-Image Pipeline

List of Contributions in this PR

• FLUX.1 image-to-image pipeline Gaudi added
• Added support for FLUX.1 inference in examples/stable-diffusion/image_to_image_generation.py sample
• Updated documentation with FLUX.1 image-to-image examples/command-lines
• Added FLUX.1 image-to-image CI tests

Quality Check

Quality of output image shows good guidance from the input image and accompanying prompt.

Input image:

Input prompt:

"cat wizard, gandalf, lord of the rings, detailed, fantasy, cute, adorable, Pixar, Disney, 8k"

Simple inference script:

import torch
from diffusers.utils import load_image
from optimum.habana.diffusers import GaudiFluxImg2ImgPipeline

pipe = GaudiFluxImg2ImgPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev",
    torch_dtype=torch.bfloat16,
    use_habana=True,
    use_hpu_graphs=True,
    gaudi_config="Habana/stable-diffusion",
)

url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png"
init_image = load_image(url).resize((1024, 1024))

prompt = "cat wizard, gandalf, lord of the rings, detailed, fantasy, cute, adorable, Pixar, Disney, 8k"

image = pipe(
    prompt=prompt, image=init_image, num_inference_steps=30, strength=0.9, guidance_scale=3.5
).images[0]
image.save("flux_img2img.png")

Output image:

Performance

Performance matches the Gaudi FLUX.1 text-to-image pipeline performance.

Tests

The added CI tests for Gaudi FLUX.1 image-to-image pipeline PASSED:

python -m pytest tests/test_diffusers.py -v -s -k "GaudiFluxImg2ImgPipelineTester"
...
============ 2 passed, 162 deselected, 1 warning in 11.50s =============