Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LocalBlend doesn't work #6

Open
CONSTANT1386 opened this issue Nov 3, 2024 · 1 comment
Open

LocalBlend doesn't work #6

CONSTANT1386 opened this issue Nov 3, 2024 · 1 comment

Comments

@CONSTANT1386
Copy link

Dear RoyiRa:
I try to test whether the LocalBlend can work so I use the run code as following:
`import os
import torch
from prompt_to_prompt_pipeline import Prompt2PromptPipeline
from processors import LocalBlend
def custom_repr(self):
return f'{{Tensor:{tuple(self.shape)}}} {original_repr(self)}'

if name == "main":
original_repr = torch.Tensor.repr
torch.Tensor.repr = custom_repr

device = torch.device('cuda:0') if torch.cuda.is_available() else torch.device('cpu')
if device.type == "cuda":
    pipe = Prompt2PromptPipeline.from_pretrained("SG161222/RealVisXL_V4.0",
                                                torch_dtype=torch.float16, use_safetensors=True).to(device)
else:
    pipe = Prompt2PromptPipeline.from_pretrained("SG161222/RealVisXL_V4.0",
                                                torch_dtype=torch.float32, use_safetensors=False).to(device)
seed = 25
g_cpu = torch.Generator().manual_seed(seed)

# prompts = [
#     "On the park bench, a man in a red tracksuit is listening to music, and next to him, a lady in a green dress is teeding pigeons.",
#            "On the park bench, a man is listening to music, and next to him, a lady is teeding pigeons.",
#            ]
# prompts = ["a pink bear riding a bicycle on the beach", "a pink dragon riding a bicycle on the beach"]
# cross_attention_kwargs = {"edit_type": "replace",
#                           "n_self_replace": 0.4,
#                           "n_cross_replace": {"default_": 1.0, "dragon": 0.4},
#                           }

# prompts = [
#     "In the bustling shopping center, a fashionable young man is trying on shoes, and opposite him, an elderly man with glasses is reading a new book.",
#     "In the bustling shopping center, a man is trying on shoes, and opposite him, a man is reading a book.", 
#            ]
prompts = [
    # "In the quiet library, a man, and next to him, a lady is writing",
    "In the quiet library, a man is studying a science book, and next to him, a lady is writing",
    "In the quiet library, a man with glasses is is studying a science book, and next to him, a lady is writing",
    ]
        #    "In the quiet library, a man with glasses is studying a science book, and next to him, a lady in a blue sweater is writing her novel."]
# prompts = ["A golden retriever carrying a Frisbee, wearing a  suit and a  cowboy hat",
#            "A golden retriever carrying a  Frisbee, wearing a blue suit and a  cowboy hat"]
# lb = LocalBlend(prompts, ("lady","lady"),tokenizer=pipe.tokenizer, device=device)
cross_attention_kwargs = {"edit_type": "refine",
                          "n_self_replace": 0.4,
                          "n_cross_replace": {"default_": 1.0},
                          "local_blend_words": ("man","man")
                          }

image = pipe(prompts, cross_attention_kwargs=cross_attention_kwargs, generator=g_cpu)
# Create a directory to save images if it doesn't exist
output_dir = "generated_images"
os.makedirs(output_dir, exist_ok=True)

# Save each image to the output directory
for i, img in enumerate(image['images']):
    img.save(os.path.join(output_dir, f"P2P_REAL_{i+1}.jpg"))
    print(f"Saved image: {os.path.join(output_dir, f'P2P_REAL_{i+1}.jpg')}")

# print(f"Num images: {len(image['images'])}")
# from IPython.display import display

# for img in image['images']:
#     display(img)

`

But I found the output image have no difference No matter I added the line: "local_blend_words": ("man","man")

Can you help me?
Thank you!

@CONSTANT1386
Copy link
Author

Please allow me to add the output image, and they separately are the output with blended word "man" and w/o it. I try to add the glasses.
image
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant