Generate video with consistent/constant background

[KaustubhPatange]

Hi @s9roll7, first of all, great work on your project. I am able to generate a consistent animation with it, however, it seems the background is not constant when the character moves a lot.

For eg: I've used this input video frames

laughing_guy.mp4

config:

• openpose: scale: 1.0, start_guidance: 0.0, end_guidance: 1.0
• depth: 0.75, start_guidance: 0.55, end_guidance: 0.85
• prompt

  "head_prompt": "",
  "prompt_map": {
    "0":  "ghibli style, a 8nxJMm girl in yellow t-shirt standing in front of a chalkboard, which is covered with various equations and formulas" # 8nxJMm is my character code from my custom model
  },
  "tail_prompt": "",

the command is animatediff generate

The output is:

rajveer_video.mp4

---

It seems like the character is consistent but the background is not. How can I make it consistent or keep a consistent plain background suppose white background? What could I change or add in my config to make this happen?

Thanks for help

[tnil25]

Try this:
https://civitai.com/models/143148/green-screen

[yukselyusuf]

What arguments do you add when creating the video's config?

animatediff stylize create-config YOUR_SRC_MOVIE_FILE.mp4 -- ??
and share ur config

[KaustubhPatange]

Hi @tnil25, thanks for responding I added the lora as you said and here are the results. I've generated first 10 frames of the video.

Original 10 frames video, constant seed and prompt for all.

Original Video	Lora: 1.5 weight	Lora: 1.5 weight, Tile upscale (with green png): scale: 0.2, start and end guidance: 0.0, 0.2
5036508108_output.mp4	5036508109_output.mp4	5036508110_output.mp4

It does make the skin color off and add a purple-ish filter on an image. This is my config: https://pastebin.com/rt9gXphA

[KaustubhPatange]

Hey @yukselyusuf, I've not used stylize command but here is my original config that generated this output https://pastebin.com/Ai6Mt09R

[s9roll7]

00_341774366206100_no_humans_simple_background_white_background_.mp4

prompt.json

controlnet_openpose and qr_code_monster_v2

[tnil25]

Hi @tnil25, thanks for responding I added the lora as you said and here are the results. I've generated first 10 frames of the video.

Original 10 frames video, constant seed and prompt for all.

Original Video Lora: 1.5 weight Lora: 1.5 weight, Tile upscale (with green png): scale: 0.2, start and end guidance: 0.0, 0.2
5036508108_output.mp4
5036508109_output.mp4
5036508110_output.mp4
It does make the skin color off and add a purple-ish filter on an image. This is my config: https://pastebin.com/rt9gXphA

I see, I've personally never tried that lora myself but I thought it might be an option. Getting consistent backgrounds with animatediff is a challenge. It will mostly come down to better prompting and controlnet use.

[KaustubhPatange]

Hey @s9roll7, thanks for prompt.json, will be trying your approach. Thanks again.

[KaustubhPatange]

Hey @s9roll7, would it be possible for you to share the stylize folder that you've created? I just wanted to know what ip adapter images you used for the root and region as well as condition images for QR code monsiter v2 as I am not able to generate the output you are getting.

Thanks again

[s9roll7]

Hey @s9roll7, would it be possible for you to share the stylize folder that you've created? I just wanted to know what ip adapter images you used for the root and region as well as condition images for QR code monsiter v2 as I am not able to generate the output you are getting.

Thanks again

A better way has been implemented, so you'd better use that.
#189