[2.1.831] Fooocus Describe (in line with Midjourney Describe)

[lllyasviel]

Since 2.1.831, Fooocus added a Describe tool to convert image to prompts.

You can simply click "Describe this Image into Prompt" to use it

Also works for anime tags

We are different! Fooocus is aimed at functionalities with highest quality and almost all tools in Fooocus are highly unique.

You may be interested in the difference between CLIP Interrogator (in other open-source software) and Fooocus Describe:

1. Fooocus Describe is less likely to yield repeated failure cases than CLIP Interrogator, like “in a hat, in a hat, in a hat, in a hat, in a hat, in a hat”.
2. Fooocus Describe will not randomly add artist names and website names into your prompt. (Most CLIP Interrogator implementation will add nearly random artist names and noisy website names)
3. Fooocus Describe is similar to Midjourney Describe that will give you multiple possible descriptions. To use it, just click "Describe this Image into Prompt" multiple times. (Most CLIP Interrogator implementation give single result.)
4. Only "Photograph" Describe supports giving multiple possible descriptions.
5. Besides the above 1234, Fooocus Describe is also based on BLIP like CLIP Interrogator, but the model choice is based on computation power of most devices.
6. The Anime/Art Describe is based on WD-Tagger-V2 best model. Note that if your are previously using DeepDanboru or WD-Tagger-V1’s smaller or older model, you can expect this implementation to give you better results. If you previously use WD-Tagger-V2 in other UI, then the result should be same. (But, are you sure that you are previously using v2 model? If you use Fooocus, you always get best model with no research.)

[GfxFan]

Love Fooocus so much, Thanks for this easy and powerful software!

[tbuyle]

Love it !

I made a quick video to demo it.

describe.mp4

[fewjative]

I ran that image through MJ and it seemed like MJ version has more creativity that would allow for a better chance of recreation? I use this feature a lot in MJ so definitely happy to see it coming here too now.

MJ results ( I did crop it and it's a snip from the video FYI ):

• an antique cat wearing a top hat, in the style of animated gifs, realist detail, richly colored --ar 108:73
• a cat in a top hat and flowery pants, in the style of animated gifs, jim burns, realist detail, frederick morgan, avocadopunk, hispanicore --ar 108:73
• top hat with flowers cat ornament, in the style of colorful animation stills, james mcintosh patrick, animated gifs, kurt wenner, 16th century, western-style portraits, realist detail --ar 108:73
• a fancy hat top cat christmas tree stickers, in the style of western-style portraits, trompe-l'œil illusionistic detail, detailed realism, animated gifs, 1860–1969, close-up, manticore --ar 108:73

[lllyasviel]

as discussed above, you can click "describe this image" multiple times to get multiple results. some will be short and concise while some will be long and creative.

this is different from other SD software but more similar to MJ

[GfxFan]

Thank you so much! Truly yours -------- ข้อความดั้งเดิม --------จาก: Thomas ***@***.***> วัน​ที่: 13/12/23 15:44 (GMT+07:00) ถึง: lllyasviel/Fooocus ***@***.***> Cc: GfxFan ***@***.***>, Comment ***@***.***> เรื่อง: Re: [lllyasviel/Fooocus] [2.1.831] Fooocus Describe (in line with Midjourney Describe) (Discussion #1363) Love it ! I made a quick video to demo it. Uploading describe.mp4… —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>

[xhoxye]

A small problem, the model used in the photo description, this information will pop up every time, and it will take a long time.

[xujianjian2004]

请事先下载一个854MB大小的文件：model_base_caption_capfilt_large.pth，放到\SimpleSDXL2_win\SimpleSDXL\presets目录下

[Leroy-X]

It seems an error was reported

[lllyasviel]

looks like internet problem. may try later

[Leroy-X]

looks like internet problem. may try later

Hi, I am in China, and affected by the GFW, so many international networks are inaccessible.
Does this need to be connected to the Internet every time? If not, I think I can download it offline through VPN.
Thanks.

[lllyasviel]

no. only first time it needs internet to download models.

[Leroy-X]

Thank you, working now.

[FlyingHermes]

waiting

[nej-dot]

is there any way i can use this for a whole folder? With writing the prompt in .txt files and all that stuff. That would be amazing!

[lllyasviel]

hi I think kohya-ss has similar codes that can be taken a look but sould not be exactly same

[EyeRis10]

I get an error "ModuleNotFoundError: No module named 'timm'.

[lllyasviel]

this means you are probably not using official method to launch fooocus.

[reccea]

i have the same problem. what is wrong?

[ikusic]

Wow... this is really great stuff. I can see Fooocus becoming the first txt2img tool that even people who have never heard of diffusion can use. Keep up the great work

[caffeine-cn]

I encountered a wired output. not sure if there is really a "desert dessert"

[lllyasviel]

you may have hit some data point in some dataset like https://www.google.com/imgres?imgurl=https://media-cdn.tripadvisor.com/media/photo-s/1c/8f/b2/9c/fish-cone-with-mochi.jpg&tbnid=d7Wc5KT9NtZ4eM&vet=1&imgrefurl=https://www.tripadvisor.com/Restaurant_Review-g31350-d21166740-Reviews-Dessert_in_Desert-Scottsdale_Arizona.html&docid=-sW2afL02UVmHM&w=338&h=450&source=sh/x/im/m1/2

[togenio]

I love your web interface, I would really like to see the same convenient web interface for creating animations like animediff

[ccwq]

i like it

[lchauvin]

Nice work.

However, is it possible to get more descriptive prompts ? I put an image and generated dozens of times, and I always get a prompt between 5 to 10 words (which make it too vague to generate similar images).
I believe CLIP Interrogator 2 has a parameter you can change to get more lengthy output.

Thank you.

[kienerj]

as discussed above, you can click "describe this image" multiple times to get multiple results. some will be short and concise while some will be long and creative.

Also in regards to Ichauvins comment above I could see 2 helpful improvements:

• a way to control the description like from a general to a more precise description
• automation: on button click generate n number of descriptions and m images for each description

[funkysugar1983]

can somebody tell me what the error means?

[poor7]

Look in the logs.

[funkysugar1983]

[Kikyo-chan]

老兄检查检查你的网络可能是GFW，初次需要到https://huggingface.co下载模型

[marceloteixeira2]

Hi guys,

It doesn't work for me. It shows:

[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 3.0
[Parameters] Seed = 7262327794084325470
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 60 - 30
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Traceback (most recent call last):
File "C:\Fooocus_win64_2-1-831\Fooocus\modules\async_worker.py", line 823, in worker
handler(task)
File "C:\Fooocus_win64_2-1-831\python_embeded\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Fooocus_win64_2-1-831\python_embeded\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Fooocus_win64_2-1-831\Fooocus\modules\async_worker.py", line 373, in handler
pipeline.refresh_everything(refiner_model_name=refiner_model_name, base_model_name=base_model_name,
File "C:\Fooocus_win64_2-1-831\python_embeded\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Fooocus_win64_2-1-831\python_embeded\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Fooocus_win64_2-1-831\Fooocus\modules\default_pipeline.py", line 236, in refresh_everything
assert_model_integrity()
File "C:\Fooocus_win64_2-1-831\python_embeded\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Fooocus_win64_2-1-831\python_embeded\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Fooocus_win64_2-1-831\Fooocus\modules\default_pipeline.py", line 53, in assert_model_integrity
raise NotImplementedError(error_message)
NotImplementedError: You have selected base model other than SDXL. This is not supported yet.
Total time: 0.08 seconds

Anybody knows what is happening?

[kienerj]

What model are you using? the error message says it right there:

NotImplementedError: You have selected base model other than SDXL. This is not supported yet.

Seem like you are trying to use a SD 1.5 model?

[marceloteixeira2]

Hmmm I'm going to check it out, thanks for pointing that out.

[kalle07]

it give much more bette blip models ;)

models--Salesforce--blip-image-captioning-base
models--Salesforce--blip2-opt-2.7b-coco

[dislive]

Thank you. And Where it must be put?

[aidec]

Styles will be reset to default options after using describe
Is there any way to make it only generate a description without changing other settings?

[xzendor7]

Is there a way to add how one wants the input image to be described. When I use this feature in ChatGPT 4o, I can tell the AI how I want it to describe the image; using a particular writing style and tone.

The result is a much more descriptive and detailed which can then be easily used to generate other more precise images.

[davehornik]

Is there any way how can we change CLIP model to users choice?
In my case.. to be more nsfw friendly, or is this limitation somewhere else?
I did read thru documentation and ethics, but didn't find any clue how..

[IPv6]

Afaik you can not change it. CLIP model is used in training, changing base CLIP will require retrain everything (checkpoints, LORAs, etc)